The Org Package

The Org Package

[David Masterson]

I still need more understanding of the Emacs packaging system.
Something doesn't seem right and I'm sure I'm missing some key in
understanding how its supposed to work.  What I see right now seems like
something doesn't match up -- particularly with the Org package:

1. Most modern Emacs have Org pre-installed.
2. Unfortunately, that Org is not up-to-date (24.3 has 7.9.3f).
3. Therefore, installing the latest Org package seems natural.
4. However, this does not uninstall the built-in Org package.
5. Packages are not initialized until after .emacs is run.
6. Therefore, any of the latest variables are not defined yet.
7. Therefore, setting a hook may not do what you think.
8. The documentation for Org suggests hooks (etc.) to set.
9. I've run into times when org-version was still 7.9.3f.

Do you see where I'm heading?  Does anyone else run into this problem?
Or do most people ignore the Org package and install the latest from
GitHub in a more manual process (a la Bernt Hansen's paper)?  Do we need
more concrete documentation on setting up the Org Package?

-- 
David Masterson

Re: The Org Package

[John Hendy]

On Fri, Apr 11, 2014 at 9:32 PM, David Masterson <dsmasterson@gmail.com> wrote:
> I still need more understanding of the Emacs packaging system.
> Something doesn't seem right and I'm sure I'm missing some key in
> understanding how its supposed to work.  What I see right now seems like
> something doesn't match up -- particularly with the Org package:
>
> 1. Most modern Emacs have Org pre-installed.
> 2. Unfortunately, that Org is not up-to-date (24.3 has 7.9.3f).
> 3. Therefore, installing the latest Org package seems natural.
> 4. However, this does not uninstall the built-in Org package.
> 5. Packages are not initialized until after .emacs is run.
> 6. Therefore, any of the latest variables are not defined yet.
> 7. Therefore, setting a hook may not do what you think.
> 8. The documentation for Org suggests hooks (etc.) to set.
> 9. I've run into times when org-version was still 7.9.3f.
>
> Do you see where I'm heading?  Does anyone else run into this problem?
> Or do most people ignore the Org package and install the latest from
> GitHub in a more manual process (a la Bernt Hansen's paper)?  Do we need
> more concrete documentation on setting up the Org Package?

I learned emacs /for/ Org-mode, so keep that in mind as I'm pretty
ignorant of emacs in genera. I *think* that packages for emacs are
sort of a recent thing, and since I was already using git, I've never
bothered to switch my setup. I find git ridiculously easy and have
never had a reason to do anything else.

Once ever:

#+begin_src sh
mkdir ~/.elisp/
cd ~/.elisp
git clone [orgmode git path] org.git  # I like adding the .git so I
know it's a git repo
cd org.git
make clean && make
#+end_src


Then put in config:

#+begin_src .emacs
(add-to-list 'load-path "~/.elisp/org.git/lisp/")
(add-to-list 'load-path "~/.elisp/org.git/contrib/lisp")
#+end_src

That's it. Anywhere between once a week and once every three months, I do:

#+begin_src sh
cd ~/.elisp/org.git
git pull
make clean && make
#+end_src

Is this more difficult than packages? What is the advantage of ELPA
vs. this? I could see it if I had a lot of these sorts of things, but
I really just use Org + ESS, so I'm not constantly
updating/installing/removing emacs add-ons other than those two.


John

>
> --
> David Masterson
>
>

Re: The Org Package

[David Masterson]

John Hendy writes:

> On Fri, Apr 11, 2014 at 9:32 PM, David Masterson wrote:

>> I still need more understanding of the Emacs packaging system.
>> Something doesn't seem right and I'm sure I'm missing some key in
>> understanding how its supposed to work.  What I see right now seems like
>> something doesn't match up -- particularly with the Org package:
>>
>> 1. Most modern Emacs have Org pre-installed.
>> 2. Unfortunately, that Org is not up-to-date (24.3 has 7.9.3f).
>> 3. Therefore, installing the latest Org package seems natural.
>> 4. However, this does not uninstall the built-in Org package.
>> 5. Packages are not initialized until after .emacs is run.
>> 6. Therefore, any of the latest variables are not defined yet.
>> 7. Therefore, setting a hook may not do what you think.
>> 8. The documentation for Org suggests hooks (etc.) to set.
>> 9. I've run into times when org-version was still 7.9.3f.
>>
>> Do you see where I'm heading?  Does anyone else run into this
>> problem?  Or do most people ignore the Org package and install the
>> latest from GitHub in a more manual process (a la Bernt Hansen's
>> paper)?  Do we need more concrete documentation on setting up the Org
>> Package?

> I learned emacs /for/ Org-mode, so keep that in mind as I'm pretty
> ignorant of emacs in genera. I *think* that packages for emacs are
> sort of a recent thing, and since I was already using git, I've never
> bothered to switch my setup. I find git ridiculously easy and have
> never had a reason to do anything else.

Basically, I'm used to your style here, but let it not be said that I
didn't, at least, try to be more modern.  ;-)

> Once ever:
>
> #+begin_src sh
> mkdir ~/.elisp/
> cd ~/.elisp
> git clone [orgmode git path] org.git  # I like adding the .git so I
> know it's a git repo
> cd org.git
> make clean && make
> #+end_src

How do you find the value of "[orgmode git path]"?

> Then put in config:
>
> #+begin_src .emacs
> (add-to-list 'load-path "~/.elisp/org.git/lisp/")
> (add-to-list 'load-path "~/.elisp/org.git/contrib/lisp")
> #+end_src

Where does "/lisp" and "/contrib/lisp" come from?  What do they contain?

> That's it. Anywhere between once a week and once every three months, I
> do:
>
> #+begin_src sh
> cd ~/.elisp/org.git
> git pull
> make clean && make
> #+end_src

Do you run into any problems where something is picked up out of the
"built-in" Org because of overlapping requires?

> Is this more difficult than packages? What is the advantage of ELPA
> vs. this? I could see it if I had a lot of these sorts of things, but
> I really just use Org + ESS, so I'm not constantly
> updating/installing/removing emacs add-ons other than those two.

It's not more difficult and you could probably easily expand on this for
any number of packages by simply expanding your last shell to walk thru
all the interesting packages and pull the latest version in.

As far as I can see, ELPA's plus is the GUI for pulling in new versions
of packages, but its minus is that it moves the package setup *somewhat*
out of .emacs and into the after-init area.  For instance, I'm
installing org-toodledo which hasn't been packagized yet, so, when you
(require 'org-toodledo) in your .emacs, the org package is also pulled
in.  However, if you've updated the org package via ELPA, the wrong
version of org will be pulled in *unless* you (package-initialize) in
your .emacs.  If you follow the basic rules, things work, but the rules
aren't explicitly documented, so it can be confusing.

-- 
David Masterson

Re: The Org Package

[Jacek Generowicz]

David Masterson <dsmasterson@gmail.com> writes:

> [ELPA's] minus is that it moves the package setup *somewhat* out of
> .emacs and into the after-init area.

This point is recognized and addressed by el-get:

  https://github.com/dimitri/el-get

Re: The Org Package

[John Hendy]

On Fri, Apr 11, 2014 at 11:10 PM, David Masterson <dsmasterson@gmail.com> wrote:
> John Hendy writes:
>
>> On Fri, Apr 11, 2014 at 9:32 PM, David Masterson wrote:
>
>>> I still need more understanding of the Emacs packaging system.
>>> Something doesn't seem right and I'm sure I'm missing some key in
>>> understanding how its supposed to work.  What I see right now seems like
>>> something doesn't match up -- particularly with the Org package:
>>>
>>> 1. Most modern Emacs have Org pre-installed.
>>> 2. Unfortunately, that Org is not up-to-date (24.3 has 7.9.3f).
>>> 3. Therefore, installing the latest Org package seems natural.
>>> 4. However, this does not uninstall the built-in Org package.
>>> 5. Packages are not initialized until after .emacs is run.
>>> 6. Therefore, any of the latest variables are not defined yet.
>>> 7. Therefore, setting a hook may not do what you think.
>>> 8. The documentation for Org suggests hooks (etc.) to set.
>>> 9. I've run into times when org-version was still 7.9.3f.
>>>
>>> Do you see where I'm heading?  Does anyone else run into this
>>> problem?  Or do most people ignore the Org package and install the
>>> latest from GitHub in a more manual process (a la Bernt Hansen's
>>> paper)?  Do we need more concrete documentation on setting up the Org
>>> Package?
>
>> I learned emacs /for/ Org-mode, so keep that in mind as I'm pretty
>> ignorant of emacs in genera. I *think* that packages for emacs are
>> sort of a recent thing, and since I was already using git, I've never
>> bothered to switch my setup. I find git ridiculously easy and have
>> never had a reason to do anything else.
>
> Basically, I'm used to your style here, but let it not be said that I
> didn't, at least, try to be more modern.  ;-)
>
>> Once ever:
>>
>> #+begin_src sh
>> mkdir ~/.elisp/
>> cd ~/.elisp
>> git clone [orgmode git path] org.git  # I like adding the .git so I
>> know it's a git repo
>> cd org.git
>> make clean && make
>> #+end_src
>
> How do you find the value of "[orgmode git path]"?

Sorry, I was lazy and didn't take time to look it up. Org has
documented this very well. It's right on the main page:
- http://orgmode.org/

That is: ~$ git clone git://orgmode.org/org-mode.git

It's also in the instructions just a bit down on "Staying on the
bleeding edge (Worg)":
- http://orgmode.org/worg/org-faq.html#keeping-current-with-Org-mode-development

That is:

$ git clone git://orgmode.org/org-mode.git    # standard repo
$ git clone git://repo.or.cz/org-mode.git    # alternate repo
$ git clone http://orgmode.org/org-mode.git  # if you can't use git:// (like me)
$ git clone http://repo.or.cz/r/org-mode.git  # alternate http

>
>> Then put in config:
>>
>> #+begin_src .emacs
>> (add-to-list 'load-path "~/.elisp/org.git/lisp/")
>> (add-to-list 'load-path "~/.elisp/org.git/contrib/lisp")
>> #+end_src
>
> Where does "/lisp" and "/contrib/lisp" come from?  What do they contain?

Not sure exactly what you mean... lisp/ and contrib/lisp/ are simply,
from what I know, where all the .el files for any emacs thingy reside.
You need to tell emacs about the contents of both of those. lisp/
contains the "core" Org stuff, and there are contributed functions in
contrib/lisp that are being evaluated or for whatever reason just not
incorporated into Org yet (I want to say that this might house things
that people who haven't signed FSF legal documents reside, or things
that just aren't quite ready yet).

>
>> That's it. Anywhere between once a week and once every three months, I
>> do:
>>
>> #+begin_src sh
>> cd ~/.elisp/org.git
>> git pull
>> make clean && make
>> #+end_src
>
> Do you run into any problems where something is picked up out of the
> "built-in" Org because of overlapping requires?

Not that I know of, but I never really got a definitive answer (at
least that sunk in my head as the "final verdict" on this thread:
- https://lists.gnu.org/archive/html/emacs-orgmode/2013-09/msg00175.html

Achim and I went back and forth a bit. My understanding from that time
(and from just now re-skimming it) is that as long as `M-x
org-version` shows what you'd expect (a git version) and you don't get
any errors, you're [probably/almost certainly?] safe. My current
output as an example:

M-x org-version
Org-mode version 8.2.5h (release_8.2.5h-881-g957177 @
/home/jwhendy/.elisp/org.git/lisp/)

>
>> Is this more difficult than packages? What is the advantage of ELPA
>> vs. this? I could see it if I had a lot of these sorts of things, but
>> I really just use Org + ESS, so I'm not constantly
>> updating/installing/removing emacs add-ons other than those two.
>
> It's not more difficult and you could probably easily expand on this for
> any number of packages by simply expanding your last shell to walk thru
> all the interesting packages and pull the latest version in.
>
> As far as I can see, ELPA's plus is the GUI for pulling in new versions
> of packages, but its minus is that it moves the package setup *somewhat*
> out of .emacs and into the after-init area.  For instance, I'm
> installing org-toodledo which hasn't been packagized yet, so, when you
> (require 'org-toodledo) in your .emacs, the org package is also pulled
> in.  However, if you've updated the org package via ELPA, the wrong
> version of org will be pulled in *unless* you (package-initialize) in
> your .emacs.  If you follow the basic rules, things work, but the rules
> aren't explicitly documented, so it can be confusing.
>

Not being familiar with ELPA, I really can't comment on how this
works. I will say that I really like the simplicity (or perceived
simplicity!) I've experienced with just using org git. I know where
everything is, I don't run `make install`, so I know that my system is
not littered with files other than where I've cloned the repo, and
to-date I've not had any issues with org setup stuff or alternate
versions (except when I set up a fresh install or something and forget
to add load paths, for example).


Best regards,
John


> --
> David Masterson
>
>

Re: The Org Package

[adam]

Thank you for this discussion, John, David and others. 

When installing a new distro, it sure seems wise to not include 
the Org options at Emacs install time (Ubuntu installers I use). 
And to set up the Org location and Git system soonest.

Re: The Org Package

[Grant Rettke]

On Fri, Apr 11, 2014 at 9:32 PM, David Masterson <dsmasterson@gmail.com> wrote:
> Do you see where I'm heading?

Yes. As stated above, Worg has nice instructions. My experience has
been that once you
start using org you end up wanting different lots of different
packages, and some that are very up to date. I ended up using Cask.
There are many options, and that is one of them, and I do like it.

I just keep a Cask configuration file and that deals with obtaining
and loading the right stuff.

Grant Rettke | AAAS, ACM, AMA, COG, FSF, IEEE, Sigma Xi
grettke@acm.org | http://www.wisdomandwonder.com/
“Wisdom begins in wonder.” --Socrates
((λ (x) (x x)) (λ (x) (x x)))
“Life has become immeasurably better since I have been forced to stop
taking it seriously.” --Thompson

Re: The Org Package

[Achim Gratz]

David Masterson writes:
> I still need more understanding of the Emacs packaging system.

That's a question you better ask of the Emacs developers, after you've
read the documentation.

> Something doesn't seem right and I'm sure I'm missing some key in
> understanding how its supposed to work.  What I see right now seems like
> something doesn't match up -- particularly with the Org package:
>
> 1. Most modern Emacs have Org pre-installed.
> 2. Unfortunately, that Org is not up-to-date (24.3 has 7.9.3f).
> 3. Therefore, installing the latest Org package seems natural.

The pitfall here is that you _must_ do the first install of the Org
package from an Emacs that didn't load any part of the built-in Org.

> 4. However, this does not uninstall the built-in Org package.

No, the built-in "package" unfortunately isn't really a package.  It's
still inextricably built-in the same way it ever was, just that it now
additionally shows up in package manager as built-in package.

> 5. Packages are not initialized until after .emacs is run.

You should add (package-initialize) to your .emacs, but read

http://www.gnu.org/software/emacs/manual/html_node/emacs/Package-Installation.html

before doing that so you'll understand where to put it.

> 6. Therefore, any of the latest variables are not defined yet.
> 7. Therefore, setting a hook may not do what you think.
> 8. The documentation for Org suggests hooks (etc.) to set.
> 9. I've run into times when org-version was still 7.9.3f.
>
> Do you see where I'm heading?  Does anyone else run into this problem?
>
> Or do most people ignore the Org package and install the latest from
> GitHub in a more manual process (a la Bernt Hansen's paper)?  Do we need
> more concrete documentation on setting up the Org Package?

See above.  The package system in Emacs isn't really designed for
something as big and complicated as Org and its interaction with
built-in packages has room for improvement; but you can make it work,
even for Org.

Whether you want to install Org from Git is another story and you should
mainly decide from whether you need to always run the very latest
development version.  If that decision rather falls to "I always want
the latest released version", then the ELPA package is a better choice,
IMHO.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: The Org Package

[Nicolas Richard]

Achim Gratz <Stromeko@nexgo.de> writes:
>> 1. Most modern Emacs have Org pre-installed.
>> 2. Unfortunately, that Org is not up-to-date (24.3 has 7.9.3f).
>> 3. Therefore, installing the latest Org package seems natural.
>
> The pitfall here is that you _must_ do the first install of the Org
> package from an Emacs that didn't load any part of the built-in Org.

That's an interesting piece of information. I can't find it in (info
"(org) Installation").

Just to make sure I understand it : I can imagine older macros being
byte compiled into the newer org, is that the reason ?

-- 
Nico.

Re: The Org Package

[David Masterson]

Achim Gratz <Stromeko@nexgo.de> writes:

> David Masterson writes:
>> Something doesn't seem right and I'm sure I'm missing some key in
>> understanding how its supposed to work.  What I see right now seems like
>> something doesn't match up -- particularly with the Org package:
>>
>> 1. Most modern Emacs have Org pre-installed.
>> 2. Unfortunately, that Org is not up-to-date (24.3 has 7.9.3f).
>> 3. Therefore, installing the latest Org package seems natural.
>
> The pitfall here is that you _must_ do the first install of the Org
> package from an Emacs that didn't load any part of the built-in Org.

What does this mean?  Does this mean you expect people to build Emacs
from scratch just to ensure they do not have Org built-in?


-- 
David Masterson
Programmer At Large

Re: The Org Package

[Thomas S. Dye]

David Masterson <dsmasterson@gmail.com> writes:

> Achim Gratz <Stromeko@nexgo.de> writes:
>
>> David Masterson writes:
>>> Something doesn't seem right and I'm sure I'm missing some key in
>>> understanding how its supposed to work.  What I see right now seems like
>>> something doesn't match up -- particularly with the Org package:
>>>
>>> 1. Most modern Emacs have Org pre-installed.
>>> 2. Unfortunately, that Org is not up-to-date (24.3 has 7.9.3f).
>>> 3. Therefore, installing the latest Org package seems natural.
>>
>> The pitfall here is that you _must_ do the first install of the Org
>> package from an Emacs that didn't load any part of the built-in Org.
>
> What does this mean?  Does this mean you expect people to build Emacs
> from scratch just to ensure they do not have Org built-in?

I believe it means the installation must be done from an Emacs instance
that hasn't loaded any Org mode functions yet.

  You must make sure there are no =Org-mode= functions loaded while
  the update is done. For that, exit Emacs and then run Emacs without
  loading your =.emacs= (=Emacs -Q=). Remove the old =Org-mode=
  #+BEGIN_SRC sh
  rm -rf ~/.emacs.d/elpa/org-”Tab”
  #+END_SRC
  where =Tab= means press =Tab= to see and auto complete the old
  =Org-mode= directory you want to remove. Finally, update =Org-mode=
  (=M-x package-install RET org RET=) and restart Emacs as usual.

See http://nickhigham.wordpress.com/2013/07/05/emacs-org-mode-version-8/

hth,
Tom
-- 
Thomas S. Dye
http://www.tsdye.com

Re: The Org Package

[David Masterson]

tsd@tsdye.com (Thomas S. Dye) writes:

> David Masterson <dsmasterson@gmail.com> writes:
>
>> Achim Gratz <Stromeko@nexgo.de> writes:
>>
>>> David Masterson writes:
>>>> Something doesn't seem right and I'm sure I'm missing some key in
>>>> understanding how its supposed to work.  What I see right now seems like
>>>> something doesn't match up -- particularly with the Org package:
>>>>
>>>> 1. Most modern Emacs have Org pre-installed.
>>>> 2. Unfortunately, that Org is not up-to-date (24.3 has 7.9.3f).
>>>> 3. Therefore, installing the latest Org package seems natural.
>>>
>>> The pitfall here is that you _must_ do the first install of the Org
>>> package from an Emacs that didn't load any part of the built-in Org.
>>
>> What does this mean?  Does this mean you expect people to build Emacs
>> from scratch just to ensure they do not have Org built-in?
>
> I believe it means the installation must be done from an Emacs instance
> that hasn't loaded any Org mode functions yet.
>
>   You must make sure there are no =Org-mode= functions loaded while
>   the update is done. For that, exit Emacs and then run Emacs without
>   loading your =.emacs= (=Emacs -Q=). Remove the old =Org-mode=
>   #+BEGIN_SRC sh
>   rm -rf ~/.emacs.d/elpa/org-Tab
>   #+END_SRC
>   where =Tab= means press =Tab= to see and auto complete the old
>   =Org-mode= directory you want to remove. Finally, update =Org-mode=
>   (=M-x package-install RET org RET=) and restart Emacs as usual.
>
> See http://nickhigham.wordpress.com/2013/07/05/emacs-org-mode-version-8/

Hmmm.  I would think that this process might work as well:

1. Start with "emacs -Q"
2. Bring up the Emacs Packages list
3. Mark the "available" org for "U"pgrade
4. Execute the upgrade
5. Restart Emacs

The only question is what about the built-in Org?

-- 
David Masterson
Programmer At Large

Re: The Org Package

[Achim Gratz]

David Masterson writes:
> Hmmm.  I would think that this process might work as well:
>
> 1. Start with "emacs -Q"

This fails with some older versions of Emacs since these had a bug where
parts of the init scripts were processed even when you said explicitly
that they shouldn't be read.

> 2. Bring up the Emacs Packages list
> 3. Mark the "available" org for "U"pgrade
> 4. Execute the upgrade
> 5. Restart Emacs

With the version of Emacs you have that should work, yes.

> The only question is what about the built-in Org?

Leave it alone.  Unless you _are_ planning to build your own Emacs
without it, nothing can be done that might not lead to hard to find
problems later on.  If you're willing to experiment, look at

testing/org-batch-test-init.el

for how to get rid of the pre-installed Org (or at least those parts
that I know about).  Obviously it works in batch mode, otherwise we'd
have tons of complaints about it; but I can't really say if there might
be problems with a full-fledged GUI Emacs session.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

Re: The Org Package

[Achim Gratz]

David Masterson writes:
> What does this mean?

What I said: don't load any part of Org until you have installed the
ELPA package.  This usually means not to run any startup scripts.

> Does this mean you expect people to build Emacs
> from scratch just to ensure they do not have Org built-in?

I don't expect anything like that from anybody.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables

Re: The Org Package

[Bastien]

Hi all,

Achim Gratz <Stromeko@nexgo.de> writes:

> David Masterson writes:
>> What does this mean?
>
> What I said: don't load any part of Org until you have installed the
> ELPA package.  This usually means not to run any startup scripts.

I've know made this clearer in the manual.

Nicolas Richard <theonewiththeevillook@yahoo.fr> writes:

>> The pitfall here is that you _must_ do the first install of the Org
>> package from an Emacs that didn't load any part of the built-in Org.
>
> That's an interesting piece of information. I can't find it in (info
> "(org) Installation").

Now it is!

Thanks,

-- 
 Bastien