Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Jonas Bernoulli]

Hello,

Until a few months ago I was able to use `org-cycle' in
`emacs-lisp-mode' buffers, and it just worked after turning on
`outline-minor-mode' but not `orgstruct-mode'.  I am using the
development versions of both Emacs (25.0.50) and Org (9.0).

But somewhere along the way `org-cycle' partially broke when used like
this.  When switching to the CHILDREN state I now see this:

  ;;; heading
  
  ...
  ;;; next heading

instead of (when there is only code in the section)

  ;;; heading
  
  (defun foo ()...
  (defun bar ()...
  ;;; next heading

or (when there are subheadings)

  ;;; heading
  ;;;; sub-heading one...
  ;;;; sub-heading two...
  ;;; next heading

I would like to use `orgstruct-mode' in `emacs-lisp-mode' but I cannot
get it to work at all.  So for now I would be happy if I could just use
`org-cycle' as I used to.

Finally I have gotten around to investigate this and came across this:

  ;; This is done in `emacs-lisp-mode':
  (setq-local outline-regexp ";;;\\(;* [^ \t\n]\\|###autoload\\)\\|(")

  (defun lisp-outline-level ()
    "Lisp mode `outline-level' function."
    (let ((len (- (match-end 0) (match-beginning 0))))
      (if (looking-at "(\\|;;;###autoload")
          1000
        len)))

This doesn't make any sense to me.  Maybe these values now take the
needs of `orgstruct-mode' into account, but when thinking just of
`outline-minor-mode', then I would expect something like this instead:

  (setq-local outline-regexp "\\(;;;+ \\)\\|(")

  (defun lisp-outline-level ()
    (cond ((match-beginning 1) (- (length (match-string 1)) 2))
          ((match-beginning 0) 6)))

In addition to that I also had to add this for `org-cycle' to work in
`emacs-lisp-mode' again.

  (add-hook 'org-cycle-hook 'org-cycle-elisp-kludge)
  
  (defun org-cycle-elisp-kludge (state)
    (when (and (eq state 'children)
               (derived-mode-p 'emacs-lisp-mode))
      (save-excursion
        (goto-char eoh)
        (beginning-of-line)
        (outline-show-branches))))
  
  (defun org-before-first-heading-p ()
    (if (derived-mode-p 'emacs-lisp-mode)
        nil ; in that mode the below always return t
      (save-excursion
        (end-of-line)
        (null (re-search-backward org-outline-regexp-bol nil t)))))

Now, I am probably barking at the wrong tree here, but since I also
couldn't get `orgstruct-mode' to work at all, I just investigated the
code which I still somewhat understood, `orgstruct-setup' et al is
beyond me.

So I guess my questions are:

1. In what way does the default value of `outline-regexp' and the
   definition of `lisp-outline-level' make sense?

2. How do I get from `emacs -Q' to TAB on a heading in `emacs-lisp-mode'
   cycling though the tree states (with or without `orgstruct-mode').

The Org manual sais this:

>   You can also use Org structure editing to fold and unfold headlines
> in _any_ file, provided you defined ‘orgstruct-heading-prefix-regexp’:
> the regular expression must match the local prefix to use before Org’s
> headlines.  For example, if you set this variable to ‘";; "’ in Emacs
> Lisp files, you will be able to fold and unfold headlines in Emacs Lisp
> commented lines.  Some commands like ‘org-demote’ are disabled when the
> prefix is set, but folding/unfolding will work correctly.

At least for me that wasn't the case at all (what keys do I have to
press to do the folding/unfolding?).

  Thanks for your help,
  Jonas

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Nicolas Goaziou]

Hello,

Jonas Bernoulli <jonas@bernoul.li> writes:

> I would like to use `orgstruct-mode' in `emacs-lisp-mode' but I cannot
> get it to work at all.  So for now I would be happy if I could just use
> `org-cycle' as I used to.

Thank you for the report. 

`orgstruct-mode' really needs some love. Being to much linked with Org's
innards, it prevents us from using faster algorithms in pure Org
buffers.

Anyway, I pushed a fix for the issue.


Regards,

-- 
Nicolas Goaziou

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Jonas Bernoulli]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> `orgstruct-mode' really needs some love.

It's really hard to understand what is going on for someone not familiar
with the Org code.  Furthermore I am not using the default Emacs
bindings (but I also tested with emacs -Q), so code that generates
commands and bindings, like `outstruct-setup' does, always makes me
uncomfortable, as it usually isn't flexible enough to deal with heavily
modified bindings.

> Anyway, I pushed a fix for the issue.

Thanks.  But could you please change it to

  (if (or outline-minor-mode orgstruct-mode)
      (call-interactively #'show-children)
    ...)

`outline-minor-mode' combined with `org-cycle' gives me the feature I
*really* want/need.  `orgstruct-mode' sounds like it could be really
useful too, but as I mentioned it doesn't work for me at all.  So for
now I would like to stick to just `outline-minor-mode'.

  Thanks
  Jonas

Ps: I am still not sure whether I am just doing something stupid or why
    else `orgstruct-mode' doesn't work for me.  So I would apprechiate a
    little tutorial starting from emacs -Q that demonstrates what setup
    is required and what keys I have to press for things to happen.

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Aaron Ecay]

Hi Jonas,

2015ko azaroak 8an, Jonas Bernoulli-ek idatzi zuen:
> 
> `outline-minor-mode' combined with `org-cycle' gives me the feature I
> *really* want/need.

Thorsten Jolitz wrote the outshine library to provide something like
this.  Sadly it’s not actively maintained ATM, but I believe it still
works fine, and it may be an alternative route towards the features you
are looking for.

The package is on github: <https://github.com/tj64/outshine>.  It’s also
available through melpa.

-- 
Aaron Ecay

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Jorge A. Alfaro-Murillo]

Jonas Bernoulli writes:

> `orgstruct-mode' sounds like it could be really useful too, but 
> as I mentioned it doesn't work for me at all.  So for now I 
> would like to stick to just `outline-minor-mode'. 
> 
>   Thanks
>   Jonas 
> 
> Ps: I am still not sure whether I am just doing something stupid 
> or why else `orgstruct-mode' doesn't work for me.  So I would 
> apprechiate a little tutorial starting from emacs -Q that 
> demonstrates what setup is required and what keys I have to 
> press for things to happen. 

Generally it is enough to set orgstruct-heading-prefix-regexp and 
turn-on-orgstruct. I do this on a per mode basis, for example:

#+BEGIN_SRC emacs-lisp
  (add-hook 'elisp-mode-hook 'turn-on-orgtbl)
  (add-hook 'elisp-mode-hook
            (lambda ()
                    (setq-local orgstruct-heading-prefix-regexp
                                ";; ")))
#+END_SRC

Then you can organize your code using

;; * section-name

;; ** subsection-name

and folding with TAB, shift-TAB, etc will work when you are in 
those lines.

To use lists you just have to start a line with -, +, 1., etc and 
then lists keybindings work, for example M-C-m for new items. I do 
not know why lists do not respect orgstruct-heading-prefix-regexp, 
it has always seem weird to me that I cannot gain orgstruct 
functionality with something like:

;; - first item
;; - second item

-- 
Jorge.

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Jonas Bernoulli]

> #+BEGIN_SRC emacs-lisp
>   (add-hook 'elisp-mode-hook 'turn-on-orgtbl)
>   (add-hook 'elisp-mode-hook
>             (lambda ()
>                     (setq-local orgstruct-heading-prefix-regexp
>                                 ";; ")))
> #+END_SRC

It appears the stupid thing I did was that I used ";;" instead.

Thanks for the help.

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Jonas Bernoulli]

Aaron Ecay <aaronecay@gmail.com> writes:

> Thorsten Jolitz wrote the outshine library

I know. I used it for a while and contributed a few commits.  But I
pretty much only used the cycling functionality at the time and when
I discovered that `org-cycle' worked for that too, I stopped using
outshine.  It also felt a bit like re-inventing the wheel.

> Sadly it’s not actively maintained ATM, but I believe it still works
> fine,

That's a bit of an understatement; if I remember correctly Thorsten
stopped using Emacs.

> and it may be an alternative route towards the features you are
> looking for.

Well currently I am just using `org-cycle' + `outline-minor-mode', which
works well enough for now.  But eventually I would like to also start
using Org's navigational commands.  Unfortunately `orgstruct-mode' only
supports org-like headings (;; * heading\n;; ** subheadng) and not
standard Emacs headings (;;; heading\n;;;; subheading).  I hope someone
teaches `orgstruct-mode' to support the latter too.  Otherwise I will
probably give `outshine' another chance.

  But thanks for the tip :-)
  Jonas

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Rasmus]

Hi Jonas,

Jonas Bernoulli <jonas@bernoul.li> writes:

> Well currently I am just using `org-cycle' + `outline-minor-mode', which
> works well enough for now.  But eventually I would like to also start
> using Org's navigational commands.  Unfortunately `orgstruct-mode' only
> supports org-like headings (;; * heading\n;; ** subheadng) and not
> standard Emacs headings (;;; heading\n;;;; subheading).  I hope someone
> teaches `orgstruct-mode' to support the latter too.  Otherwise I will
> probably give `outshine' another chance.

My init file is structured like this, and I mainly use org-cycle as well.
Note that there’s an outstanding bug related to orgstruct and
org-show-children (as I remember).

     ;;* TEXT MODES
     ;;** AUCTeX
     ;;   ...
     ;;** Reftex
     ;;   ...
     ;;* PROG MODES
     ;;  ...
     ;;** Magit
     ;;...
     ;;...

     ;; Local Variables:
     ;; outline-regexp: ";;\\*+\\|\\`"
     ;; orgstruct-heading-prefix-regexp: ";;\\*+\\|\\`"
     ;; eval: (when after-init-time (orgstruct-mode) (org-global-cycle 3))
     ;; End:

Rasmus

-- 
Vote for Dick Taid in an election near you!

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Nicolas Goaziou]

Hello,

Jonas Bernoulli <jonas@bernoul.li> writes:

> Thanks.  But could you please change it to
>
>   (if (or outline-minor-mode orgstruct-mode)
>       (call-interactively #'show-children)
>     ...)

You could set `orgstruct-mode' to a non-nil value whenever
`outline-minor-mode' is enabled.

Regards,

-- 
Nicolas Goaziou

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Jonas Bernoulli]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Hello,
>
> Jonas Bernoulli <jonas@bernoul.li> writes:
>
>> Thanks.  But could you please change it to
>>
>>   (if (or outline-minor-mode orgstruct-mode)
>>       (call-interactively #'show-children)
>>     ...)
>
> You could set `orgstruct-mode' to a non-nil value whenever
> `outline-minor-mode' is enabled.

I could (instead I am currently just maintaining a local patch),
but what is the reasoning for not just doing what I suggested?

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Nicolas Goaziou]

Hello,

Jonas Bernoulli <jonas@bernoul.li> writes:

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>
>>> Thanks.  But could you please change it to
>>>
>>>   (if (or outline-minor-mode orgstruct-mode)
>>>       (call-interactively #'show-children)
>>>     ...)
>>
>> You could set `orgstruct-mode' to a non-nil value whenever
>> `outline-minor-mode' is enabled.
>
> I could (instead I am currently just maintaining a local patch),
> but what is the reasoning for not just doing what I suggested?

There is no reason for Org's core to know about `outline-minor-mode', or
any other minor mode in the wild. 

IMO, using a hook is much more simple than maintaining a local patch.


Regards,

-- 
Nicolas Goaziou

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Jonas Bernoulli]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Jonas Bernoulli <jonas@bernoul.li> writes:
>
>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>
>>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>>
>>>> Thanks.  But could you please change it to
>>>>
>>>>   (if (or outline-minor-mode orgstruct-mode)
>>>>       (call-interactively #'show-children)
>>>>     ...)
>>>
>>> You could set `orgstruct-mode' to a non-nil value whenever
>>> `outline-minor-mode' is enabled.
>>
>> I could (instead I am currently just maintaining a local patch),
>> but what is the reasoning for not just doing what I suggested?
>
> There is no reason for Org's core to know about `outline-minor-mode', or
> any other minor mode in the wild.

I wouldn't group `outline-minor-mode' among "any other minor mode
in the wild", after all it was Outline which gave birth to Org and
`outline-minor-mode' is part of `outline.el'.  Didn't Carsten write
`outline-magic.el' (containing `outline-cycle') before moving on to
greater things?  Unfortunately though `outline-magic.el' does not appear
to have aged so well (at least that's how I remember it from when I last
checked).

Anyway I think `outline-minor-mode' deserves some special attention,
but of course that's the call of the maintainers.

> IMO, using a hook is much more simple than maintaining a local patch.

Not for me.  I install all packages using Git submodules, which is very
convenient because I contribute to most packages I use, at least in some
minor way like addressing compilation warnings.  If a patch isn't
accepted upstream, then the only additional work required is to call
`git config branch.master.rebase true' once and then occasionally
`git push tarsius master'.

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Rasmus]

Hi Jonas,

Jonas Bernoulli <jonas@bernoul.li> writes:

> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>
>>> Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>>>
>>>> Jonas Bernoulli <jonas@bernoul.li> writes:
>>>>
>>>>> Thanks.  But could you please change it to
>>>>>
>>>>>   (if (or outline-minor-mode orgstruct-mode)
>>>>>       (call-interactively #'show-children)
>>>>>     ...)
>>>>
>>>> You could set `orgstruct-mode' to a non-nil value whenever
>>>> `outline-minor-mode' is enabled.
>>>
>>> I could (instead I am currently just maintaining a local patch),
>>> but what is the reasoning for not just doing what I suggested?
>>
>> There is no reason for Org's core to know about `outline-minor-mode', or
>> any other minor mode in the wild.
>
> I wouldn't group `outline-minor-mode' among "any other minor mode
> in the wild", after all it was Outline which gave birth to Org and
> `outline-minor-mode' is part of `outline.el'.  Didn't Carsten write
> `outline-magic.el' (containing `outline-cycle') before moving on to
> greater things?  Unfortunately though `outline-magic.el' does not appear
> to have aged so well (at least that's how I remember it from when I last
> checked).

orgstruct-mode is a mess and we plan to address it.  When someone finds
the time to do so.  Indeed, there was a thread about the issue of
org-show-children some time ago.

Cheers,
Rasmus

-- 
Dung makes an excellent fertilizer

Re: Using orgstruct-mode (or just org-cycle) in emacs-lisp-mode

[Thorsten Jolitz]

Jonas Bernoulli <jonas@bernoul.li> writes:

Hi List,

> Aaron Ecay <aaronecay@gmail.com> writes:
>
>> Thorsten Jolitz wrote the outshine library
>
> I know. I used it for a while and contributed a few commits.  But I
> pretty much only used the cycling functionality at the time and when
> I discovered that `org-cycle' worked for that too, I stopped using
> outshine.  It also felt a bit like re-inventing the wheel.
>
>> Sadly it’s not actively maintained ATM, but I believe it still works
>> fine,

When I was a heavy outshine user myself it was feature rich and very
stable, and I think it still should be.  So being unmaintained does not
mean you can or should not use it, it only means that I did not add any
new features the last year (and sometimes miss a bug report,
unfortunately). But the three libraries of the outshine-suite
(outshine.el, outorg.el and navi-mode.el) already have so many features
that even the main author could not remember all of them ...

> That's a bit of an understatement; if I remember correctly Thorsten
> stopped using Emacs.

Its true that events in my life kicked me out of the (enjoyable) Emacs
universe, but now I actually use Emacs again once in a while, and hope
more frequently in the future.

>> and it may be an alternative route towards the features you are
>> looking for.
>
> Well currently I am just using `org-cycle' + `outline-minor-mode', which
> works well enough for now.  But eventually I would like to also start
> using Org's navigational commands.  Unfortunately `orgstruct-mode' only
> supports org-like headings (;; * heading\n;; ** subheadng) and not
> standard Emacs headings (;;; heading\n;;;; subheading).  I hope someone
> teaches `orgstruct-mode' to support the latter too.  Otherwise I will
> probably give `outshine' another chance.

When there are problems with orgstruct - while waiting for a new
implementation based on outline-minor-mode, why not use an existing,
much more powerfull and very stable alternative based on
outline-minor-mode?

When writing an org-minor-mode (orgstruct, outshine, ...) the most
frequent FAQ is by definition:

"I want Org-mode feature XYZ in org-minor-mode too."

The org-minor-mode author then figures out quickly that

 - Org-mode is not a library engineered for reuse by other
   applications. Org functions do many things, and its often hard or
   impossible to just call them for a result or even copy&paste them
   for reuse.
 - he does not want to rewrite Org-mode

My solution for outshine was 

,----[ C-h f outshine-use-outorg RET ]
| outshine-use-outorg is a Lisp function in `outshine.el'.
| 
| (outshine-use-outorg FUN &optional WHOLE-BUFFER-P &rest FUNARGS)
| 
| Use outorg to call FUN with FUNARGS on subtree or thing at point.
| 
| FUN should be an Org-mode function that acts on the subtree or
| org-element at point. Optionally, with WHOLE-BUFFER-P non-nil,
| `outorg-edit-as-org' can be called on the whole buffer.
| 
| Sets the variable `outshine-use-outorg-last-headline-marker' so
| that it always contains a point-marker to the last headline this
| function was called upon.
`----

With this function potentially all Org-mode commands can easily be made
available in outshine. In practise, some Org functions are really tricky
and need some extra care, but many work out of the box.

Here is what I more or less implemented already. Especially clocking was
tricky, don't remember if it really worked in the end:

,----
| 39 matches for "^[[:space:]]*(def[maus][^eltu][a-z]*\*? " in buffer: outshine.el
|      30:(defun outshine-deadline (&optional arg)
|      40:(defun outshine-export-dispatch (&optional arg)
|      59:(defun outshine-insert-link ()
|      67:(defun outshine-open-at-point (&optional whole-buffer-p arg reference-buffer)
|      84:(defun outshine-set-tags-command ()
|      97:(defun outshine-schedule (&optional arg)
|     108:(defun outshine-todo (&optional arg)
|     148:(defun outshine-time-stamp-inactive (&optional arg)
|     213:(defun outshine-priority ()
|     228:(defun outshine-time-stamp (&optional arg)
|     254:(defun outshine-toggle-fixed-width ()
|     260:(defun outshine-toggle-comment ()
|     304:(defun outshine-sort-entries (&optional arg)
|     366:(defun outshine-previous-block ()
|     373:(defun outshine-next-block ()
|     379:(defun outshine-insert-last-stored-link ()
|     571:(defun outshine-toggle-checkbox (&optional arg)
|     596:(defun outshine-clock-in ()
|     606:(defun outshine-clock-goto ()
|     625:(defun outshine-next-link ()
|     632:(defun outshine-clock-out ()
|     644:(defun outshine-previous-link ()
|     651:(defun outshine-clock-cancel ()
|     662:(defun outshine-clock-report (&optional arg)
|     732:(defun outshine-timer-pause-or-continue (&optional arg)
|     738:(defun outshine-timer-item ()
|     744:(defun outshine-timer ()
|     750:(defun outshine-timer-start ()
|     756:(defun outshine-timer-cancel-timer ()
|     762:(defun outshine-timer-set-timer ()
|     768:(defun outshine-agenda-set-restriction-lock (&optional arg)
|     780:(defun outshine-agenda-remove-restriction-lock (&optional include-org-p)
|     797:(defun outshine-inc-effort ()
|     818:(defun outshine-set-property-and-value ()
|     847:(defun outshine-toggle-archive-tag ()
|     865:(defun outshine-insert-drawer ()
|     871:(defun outshine-set-effort (&optional arg)
|     878:(defun outshine-footnote-action (&optional special)
|     906:(defun outshine-set-property ()
`----

There are much more outcommented function skeletons waiting for (your?)
implementation in section

,----
| ;;; Use Outorg for calling Org
`----

of outshine.el. Here are a few implementation examples to show how
easy this is:

,----
| ;; C-c C-x d	org-insert-drawer
| (defun outshine-insert-drawer ()
|   "Call outorg to trigger `org-insert-drawer'."
|   (interactive)
|   (outshine-use-outorg 'org-insert-drawer))
| 
| ;; C-c C-x e	org-set-effort
| (defun outshine-set-effort (&optional arg)
|   "Call outorg to trigger `org-set-effort'."
|   (interactive "p")
|   (outshine-use-outorg
|    'org-set-effort nil arg))
| 
| ;; C-c C-x f	org-footnote-action
| (defun outshine-footnote-action (&optional special)
|   "Call outorg to trigger `org-footnote-action'."
|   (interactive "P")
|   (outshine-use-outorg
|    'org-footnote-action 'WHOLE-BUFFER-P special))
| 
| ;; C-c C-x p	org-set-property
| (defun outshine-set-property ()
|   "Call outorg to trigger `org-set-property'."
|   (interactive)
|   (outshine-use-outorg 'org-set-property))
`----

With these commands, you can call Org-mode functionality directly in
comment sections of programming modes like emacs-lisp-mode, e.g. add
TODOs, TAGS, Properties, timestamps, (maybe) even clock-in and out. 

Here I added, as an example, some Org-mode headline stuff to org.el,
directly in the elisp buffer (with outline-minor-mode and outshine
activated):

,----
| ;;; TODO [#C] Commentary: :urgent:
| ;;   :PROPERTIES:
| ;;   :CATEGORY: office
| ;;   :END:
| 
| ;; Org-mode is a mode for keeping notes, maintaining ToDo lists, and doing
| ;; project planning with a fast and effective plain-text system.
`----

-- 
cheers,
Thorsten