Re: #3 Org mode profiling meetup on Sat, Apr 23 (was: #2 Org mode profiling meetup on Sat, Mar 26)

[Ihor Radchenko <yantar92@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 290 bytes --]

Ihor Radchenko <yantar92@gmail.com> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> I will post the link to the meeting one hour before the meeting start.
>
> https://teamjoin.de/Org-dev-profiling-202204-23-d708k

Summary of the discussion is in the attached .org.

Best,
Ihor


[-- Attachment #2: summary.org --]
[-- Type: application/vnd.lotus-organizer, Size: 9936 bytes --]

# Created 2022-04-24 Sun 12:25
#+title: <2022-04-23 Sat 20:00>--<2022-04-23 Sat 23:00>  Ihor Radchenko [ML:Org mode] (2022) #3 Org mode profiling meetup on Sat, Apr 23 (was: #2 Org mode profiling meetup on Sat, Mar 26)
#+date: May 11, 2020
#+author: Ihor Radchenko
Meeting link: https://teamjoin.de/Org-dev-profiling-202204-23-d708k

Summary of the discussions:
- marking bugs in updates.orgmode.org: Woof!
  - This is mainly for Org maintainers
  - There are some bugs in the current version of the mailing list control code
  - It should be updated in the coming weeks though, according to Bastien
- patches backlog and merge policy
  - We have accumulating patch backlog at https://updates.orgmode.org
  - Mostly because our main maintainer has been busy
    - We need more maintainers! Feel free to apply by writing to Bastien (https://bzg.fr/en/)
  - I recently figured that maintainers with write access can freely push new feature patches to unmaintained
    (with author/maintainer missing or not being active on the list)
    files.
- debugging infinite recursion in org-eldoc
  - https://list.orgmode.org/CAFyQvY1QNfxBOrCor3pLR3MoMpMemD9znhX+GaV4nQKiDS=bjQ@mail.gmail.com/T/#u
- debugging recent bug report in to-be-merged org-fold branch (fontification)
  - https://orgmode.org/list/8735i5gd8n.fsf@gmail.com
  - I now managed to fix it. Going to push soon
- searching specific changes via magit-log
  - Some places to Org codebase may be hard to understand
    - Using ~magit-file-dispatch~ allows to search git log history associated with selected region
    - Commit messages in the history may reveal why one or another piece of code is there

Also, I have prepared and even discussed small pieces of the presentation below.
However, most of the people who joined the meeting already knew all
that or were not interested. Still leaving it below to make it not go to complete waste.

* Contributing patches to Org

Before we start:
1. Clone the latest Org repo (see https://orgmode.org)
   #+begin_src bash
      git clone git://git.sv.gnu.org/emacs/org-mode.git
      cd org-mode
   #+end_src
2. If you are contributing/testing a new feature
   #+begin_src bash
      git checkout main
   #+end_src
3. If you are contributing/testing a bugfix
   #+begin_src bash
      git checkout bugfix # later, also need to confirm that everything works fine on main
   #+end_src
4. Use Magit! https://magit.vc/
   - Changing branch is ~magit-branch~ "b" -> branch "b"

** Applying patches sent by email

Case 1: Attachments to emails
- Attachments can be simply saved
- It is a good practice to create a temporary branch
  - =magit-branch= "b" -> new "c" -> starting at main/bugfix -> patch/mypatch-or-any-other-name
- ~magit-patch~ or "W" in magit status buffer -> Apply patches (w) -> patches (w)
- *Or just use piem* (see below)

Case 2: Embedded patches (I do not like them)
- Can directly use ~git-am~, but need to remember all that command line args
  - http://git-scm.com/docs/git-am
- I prefer https://git.kyleam.com/piem
  #+begin_src emacs-lisp
      (require 'piem)
      (add-to-list 'piem-inboxes
               '( "Org mode"
                  :coderepo "~/Git/org-mode/"
                  :address "emacs-orgmode@gnu.org"
                  :listid "emacs-orgmode.gnu.org"
                  :url "https://orgmode.org/list/"
                  :maildir "~/Mail/Orgmode-maillist/orgmode/"))
  	(piem-notmuch-mode +1)
  	;; (piem-gnus-mode +1)
  	;; (piem-elfeed-mode +1)
  	;; (piem-rmail-mode +1)
  #+end_src
- Just run ~M-x piem-am~ from email buffer
  - It will do everything from Case 1 automatically

** Testing Org mode patches
It's generally easy:
#+begin_src emacs-lisp
cd org-mode
make test
#+end_src

Result (good one):
#+begin_quote
Ran 927 tests, 927 results as expected, 0 unexpected (2022-04-23 18:39:33+0800, 25.511741 sec)
15 expected failures
#+end_quote


For more in-depth testing, there are two things to consider:
1. Different emacs versions
   #+begin_src emacs-lisp
      # emacs-26 is executable name or path to install Emacs version 26
      make cleanall
      make EMACS=emacs-26 test
   #+end_src

2. Different language environments
   #+begin_src bash
      LANG="de_DE.UTF-8" make test
   #+end_src

:CATCHALL-SCRIPT:
#+begin_src bash
#!/bin/bash
# [[file:../../Org/system-config.org::*Testing emacs repo][Testing emacs repo:1]]
function yes_or_no {
    while true; do
        read -p "$* [y/n]: " yn
        case $yn in
            [Yy]*) return 0  ;;  
	    [Nn]*) echo "Aborted" ; return  1 ;;
	esac
done
}

set -e
make cleanall
make EMACS=emacs-26 $* test || (echo "Failed to run tests using $(emacs-26 --version | head -n1)"; yes_or_no " Continue?")
make cleanall
make EMACS=emacs-27 $* test || (echo "Failed to run tests using $(emacs-27 --version | head -n1)"; yes_or_no " Continue?")
make cleanall
make EMACS=emacs-28-vcs $* test || (echo "Failed to run tests using $(emacs-28-vcs --version | head -n1)"; yes_or_no " Continue?")
make cleanall
make EMACS=emacs-29-vcs $* test || (echo "Failed to run tests using $(emacs-29-vcs --version | head -n1)"; yes_or_no " Continue?")
make cleanall
LANG="C" make EMACS=emacs-29-vcs $* test || (echo "Failed to run tests using LANG=C $(emacs-29-vcs --version | head -n1)"; yes_or_no " Continue?")
make cleanall
LANG="de_DE.UTF-8" make EMACS=emacs-29-vcs $* test || echo "Failed to run tests using LANG=de_DE.UTF-8 $(emacs-29-vcs --version | head -n1)"
# Testing emacs repo:1 ends here
#+end_src
:END:

*** What to do in case of error?

#+begin_quote
17 unexpected results:
   FAILED  test-org-clock/clocktable/compact
   FAILED  test-org-clock/clocktable/extend-today-until
   ...
   FAILED  test-org/string-width
#+end_quote
**** Manual testing

Org mode uses ERT testing library built into Emacs.

1. Open ~cd org-mode; make cleanall; make autoloads; emacs -Q -L ./lisp -l org~
   - *It is important _not_ to load personal config*
     Tests are using certain assumptions about Org settings
2. Open ~org-mode/testing/org-test.el~ and M-x eval-buffer
3. Open ~org-mode/test/lisp/test-with-failing-test-name.el~ and M-x eval-buffer
4. M-x ert <RET> failed-test-name <RET>
   - For all tests, use M-x ert <RET> t <RET>
5. For more fine-grained testing, can as well use C-x C-e (eval-last-sexp) on "should" forms inside the test

   *Keep in mind that some test failures (especially related to
   asynchronous code like font-locking) may not be reproducible in
   interactive Emacs*

**** ~git bisect~

1. Go to magit status buffer
2. ~magit-bisect~ B -> start (B) -> this revision is erroneous <RET> -> some recent working rev (e.g. main~20)
3. From terminal
   #+begin_src bash
      make cleanall; make BTEST_RE="^test-org-colview/columns-move-left" test
      # BTEST_RE limits the number of checked tests to what you specify.
      # There is no need to re-run all the tests again and again.
   #+end_src
4. From magit status buffer:
   - If error is still there, ~magit-bisect~ B -> bad (B)
   - No error,  ~magit-bisect~ B -> good (g)

** Creating patches and sending them to Org mailing list

With Magit, it is pretty much trivial:
1. Make sure that you are using the latest Org mode version
   - ~git-fetch~ F -> origin (usually "u")
2. Do the changes in org-mode code
3. Test them (as above)
4. From magit status buffer: stage all (S) -> commit (c) -> commit (c)
5. Write detailed commit message (see below)
6. Create the patch by ~magit-patch~ W -> create (c) -> create (c) -> <RET> (will create patches from all new commits)
7. Write and email to emacs-orgmode@gnu.org
   - Subject shoud start with [PATCH]
     =[PATCH] org.el: Refactor function=
   - Tell why your patch should be merged in the body
   - Attach your patch file
   - The patch will soon (5-15 min) appear at https://updates.orgmode.org/

   *Note that part of the above steps can be automated with
    https://git.sr.ht/~yoctocell/git-email, but I do not (yet,
    [2022-04-23 Sat]) recommend it as it is too early in development and
    has bugs)*

*** Writing commit messages

You need to follow specific format detailed in https://orgmode.org/worg/org-contribute.html#commit-messages

Briefly:
- We generally put the main library or file that is affected by patch at the beginning
  ~org-element.el Fix headline caching~
- The commit body should details which files and functions have been changed and what exactly has been changed
  #+begin_quote
  - org-timer.el (org-timer-cancel-timer, org-timer-stop): Enhance
  message.
  (org-timer-set-timer): Use the number of minutes in the Effort
  property as the default timer value. Three prefix arguments will
  ignore the Effort value property.
  #+end_quote
  - *There is no need to list the function/file names manually here*
    - In magit commit message buffer, there is a diff window beside
    - From the diff window, pressing "c" on a chunk will insert the changed function name as needed
  - Use double space to separate sentences
  - Quote =`function-or-symbol-name'= for easier automatic analysis
  - It is a good practice to reference the related discussion in the mailing list
    #+begin_quote
    See discussion in https://list.orgmode.org/orgmode/87levyzwsk.fsf@localhost/
    #+end_quote
  - *If you haven't signed copyright assignment with FSF, put*
    #+begin_quote
    TINYCHANGE
    #+end_quote
    at the end of the commit message
    - _Beware that patches >15LOC require FSF copyright assignment_

*** Copyright assignment with FSF

To contribute significant (>15LOC) patches, we have a legal requirement that you transfer the copyright to FSF.

All the details in https://orgmode.org/worg/org-contribute.html#copyright

- It is generally fairly easy unless you employer has weird policies
- Just send https://orgmode.org/request-assign-future.txt form to assign@gnu.org
  - They usually reply within short time
  - If there is no reply 1 month, feel free to contact Org mailing list to assist