Upstream synchronization documentation

Upstream synchronization documentation

[Rasmus]

[-- Attachment #1: Type: text/plain, Size: 396 bytes --]

Hi,

The documentation in README_maintainer is a bit outdated
wrt. synchronizing with upstream Emacs.

I have updated the to best of my knowledge in the attached patch.  I would
be happy to push it, if there are no objections.

It would also be great if someone would let me know about the mistakes
that are undoubtedly in the draft.

Rasmus

-- 
I hear there's rumors on the, uh, Internets. . .

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Update-README_maintainer-with-upstream-synchronizati.patch --]
[-- Type: text/x-diff, Size: 4012 bytes --]

From 2126076308e13efbacf115afc3856d2c1da541fa Mon Sep 17 00:00:00 2001
From: Rasmus <rasmus@gmx.us>
Date: Mon, 3 Jul 2017 11:48:58 +0200
Subject: [PATCH] Update README_maintainer with upstream synchronization
 instructions

* README_maintainer: Update with upstream synchronization instructions.
---
 README_maintainer | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 57 insertions(+), 4 deletions(-)

diff --git a/README_maintainer b/README_maintainer
index 6b162aa52..bf2678da3 100644
--- a/README_maintainer
+++ b/README_maintainer
@@ -88,9 +88,62 @@ Org and contributed libraries.
 
 org-latest* snapshots are built from the *master* branch.
 
-* Synchronization with Emacs
-
-** Updating etc/ORG-NEWS
+* Synchronization Org and upstream Emacs
+Org should be kept in sync with the upstream [[http://git.savannah.gnu.org/cgit/emacs.git/tree/][Emacs repository]].
+Sometimes Org is changed in the Emacs upstream repo.  These changes
+should be backported.  Likewise, new stable releases of Org should be
+added to Emacs.
+** Backporting changes from upstream Emacs
+To check for backports from the Emacs upstream repository, one can use
+the following =git= command, cutesy of [[http://permalink.gmane.org/gmane.emacs.devel/215861][Kyle Meyer]],
+
+: git log $rev..origin/emacs-25 -- lisp/org doc/misc/org.texi etc/refcards/orgcard.tex
+
+where =$rev= is the last commit from the =emacs-25= branch that was
+backported.  The same should be done for the =master= branch.
+
+One may also use this [[http://git.savannah.gnu.org/cgit/emacs.git/atom/lisp/org/][Atom feed]] to see upstream changes.
+** Updating the Org version in upstream Emacs
+After a new release of Org, it should be synced to the Emacs
+repository.
+
+Typically, Org can be synchronized by copying over files from the
+=emacs-sync= branch of the Org repository to the =master= branch of Emacs
+repository.  The =emacs-sync= branch has a few extra changes compared to
+the =maint= branch.  If the Emacs maintainers are planning a new release
+of Emacs soon, it is possible that another branch, e.g. =emacs-25=,
+should be used.
+
+If you are synchronizing a major release of Org, it may be useful to
+use a separate branch before merging, e.g. =scratch/org-mode-merge=.
+This can then later be merged with the =master= branch, when everything
+has been tested.
+
+Please also see [[http://git.savannah.gnu.org/cgit/emacs.git/tree/CONTRIBUTE][CONTRIBUTE]] in the Emacs repository.
+*** Where to files go
+The following detail where files in Org repository are copied to in
+the Emacs repository.
+*** =org-mode/doc=
+- =org.texi= :: Copy to =emacs/doc/misc=.  It may be necessary to replace,
+		~@include org-version.inc~ #+end_src with ~@set VERSION 9.0.9~
+		or similar. 
+- =orgcard.tex= :: Copy to =emacs/doc/refcards=.  Make sure that
+		   ~\def\orgversionnumber~ and ~\def\versionyear~ are up
+		   to date, if necessary.
+- =library-of-babel.org= :: Copy to =emacs/etc/org=.
+***  =org-mode/etc=
+- =styles/*= :: Copy to =emacs/etc/org=.
+- =schema/*.rnc= :: Copy to =emacs/etc/schema=.
+- =schema/schemas.xml= :: New entries of this files should be added to
+     =emacs/etc/schema/schemas.xml=.
+- =ORG-NEWS= :: Copy to =emacs/etc=
+*** =org-mode/lisp=
+- Copy =*.el= files to  =emacs/lisp/org=, except =org-loaddefs.el=!
+- You should create =org-version.el= in =emacs/lisp/org=.  The file is
+  created when you =make= Org.
+*** TODO =org-mode/testing=
+** Outdated notes
+*** Updating etc/ORG-NEWS
 
 Latest changes in Emacs are described in Emacs =etc/NEWS=, and latest
 changes in major Emacs packages are described in =etc/ORG-NEWS=. 
@@ -100,7 +153,7 @@ always should), you need to update Org's =etc/ORG-NEWS= file so that
 you can merge it with that of Emacs.  There is one top-level section
 for each release that is merged with Emacs.
 
-** Merging with Emacs trunk branch
+*** Merging with Emacs trunk branch
 
 This is still a significant headache.  Some hand work is needed here.
 
-- 
2.13.2

Re: Upstream synchronization documentation

[Neil Jerram]

Hi Rasmus,

On 03/07/17 11:35, Rasmus wrote:
> Hi,
>
> The documentation in README_maintainer is a bit outdated
> wrt. synchronizing with upstream Emacs.
>
> I have updated the to best of my knowledge in the attached patch.  I would
> be happy to push it, if there are no objections.
>
> It would also be great if someone would let me know about the mistakes
> that are undoubtedly in the draft.
>
> Rasmus
>

I just noticed that your text talks about 'backporting', and wondered if 
that is the clearest term to use.  To me, a 'backport' usually means 
porting a change or fix from the master or development branch to some 
older or more stable branch.  Whereas in your text I think it is the 
other way round, isn't it?  (I.e. the Emacs branch is more stable, and 
you are talking about porting a fix that someone has made in that branch 
to the Org master.)  So perhaps 'forward port' would be clearer?

Regards - Neil

Re: Upstream synchronization documentation

[Kyle Meyer]

Neil Jerram <neil@ossau.homelinux.net> writes:

> I just noticed that your text talks about 'backporting', and wondered if 
> that is the clearest term to use.  To me, a 'backport' usually means 
> porting a change or fix from the master or development branch to some 
> older or more stable branch.

Right.  Thanks for bringing this up.

> Whereas in your text I think it is the other way round, isn't it?
> (I.e. the Emacs branch is more stable, and you are talking about
> porting a fix that someone has made in that branch to the Org master.)
> So perhaps 'forward port' would be clearer?

I suspect that Org's maint (where the Emacs changes land) is generally
more stable than the Org in Emacs's master, but, yes, Emacs's version is
the older version.  (Well, with v9.0.9 just synced the versions match,
but maint still has quite a few more commits.)

Since before I took over "backporting" changes from the Emacs repo, it's
been referred to as this.  Although I agree it isn't great word choice,
I'd prefer that we remain consistent so that, for example, "git log -i
--grep=backport" remains informative.

But if people think using "backport" is too confusing, I'm OK switching
to another term.  Of "forward port" and "propagate" (suggested in this
thread by Eric), I prefer "propagate"---or maybe just "port", though
grepping for that might lead to too many false positives.  And if we
stick with "backport", it still might be a good idea to clarify in
README_maintainer that we're abusing the term.

-- 
Kyle

Re: Upstream synchronization documentation

[Rasmus Pank Roulund]

Kyle Meyer <kyle@kyleam.com> writes:

>> Whereas in your text I think it is the other way round, isn't it?
>> (I.e. the Emacs branch is more stable, and you are talking about
>> porting a fix that someone has made in that branch to the Org master.)
>> So perhaps 'forward port' would be clearer?
>
> I suspect that Org's maint (where the Emacs changes land) is generally
> more stable than the Org in Emacs's master, but, yes, Emacs's version is
> the older version.  (Well, with v9.0.9 just synced the versions match,
> but maint still has quite a few more commits.)
>
> Since before I took over "backporting" changes from the Emacs repo, it's
> been referred to as this.  Although I agree it isn't great word choice,
> I'd prefer that we remain consistent so that, for example, "git log -i
> --grep=backport" remains informative.
>
> But if people think using "backport" is too confusing, I'm OK switching
> to another term.  Of "forward port" and "propagate" (suggested in this
> thread by Eric), I prefer "propagate"---or maybe just "port", though
> grepping for that might lead to too many false positives.  And if we
> stick with "backport", it still might be a good idea to clarify in
> README_maintainer that we're abusing the term.

So at least I’m not crazy for "coming up with" it backporting!

So I will keep calling it "backporting" but explain that it is more like
propagating changes from the Emacs repository (back) to the Org
repository.

Thanks,
Rasmus

-- 
Dung makes an excellent fertilizer

Re: Upstream synchronization documentation

[Neil Jerram]

No problem to stick with 'backport' if that is the conventional term here. Thanks for considering and explaining the point. 

     Neil 


  Original Message  
From: Rasmus Pank Roulund
Sent: Monday, 3 July 2017 18:22
To: kyle@kyleam.com
Cc: neil@ossau.homelinux.net; emacs-orgmode@gnu.org
Subject: Re: [O] Upstream synchronization documentation

Kyle Meyer <kyle@kyleam.com> writes:

>> Whereas in your text I think it is the other way round, isn't it?
>> (I.e. the Emacs branch is more stable, and you are talking about
>> porting a fix that someone has made in that branch to the Org master.)
>> So perhaps 'forward port' would be clearer?
>
> I suspect that Org's maint (where the Emacs changes land) is generally
> more stable than the Org in Emacs's master, but, yes, Emacs's version is
> the older version. (Well, with v9.0.9 just synced the versions match,
> but maint still has quite a few more commits.)
>
> Since before I took over "backporting" changes from the Emacs repo, it's
> been referred to as this. Although I agree it isn't great word choice,
> I'd prefer that we remain consistent so that, for example, "git log -i
> --grep=backport" remains informative.
>
> But if people think using "backport" is too confusing, I'm OK switching
> to another term. Of "forward port" and "propagate" (suggested in this
> thread by Eric), I prefer "propagate"---or maybe just "port", though
> grepping for that might lead to too many false positives. And if we
> stick with "backport", it still might be a good idea to clarify in
> README_maintainer that we're abusing the term.

So at least I’m not crazy for "coming up with" it backporting!

So I will keep calling it "backporting" but explain that it is more like
propagating changes from the Emacs repository (back) to the Org
repository.

Thanks,
Rasmus

-- 
Dung makes an excellent fertilizer

Re: Upstream synchronization documentation

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 130 bytes --]

And "propagate" would arguably be better than "forward port"?
-- 
: Eric S Fraga via Emacs 26.0.50, Org release_9.0.9-551-g92e8c8

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: Upstream synchronization documentation

[Kyle Meyer]

Rasmus <rasmus@gmx.us> writes:

> The documentation in README_maintainer is a bit outdated
> wrt. synchronizing with upstream Emacs.

Thanks for taking the time to update this.

> +** Backporting changes from upstream Emacs
> +To check for backports from the Emacs upstream repository, one can use
> +the following =git= command, cutesy of [[http://permalink.gmane.org/gmane.emacs.devel/215861][Kyle Meyer]],
                                ^courtesy
> +
> +: git log $rev..origin/emacs-25 -- lisp/org doc/misc/org.texi etc/refcards/orgcard.tex

This pathspec will catch nearly all the changes that should be
considered for backporting.  In the process of this sync, I realized
that the check should include a few more files for completeness.  I now
add etc/ORG-NEWS and etc/org to the pathspec above.  On top of that, the
latest sync introduces od-manifest-schema-v1.2-os.rnc and
od-schema-v1.2-os.rnc to Emacs's etc/schema directory, so I've added
those too.

Anyway, that's just for the record.  Together these files should be the
source of very few backports, but I think they're worth monitoring to be
to make sure we don't need to deal with any changes on the Emacs-side.
I'm not sure if you want to document the extended and unwieldy pathspec
here, but it would be

  -- lisp/org doc/misc/org.texi etc/refcards/orgcard.tex etc/ORG-NEWS etc/org \
  etc/schema/od-manifest-schema-v1.2-os.rnc etc/schema/od-schema-v1.2-os.rnc

> +where =$rev= is the last commit from the =emacs-25= branch that was
> +backported.  The same should be done for the =master= branch.

[...]

> +- =org.texi= :: Copy to =emacs/doc/misc=.  It may be necessary to replace,
> +		~@include org-version.inc~ #+end_src with ~@set VERSION 9.0.9~

Leftover "#+end_src" from a previous edit?

> +** Outdated notes

Instead of creating this heading, should we just delete the outdated
notes?

-- 
Kyle

Re: Upstream synchronization documentation

[Rasmus]

[-- Attachment #1: Type: text/plain, Size: 664 bytes --]

Hi,

Thanks for the comments.

>   -- lisp/org doc/misc/org.texi etc/refcards/orgcard.tex etc/ORG-NEWS etc/org \
>   etc/schema/od-manifest-schema-v1.2-os.rnc etc/schema/od-schema-v1.2-os.rnc

That's better.

>> +- =org.texi= :: Copy to =emacs/doc/misc=.  It may be necessary to replace,
>> +		~@include org-version.inc~ #+end_src with ~@set VERSION 9.0.9~
>
> Leftover "#+end_src" from a previous edit?

Yes, indeed.  Thanks.

>> +** Outdated notes
>
> Instead of creating this heading, should we just delete the outdated
> notes?

Deleting is fine by me.  I don't know if it is useful to anyone else.
Probably not.

Rasmus

--
⠠⠵

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Update-README_maintainer-with-upstream-synchronizati.patch --]
[-- Type: text/x-diff, Size: 4456 bytes --]

From 556c2937abc5b3fe0f5b9e8861a30aef748215cc Mon Sep 17 00:00:00 2001
From: Rasmus <rasmus@gmx.us>
Date: Mon, 3 Jul 2017 11:48:58 +0200
Subject: [PATCH] Update README_maintainer with upstream synchronization
 instructions

* README_maintainer: Update with upstream synchronization instructions.
---
 README_maintainer | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 66 insertions(+), 4 deletions(-)

diff --git a/README_maintainer b/README_maintainer
index 6b162aa52..60821059c 100644
--- a/README_maintainer
+++ b/README_maintainer
@@ -88,9 +88,71 @@ Org and contributed libraries.
 
 org-latest* snapshots are built from the *master* branch.
 
-* Synchronization with Emacs
-
-** Updating etc/ORG-NEWS
+* Synchronization Org and upstream Emacs
+Org should be kept in sync with the upstream [[http://git.savannah.gnu.org/cgit/emacs.git/tree/][Emacs repository]].
+Sometimes Org is changed in the Emacs upstream repo.  These changes
+should be backported.  Likewise, new stable releases of Org should be
+added to Emacs.
+** Backporting changes from upstream Emacs
+Sometimes Emacs maintainers make changes to Org files.  The process of
+propagating the changes back to the Org repository is called
+/backporting/ for historical reasons.
+
+To check for changes that needs to be backported from the Emacs
+repository, one can use the following =git= command, courtesy of [[http://permalink.gmane.org/gmane.emacs.devel/215861][Kyle
+Meyer]],
+#+begin_src shell
+  git log $rev..origin/emacs-25 -- lisp/org doc/misc/org.texi \
+    etc/refcards/orgcard.tex etc/ORG-NEWS etc/org \
+    etc/schema/od-manifest-schema-v1.2-os.rnc \
+    etc/schema/od-schema-v1.2-os.rnc
+#+end_src
+where =$rev= is the last commit from the =emacs-25= branch that was
+backported.  The should also be done for the =master= branch.
+
+One may also use Atom feeds to keep track of upstream changes,
+e.g. [[http://git.savannah.gnu.org/cgit/emacs.git/atom/lisp/org/][this feed]] show changes to the =lisp/org= folder.
+** Updating the Org version in upstream Emacs
+After a new release of Org, it should be synced to the Emacs
+repository.
+
+Typically, Org can be synchronized by copying over files from the
+=emacs-sync= branch of the Org repository to the =master= branch of Emacs
+repository.  The =emacs-sync= branch has a few extra changes compared to
+the =maint= branch.  If the Emacs maintainers are planning a new release
+of Emacs soon, it is possible that another branch, e.g. =emacs-25=,
+should be used.
+
+If you are synchronizing a major release of Org, it may be useful to
+use a separate branch before merging, e.g. =scratch/org-mode-merge=.
+This can then later be merged with the =master= branch, when everything
+has been tested.
+
+Please also see [[http://git.savannah.gnu.org/cgit/emacs.git/tree/CONTRIBUTE][CONTRIBUTE]] in the Emacs repository.
+*** Where to files go
+The following detail where files in Org repository are copied to in
+the Emacs repository.
+*** =org-mode/doc=
+- =org.texi= :: Copy to =emacs/doc/misc=.  It may be necessary to replace,
+                ~@include org-version.inc~ with ~@set VERSION 9.0.9~ or
+                similar.
+- =orgcard.tex= :: Copy to =emacs/doc/refcards=.  Make sure that
+                   ~\def\orgversionnumber~ and ~\def\versionyear~ are up
+                   to date, if necessary.
+- =library-of-babel.org= :: Copy to =emacs/etc/org=.
+***  =org-mode/etc=
+- =styles/*= :: Copy to =emacs/etc/org=.
+- =schema/*.rnc= :: Copy to =emacs/etc/schema=.
+- =schema/schemas.xml= :: New entries of this files should be added to
+     =emacs/etc/schema/schemas.xml=.
+- =ORG-NEWS= :: Copy to =emacs/etc=
+*** =org-mode/lisp=
+- Copy =*.el= files to  =emacs/lisp/org=, except =org-loaddefs.el=!
+- You should create =org-version.el= in =emacs/lisp/org=.  The file is
+  created when you =make= Org.
+*** TODO =org-mode/testing=
+** Outdated notes
+*** Updating etc/ORG-NEWS
 
 Latest changes in Emacs are described in Emacs =etc/NEWS=, and latest
 changes in major Emacs packages are described in =etc/ORG-NEWS=. 
@@ -100,7 +162,7 @@ always should), you need to update Org's =etc/ORG-NEWS= file so that
 you can merge it with that of Emacs.  There is one top-level section
 for each release that is merged with Emacs.
 
-** Merging with Emacs trunk branch
+*** Merging with Emacs trunk branch
 
 This is still a significant headache.  Some hand work is needed here.
 
-- 
2.13.2