From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jambunathan K Subject: Re: [PATCH] (ELPA): Support for building Package Variants and Package Upload Date: Fri, 12 Nov 2010 22:23:36 +0530 Message-ID: <814obmlbzj.fsf@gmail.com> References: <81wroiof9p.fsf@gmail.com> <9F5332A9-7CFB-49C3-8430-A7E193B4515C@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from [140.186.70.92] (port=55735 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1PGwsr-0000p5-MF for emacs-orgmode@gnu.org; Fri, 12 Nov 2010 11:53:55 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1PGwsp-0002eI-Mc for emacs-orgmode@gnu.org; Fri, 12 Nov 2010 11:53:53 -0500 Received: from mail-yx0-f169.google.com ([209.85.213.169]:60230) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1PGwsp-0002e4-DV for emacs-orgmode@gnu.org; Fri, 12 Nov 2010 11:53:51 -0500 Received: by yxf34 with SMTP id 34so1585401yxf.0 for ; Fri, 12 Nov 2010 08:53:50 -0800 (PST) In-Reply-To: <9F5332A9-7CFB-49C3-8430-A7E193B4515C@gmail.com> (Carsten Dominik's message of "Fri, 12 Nov 2010 09:09:18 -0600") List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Carsten Dominik Cc: bzg@altern.org, emacs-orgmode@gnu.org Carsten Dominik writes: > Hi Jambunathan, > > thanks for this! > >> 2.2 Build Procedure >> ==================== >> 1. make PKG_TAG=7.3 >> This builds org-7.3.tar > > I guess this must be > > make pkg PKG_TAG=... > > right? > Yes, I missed the target - 'pkg'. A small but a significant slip. > I think it would be good to *not* use a name that is just like the > normal org distribution files. There is a config variable for that :-). It is PKG_BNAME (as in 'Package Base Name'). > I am distributing under the names > > org-7.3.tar.gz > org-7.3.zip > I believe the question here is: How is an ELPA-tar differntiated from a plain-old snapshot tar? From the user perspective, is there an element of ambiguity and confusion? As things stand today, ELPA-tarballs are plain tar files i.e., they are not zipped. So the packages with above names can only be non-ELPA tarballs [1] The only difference between ELPA and non-ELPA tarballs is in the tree structure. ELPA-tarballs *require* that lisp files and info files be in the root directory [2]. This is not the case with plain old tarballs. As it stands today, the 'contrib' files are not zipped in to the ELPA tarball [3]. One approach of addressing a possible divide between ELPA and non-ELPA tarballs is to distribute just a single tarball which is also ELPA-tarball. This would require a modification in the tree-structure of the conventional tarball and may be some changes to the Makefile. A second approach of addressing the problem is to wait and watch. May be by the time Emacs-24 gets out of the door, package manager would not be too restrictive on where it wants to see the elisp and info files. And a plain old tar file could as well work well with package manager. > I am distributing under the names > > org-7.3.tar.gz > org-7.3.zip > > or is this not a possible problem? The advantage of using 'org' namespace for the ELPA tarballs is that it would be deemed as an 'upgrade' to the builtin package 'org'. In that sense there is a continuity. This privilege would be lost for packages that doesn't share 'builtins' namespace [4]. As package are identified by their names, a org-snapshot-7.3.tar is considered by package manager interface as different from org-7.3.tar. AFAICS, packages get in to '~/.emacs.d/elpa/org-snapshot-7.3/...' and '~/.emacs.d/elpa/org-7.3/...' and load-path is appropriately modified. So a reasonably experienced user can have them installed 'in parallel' and pick and choose what he uses by playing around with load-path. He can also tie down the package manager not to activate the files, thus minimizing the package manager functionality to that of a downloader, compiler and unpacker but not as a 'loader' [5]. ,----[ C-h f package-initialize RET ] | package-initialize is an interactive Lisp function in `package.el'. | | (package-initialize &optional NO-ACTIVATE) | | Load Emacs Lisp packages, and activate them. | The variable `package-load-list' controls which packages to load. | If optional arg NO-ACTIVATE is non-nil, don't activate packages. | | [back] `---- > I am distributing under the names > > org-7.3.tar.gz > org-7.3.zip One another way to interpret the question is how would one distinguish between tarballs that are 'official' (as in released directly by the maintainer from his official website) and those that are 'unofficial' (as in forks and derived works). For the sake of recording my Makefile changes, as well as for the sake of argument, I could potentially use the following setting: PKG_BNAME=org PKG_SEP=@ PKG_FLAVOUR=OpenOffice and get a org@OpenOffice-20101112.tar. This is where the notion of 'Variants/Flavours/Release Streams/' could be useful. The string between '@' and 'the version number' could be standardized as a variant. The advantage of variants is obvious [6]. [1] Possible that Package Manager might start supporting zipped files. [2] This is a constraint only because 'package manager' is barely born. Who knows it can become a superman very soon ... [3] This is so because just because it was done so. Package Manager provides support for packaging arbitrary data files - icons, sound files etc etc. There is no reason (apart from download size) which prevent one from including contrib files. [4] I haven't experimented much with builtin packages, upgrades, downgrades etc. Take my remarks here with a pinch of salt. May be someone who has experimented with package manager could pitch in with additional insights here ... I am forming my opinion as I go along. [5] In some sense package manager is a good 'eye candy'. It is not my intention to use the word in a negative sense but only as a convenient metaphor. [6] This is something that I have proposed to emacs-devel. Variants etc are purely my own terminology i.e., it doesn't have any official sanction. See http://lists.gnu.org/archive/html/emacs-devel/2010-10/msg01026.html This is also one of the reasons why I had commented out 'PKG_SEP' and 'PKG_FLAVOURS'. One of biggest advantage of having variants is to support both snapshot and stable tarballs that could be simultaneously released with obvious advantages. Jambunathan K. > > - Carsten > > >> 2. make pkg >> This builds org-20101112.tar >> >> 3 Building Package Variants >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> 3.1 Notes >> ========== >> Modify/Uncomment PKG_SEP and PKG_FLAVOUR setting in Makefile as >> desired. >> >> There is a choice of PKG_SEP. Pick what you like. Use of '-' is >> discouraged. >> >> Assumption is: >> - stable releases are built with an explicit PKG_TAG >> - stable releases are non-flavored while the snapshot builds are >> flavored. >> >> 3.2 Build Procedure >> ==================== >> 1. make PKG_TAG=7.3 >> This builds org-7.3.tar >> 2. make pkg >> This builds org@snaphsot-20101112.tar >> >> 4 Package Uploads >> ~~~~~~~~~~~~~~~~~~ >> >> 4.1 Notes >> ========== >> - PKG_UPLOAD_DIR: directory where the package files are >> uploaded. i.e., it is the *local directory* behind the repo URL >> ([http://orgmode.org/pkg/daily/]). For eg, /var/www/pub/pkg/ >> daily/. >> >> IMPORTANT NOTE: Remember to include the trailing '/' in the >> directory while setting this variable. >> >> - PKG_EL_DIR: This is added to the load-path during BATCH >> compilation. The intention is two fold: >> >> 1. load package.el and package-x.el if the build machine is using >> Emacs-23 or less (which don't ship with package manager) >> >> 2. Override package.el and package-x.el that ship with Emacs with >> a custom or a patched up version. >> >> IMPORTANT NOTE: The rule for 'pkg-upload' depends on my patch to >> Package Manager which is accepted but not committed yet because >> of the (postal?) delay in FSF copyright requirements. >> >> [http://lists.gnu.org/archive/html/emacs-devel/2010-11/msg00026.html >> ] >> >> >> 4.2 Build Procedure >> ==================== >> >> 1. make PKG_EL_DIR="~/elisp" PKG_UPLOAD_DIR="~/packages/" >> PKG_TAG=7.3 pkg >> >> This builds org-7.3.tar and uploads the same to "~/packages" >> using package-x.el in "~/elisp/" >> >> 2. make PKG_EL_DIR="~/elisp" PKG_UPLOAD_DIR="~/packages/" pkg >> >> This builds org-20101112.tar and uploads the same to >> "~/packages" using package-x.el in "~/elisp/" >> >> Jambunathan K. >> >> Attachments >> ~~~~~~~~~~~ >> >> From 06d227cc4d4e3ac86320690ebbe184bb804d9bf1 Mon Sep 17 00:00:00 2001 >> From: Jambunathan K >> Date: Fri, 12 Nov 2010 16:32:51 +0530 >> Subject: [PATCH] (ELPA): Support for building Package Variants and >> Package Upload. >> >> (PKG_TAG): Use PKG_TAG to control package flavour. If unspecified, >> assume a snapshot build. >> >> (PKG_SEP, PKG_FLAVOUR): New config variables for building various >> flavours of ELPA tarball. For eg., org-7.3.tar, >> org@snapshot-20101112.tar, org-snapshot-20101112 etc. >> >> (PKG_FILES): Removed >> (PKG_BASE_FILES): Files here end up in the root dir. >> (PKG_DOC_FILES): Files here end up in the doc dir. >> >> (PKG_README_ORG_FILE): README file >> >> (PKG_EL_DIR): Load path for package.el and package-x.el. Useful if the >> build machine runs an 'old' Emacs or if the build process requires a >> patched up versions of these files. >> >> (PKG_UPLOAD_DIR): 'pub' dir of the webserver serving the ELPA tarball. >> >> (pkg-upload, pkg-readme): New targets subsumed within pkg target. >> --- >> diff --git a/Makefile b/Makefile >> old mode 100644 >> new mode 100755 >> index 0b15abe..5c13d8b >> --- a/Makefile >> +++ b/Makefile >> @@ -165,18 +165,44 @@ TEXIFILES = doc/org.texi >> INFOFILES = doc/org >> >> # Package Manager (ELPA) >> + >> +# PKG_TAG = >> + >> +ifndef PKG_TAG >> +# PKG_SEP = @ >> +# PKG_SEP = - >> +# PKG_SEP = . >> +# PKG_FLAVOUR = snapshot >> PKG_TAG = $(shell date +%Y%m%d) >> +endif >> + >> +PKG_BNAME = org >> +PKG_QNAME = $(PKG_BNAME)$(PKG_SEP)$(PKG_FLAVOUR) >> +PKG_FNAME = $(PKG_QNAME)-$(PKG_TAG) >> + >> PKG_DOC = "Outline-based notes management and organizer" >> PKG_REQ = "nil" >> >> -PKG_FILES = $(LISPFILES0) \ >> - doc/dir doc/org \ >> - doc/pdflayout.sty \ >> - doc/org.pdf \ >> - doc/orgguide.pdf \ >> - doc/orgcard.tex \ >> - doc/orgcard.pdf \ >> - doc/orgcard_letter.pdf >> +PKG_BASE_FILES = $(LISPFILES0) \ >> + doc/dir doc/org >> + >> +PKG_DOC_FILES = \ >> + doc/org.pdf \ >> + doc/org.html \ >> + doc/orgguide.pdf \ >> + doc/orgcard.txt \ >> + doc/orgcard.pdf \ >> + doc/orgcard_letter.pdf >> + >> +PKG_README_ORG_FILE = ORGWEBPAGE/Changes.org >> +PKG_README_TXT_FILE = $(PKG_README_ORG_FILE:%.org=%.txt) >> + >> +# PKG_EL_DIR = >> +# PKG_UPLOAD_DIR = >> + >> +ifdef PKG_EL_DIR >> +BATCH_EXTRA = -eval "(setq load-path (cons (expand-file-name \"$ >> (PKG_EL_DIR)\") load-path))" >> +endif >> >> .SUFFIXES: .el .elc .texi >> SHELL = /bin/sh >> @@ -372,16 +398,46 @@ distfile: >> zip -r org-$(TAG).zip org-$(TAG) >> gtar zcvf org-$(TAG).tar.gz org-$(TAG) >> >> -pkg: >> +pkg: >> @if [ "X$(PKG_TAG)" = "X" ]; then echo "*** No tag ***"; exit 1; fi >> + echo "Building $(PKG_FNAME).tar ..." >> touch doc/org.texi doc/orgcard.tex # force update >> ${MAKE} info >> ${MAKE} doc >> - rm -rf org-$(PKG_TAG) org-$(PKG_TAG).tar >> - $(MKDIR) org-$(PKG_TAG) >> - cp -r $(PKG_FILES) org-$(PKG_TAG) >> - echo "(define-package \"org\" \"$(PKG_TAG)\" \"$(PKG_DOC)\" $ >> (PKG_REQ))" > org-$(PKG_TAG)/org-pkg.el >> - tar cf org-$(PKG_TAG).tar org-$(PKG_TAG) --remove-files >> + rm -rf $(PKG_FNAME) $(PKG_FNAME).tar >> + $(MKDIR) $(PKG_FNAME) >> + cp -r $(PKG_BASE_FILES) $(PKG_FNAME) >> + $(MKDIR) $(PKG_FNAME)/doc >> + cp -r $(PKG_DOC_FILES) $(PKG_FNAME)/doc >> + ${MAKE} pkg-readme >> + echo "(define-package \"$(PKG_QNAME)\" \"$(PKG_TAG)\" >> \"$(PKG_DOC) >> \" $(PKG_REQ))" > $(PKG_FNAME)/$(PKG_QNAME)-pkg.el >> + tar cf $(PKG_FNAME).tar $(PKG_FNAME) --remove-files >> + ${MAKE} pkg-upload >> + >> +pkg-readme: >> + $(MKDIR) $(PKG_FNAME) >> + cp $(PKG_README_ORG_FILE) $(PKG_FNAME)/README.org >> + $(BATCH) \ >> + --eval "(require 'org-exp)" \ >> + --eval "(require 'org-ascii)" \ >> + --visit=$(PKG_FNAME)/README.org \ >> + --eval "(org-export-as-ascii 3 t)" >> + cp $(PKG_FNAME)/README.txt $(PKG_FNAME)/README >> + rm -f $(PKG_FNAME)/README.txt $(PKG_FNAME)/README.org >> + >> +pkg-upload: >> + @if [ "X$(PKG_UPLOAD_DIR)" = "X" ]; \ >> + then echo "*** WARNING: Skipping Upload of $(PKG_FNAME).tar ***"; \ >> + else \ >> + echo "Uploading $(PKG_FNAME).tar to $(PKG_UPLOAD_DIR) ..."; \ >> + $(MKDIR) $(PKG_UPLOAD_DIR); \ >> + $(BATCH) $(BATCH_EXTRA) \ >> + --eval '(ignore)' \ >> + --eval "(require 'package-x)" \ >> + --eval '(setq package-archive-upload-base "$(PKG_UPLOAD_DIR)")' \ >> + --eval '(setq package-update-news-on-upload nil)' \ >> + --eval '(package-upload-file "$(PKG_FNAME).tar")'; \ >> + fi >> >> makerelease: >> @if [ "X$(TAG)" = "X" ]; then echo "*** No tag ***"; exit 1; fi >> -- >> 1.7.2.3 >> >> _______________________________________________ >> Emacs-orgmode mailing list >> Please use `Reply All' to send replies to the list. >> Emacs-orgmode@gnu.org >> http://lists.gnu.org/mailman/listinfo/emacs-orgmode