From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tim Cross Subject: Re: [Idea] Org Collections Date: Mon, 16 Dec 2019 23:28:51 +1100 Message-ID: <87sglko3ng.fsf@gmail.com> References: <87tv60ldf2.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:52589) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1igpUi-0000Nb-TS for emacs-orgmode@gnu.org; Mon, 16 Dec 2019 07:29:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1igpUg-0007Bd-4c for emacs-orgmode@gnu.org; Mon, 16 Dec 2019 07:29:00 -0500 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]:38754) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1igpUf-00078F-PX for emacs-orgmode@gnu.org; Mon, 16 Dec 2019 07:28:58 -0500 Received: by mail-pj1-x1034.google.com with SMTP id l4so2912004pjt.5 for ; Mon, 16 Dec 2019 04:28:57 -0800 (PST) Received: from tim-desktop (220-235-169-176.dyn.iinet.net.au. [220.235.169.176]) by smtp.gmail.com with ESMTPSA id m15sm21755282pgi.91.2019.12.16.04.28.53 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Dec 2019 04:28:55 -0800 (PST) In-reply-to: <87tv60ldf2.fsf@gmail.com> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Sender: "Emacs-orgmode" To: emacs-orgmode@gnu.org I would have to say the hardest thing I ever have to do as a developer is naming things. It is hard enough to do within a context of a single group, even harder when speaking about something with a global user base (language, social/cultural differences etc). Despite this, it is so very important as it defines expectations, which will in turn determine satisfaction.=20 As an example, 'aspects' for me is more like a different view into a 'thing' while collections are more like distinct, separate collections of 'things'. To some extent, aspects feels like a 'virtual collection' where collection is more like a concrete separation. I can see pros and cons with both.=20=20 Roland Everaert writes: > +1 for this idea. > > You speak about one document used by multiple collections, how do you > plan to manage that from a file system point of view? > > How will be organized a collection, still from the FS point of view? > > As some are delving into the abyss of sementic, I propose aspects > instead of collections or contexts. Ultimately we are trying to manage > various aspects of our life, by splitting those aspects into files or > diretories and what not. So, if it is the intent of your idea, the term > aspect seems more appropriate than collection or context IMHO. > > Did you think about the specific UI of aspects management? > Proposal of UI I particularly like: > - Mu4E > - forge/magit > > How to keep track of all those aspects? > > I will surely have more to say, but, as of know I am at work. > > Regards, > > Roland. > > Gustav Wikstr=C3=B6m writes: > >> Hi list and all honored readers! >> >> I have an idea. One that I've mentioned before in side notes. And I want= to emphasize that this still only is an idea. But I want to present this i= dea to you. As a way to gather feedback and get input. Maybe the idea is st= upid!? Maybe you think it's already solved? Or maybe it's not, and lots of = you resonate with it as well. In any case, please let me know what you thin= k on the piece below! >> >> ________________________________ >> >> ORG MODE: COLLECTIONS/PROJECTS >> >> Gustav Wikstr=C3=B6m >> ________________________________ >> >> >> Table of Contents >> _________________ >> >> 1. Motivation >> 2. Idea >> 3. Benefit >> .. 1. For the user >> .. 2. For the developer >> 4. Example use cases >> .. 1. Separate actions from reference >> .. 2. Work / Personal separation >> .. 3. Separated book library >> .. 4. More? >> 5. Risks and challenges >> .. 1. Which configuration to use? >> .. 2. Should project config allow local variables? >> ..... 1. How to initialize the local variables? >> .. 3. Conflict with other customizations >> .. 4. Files that belong to multiple collections >> .. 5. Dynamic lists of files and folders for a collection? >> 6. Alternatives >> 7. References >> >> >> 1 Motivation >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> Org mode is more than a major mode for emacs buffers. That has been >> clear for quite some time. Org mode can operate on sets of files. >> Consolidate TODO's and calendar information into custom views. Publish >> sets of files. To do this Org mode assumes that the user has a >> configuration for each of those features. Each feature is responsible >> for maintaining its own context. And almost all of that context has >> to be set globally. So even though Org mode has commands and features >> that operate on sets of files and folders it has not yet developed >> that in a congruent, extensible and composable way. Thus, for the >> sanity of our users and developers I think it's time to ... introduce >> another concept! One that hopefully can simplify things both for users >> and developers. >> >> >> 2 Idea >> =3D=3D=3D=3D=3D=3D >> >> I propose to introduce `Collection' as a concept in the realm of Org >> mode. [1] >> >> An Org mode collection is defined as the combination of: >> 1. A short name and description >> 2. A collection of Org mode documents >> 3. A collection of files and/or folders called attachments and >> attachment-locations for the project >> 4. A collection of configurations for the given project >> >> Globally available collections are defined in a list, >> `org-collections'. Org mode should include a safe parameter that can >> be set as a folder customization to the local active project, >> `org-collections-active'. The default should be to the first entry in >> `org-collections' unless customized. This local parameter would be >> used to instruct Emacs and Org mode on which collection is active. >> Only one collection at a time can be active. >> >> Org agenda should use `org-collections-active' as default for the >> collection of Org mode documents to operate on. Org agenda should get >> a new command to switch between active projects. >> >> I'm thinking that there could be a special Emacs major mode for the >> collection as well, called "Org collections mode". Not sure exactly >> what to display and how to represent the project there... But >> certainly some kind of list of included documents and attachments. >> When in that mode there should possibly be single key >> keyboard-shortcuts to the most important features that operate on the >> collection. And switch between them. >> >> >> 3 Benefit >> =3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> 3.1 For the user >> ~~~~~~~~~~~~~~~~ >> >> A user would gain mainly two benefits as I can see right now: >> 1. The ability to clearly define (multiple) collections of files that >> belong together across org mode, with unique configurations. >> 2. Less global configuration state to manage and worry about! >> >> The second point might not look like much but is sooo important! Most >> programmers know that global state should be avoided. Putting things >> in a context most of the time makes things better. And if we can >> configure Org mode connected to a context it makes it much more useful >> for those who use Org mode for multiple purposes. >> >> The first point is equally important in my opinion. Today one must >> configure Org mode per feature. If you want to configure publishing >> you do that globally. If you want to configure the agenda, you have to >> do that globally as well. If you want to define a location for >> attachments, do it globally! What about custom TODO-keywords? Do it >> globally! Track ID-locations? Define a location globally! >> >> All above adds cognitive load to the user and makes it difficult to >> maintain the configuration as the use of Org mode grows (as it should >> ;) ). You have to define the context for each and every feature for it >> to know what to operate on. I claim that both the human psyche and the >> system itself will have a much more easy time if it could configure >> these features together, in a given context! >> >> >> 3.2 For the developer >> ~~~~~~~~~~~~~~~~~~~~~ >> >> I claim there will be benefits for developers as well. Today there >> exists many packages that extend Org mode functionality. Many work >> with the idea of collections. Some that come to mind: >> - Org brain () >> - Org ql () >> - Ox hugo () >> >> I think that with the addition of the `collections' concept into Org >> mode, package developers get a concept they can easily attach to. Yes, >> you can easily define your own package-specific concept for that as >> well. But then the user loses out in having to configure another >> feature. And yes, today you as a developer can say that Org agenda >> will be my collection to operate on. But this is a big limitation >> since it limits what your package effectively can only work to a >> single list of files. >> >> Having a collections concept means you as a developer have another >> base on which you can extend. No need to define your own concept if >> `Org-agenda-files' isn't enough; make it work together with >> `org-collections' instead. Org mode users will be happy because what >> they have already defined as important for them can be reused for new >> things with ease. >> >> Developing features inside Org mode itself hopefully also can benefit >> from this concept. I'm sure there are many people out there with cool >> ideas on how to extend and work with Org documents. And I'm equally >> sure that the value of developing many of those features will be >> bigger if they could naturally attach to an Org collections >> definition! >> >> >> 4 Example use cases >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> 4.1 Separate actions from reference >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> One practice promoted by GTD is to separate actionable items from >> reference information. While that practice can be overcome by search >> etc. some might still value a clear separation. >> >> Want to look up something related to my general references? Search the >> Org collection related to reference-information! Maybe set up custom >> views and uses of TODO keywords for reference information for special >> agenda views. >> >> Want to only display not yet finished tasks? Switch to the Org >> collection for actionable items and browse away. >> >> >> 4.2 Work / Personal separation >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> The heading says it all. Some like to separate work and personal stuff >> out from each other. What more clear way to do that than can there be >> than to separate them into their own Org collections? That way you >> potentially could let your work-related workflow (I.e. TODO-keywords) >> be different than the personal workflow. Without having to think about >> a global configuration that has to allow for both. >> >> >> 4.3 Separated book library >> ~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> Org mode can be used as a media manager of sort. Just define your >> conventions for the Org collection using TODO-keywords, categories and >> properties. Attach the e-books you have as attachments in an >> attachment-scheme special for your book library. Configure export of >> the library using maybe a custom HTML/CSS-visual and publish it >> somewhere for yourself to look at when on your phone. And do this >> without having to think of how changing all these things will affect >> the global state of Org mode, potentially messing up your other uses >> for task management or other notes and libraries you're trying to >> manage! >> >> Note that one can still have a holistic view on all Org mode documents >> as well, if important. It only requires a definition of a collection >> as the collection of all other collections! >> >> >> 4.4 More? >> ~~~~~~~~~ >> >> Please add more ideas when you think of them! >> >> >> 5 Risks and challenges >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> 5.1 Which configuration to use? >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> When I'm visiting a file that belongs to a collection, how should >> Emacs resolve configurations for that file? >> >> There may be configurations in the following places: >> - Global in `emacs-custom.el' or `.emacs.d/init.el' >> - Directory local variables in the tree >> - File local variables >> - Local variables for the project definition in which the file >> belongs? >> >> Should visiting a file always have to scan the collections list to see >> if the file belongs to any of them, in order to load customizations? >> Hmm... Maybe!? Or - maybe not if Emacs can rely on the fact that the >> user cares to set the local variable `org-collections-active' (or >> whatever it should be called)? In that case, just evaluate the >> settings for that project without doing any scan. >> >> >> 5.2 Should project config allow local variables? >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> Should the collections definition allow customization of variables >> that apply for Org mode features? Hmm... Maybe!? One thing that comes >> to mind is that a project should be able to define a custom attachment >> directory... How else would the attachment-feature know what >> attachment directory to use for files in that collection? >> >> Another option could ofc. be that each feature would have to add >> support for looking into the collection definition and override the >> local variable. But that will add development effort and complexity to >> each feature. Not suggested. >> >> >> 5.2.1 How to initialize the local variables? >> -------------------------------------------- >> >> When visiting a file that belongs to a collection, should Emacs at >> that point initialize the collection-configuration for that >> collection? Ideally some kind of collection-resolution would be made. >> Otherwise users will get strange behaviors when the think they are in >> one project but Org mode hasn't changed the local variables to match >> it. On the other hand, it doesn't sound very performant to have to >> check collection-belonging every time an Org mode file is visited! >> >> Possibly solve this with a variable that can be localized - >> `org-collections-active'? >> >> >> 5.3 Conflict with other customizations >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> Maybe I've defined an attachment directory as a directory local >> variables in a folder, for all subfolders and files to inherit. Should >> collection-customizations override that? Or should the directory local >> variables take precedence? >> >> Maybe could be solved by letting the (advanced) user choose using a >> customization itself, something like >> `org-collections-precede-local-variables' ? Need a intuitive default >> though. Most sane default is probably to let local variables take >> precedence. Those are created by the user anyways, so she should be >> aware. >> >> The more I think of it, there shouldn't be a customization for this at >> all. I think local definitions always should override the collection >> definition. >> >> >> 5.4 Files that belong to multiple collections >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> What if I'm being a clever user and define multiple collections for >> the same files (I.e. overlap in the Venn-diagram of files grouped by >> collections). Which collection is "active" when I'm visiting the file? >> >> This depends on if Emacs should evaluate the collection-settings for >> each file visit or not. If they are evaluated for each file visit then >> the first matching project in the list of collections should apply for >> that file. If a cache is created that lists file and collection >> relationships then each file should relate to a list of collections >> where the first collection in that list should apply. >> >> If Emacs can rely on `org-collections-active' being set, then the >> collection referenced there should be used. >> >> >> 5.5 Dynamic lists of files and folders for a collection? >> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> Should the list of files allow for folders with recursion and patterns >> should it be required to provide a fixed defined list of files? >> >> Preferably the same way as `org-agenda-files' work today. Maybe some >> kind of caching-mechanism is needed though, for commands that might >> have to look for file, collection relations. A cache adds potential >> pain for the user though. If a file is added to a folder in a >> collection and a "collection-command" is run then the new file might >> not show up in the results anyway... So the user will be affected by >> caching and will have to know about it. Not good... >> >> >> 6 Alternatives >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> Doing research for this feature made me realize that much of what I'm >> proposing already exist! In another form though, as [directory >> variables]. That requires customizations to be defined as safe though. >> And today some of the things I would consider to define a collection >> aren't safe. For example `org-agenda-files', `org-todo-keywords', >> `org-publish-project-alist'. >> >> Some issues with relying on directory variables (Assuming they also >> are made safe): >> - When invoking Org agenda I will have to first visit a file inside a >> specific folder to get the agenda for the correct project >> - .... >> >> >> [directory variables] >> >> >> 7 References >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> >> I've mentioned this idea the Org mode mailing list previously, but >> only as short side notes to other topics: >> - >> - >> >> Note that I've talked about it as "project". I think that name still >> could be considered instead of "collection". Collection is more >> general and less overloaded in terms of productivity software. And it >> shifts the focus away from task management a bit, which I think can be >> a good thing. Because while Org mode may often start to be used as a >> task/project manager software, it's useful in a much wider context >> than that! >> >> >> >> Footnotes >> _________ >> >> [1] I've previously written about this as "Projects". While Project >> was my initial name for this feature I think collection may be a >> better option. For the sake of this text both options work just fine. >> The idea is the same. --=20 Tim Cross