From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id kClQHr37u189PwAA0tVLHw (envelope-from ) for ; Mon, 23 Nov 2020 18:13:17 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id CBQVGr37u19NUwAAbx9fmQ (envelope-from ) for ; Mon, 23 Nov 2020 18:13:17 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A5D5E94051B for ; Mon, 23 Nov 2020 18:13:16 +0000 (UTC) Received: from localhost ([::1]:41390 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1khGKx-0002kA-Ij for larch@yhetil.org; Mon, 23 Nov 2020 13:13:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:54672) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1khGJq-0002hK-It for emacs-orgmode@gnu.org; Mon, 23 Nov 2020 13:12:06 -0500 Received: from static.rcdrun.com ([95.85.24.50]:37425) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1khGJn-0008Hu-Lz for emacs-orgmode@gnu.org; Mon, 23 Nov 2020 13:12:06 -0500 Received: from localhost ([::ffff:41.202.241.56]) (AUTH: PLAIN admin, TLS: TLS1.2,256bits,ECDHE_RSA_AES_256_GCM_SHA384) by static.rcdrun.com with ESMTPSA id 00000000002C1AE7.000000005FBBFB71.00007E3E; Mon, 23 Nov 2020 18:12:01 +0000 Date: Mon, 23 Nov 2020 21:08:55 +0300 From: Jean Louis To: Ihor Radchenko Subject: Is Org really so simple? Message-ID: References: <87y2ive1i4.fsf@localhost> <878sauhhv1.fsf@web.de> <875z5ygwwr.fsf@web.de> <87r1olfvh4.fsf@web.de> <878sascg5j.fsf@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline In-Reply-To: <878sascg5j.fsf@localhost> User-Agent: Mutt/2.0 (3d08634) (2020-11-07) Received-SPF: pass client-ip=95.85.24.50; envelope-from=bugs@gnu.support; helo=static.rcdrun.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Dr. Arne Babenhauserheide" , Texas Cyberthal , "emacs-orgmode@gnu.org" Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: ns3122888.ip-94-23-21.eu Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Spam-Score: 1.99 X-TUID: bg0dN6R5yThu * Ihor Radchenko [2020-11-23 17:18]: :PROPERTIES: :CREATED: [2020-11-23 Mon 18:42] :ID: edebb3e7-e755-4ecc-a5e8-a3353a3f5fd0 :END: > Dear Jean Louis, > > Your description of the database reminds me how org-roam handles the > files - it also uses an external database for linking and allows quick > incremental search that does not really depend on where the > files/headings are stored. Sounds good, I can see there is graph database used. > However, what you are talking about is against org-mode philosophy, > as I know it. Only philosophy I know is that it is plain text. Is there any official philosophy? I have no idea, at least manual does not give me references. I cannot find "philosophy", send me references. It says "to keep simple things simple". But Org is far far from being simple any more. It offers good principles, paradigms and people built many enhancements upon those. Speedily it becomes way much more than simple. Headings do not look any more as headings, it looks like pieces of code to a person that is new. Properties, tags, clocks, schedule, deadline, all that tries to store itself under specific heading. There is easily too much of it and things are not simple any more. > Currently, the dev's stance is that org files must be > self-sufficient. There is no compact principle there practically. Anything is possible. That Org files are not practically self-sufficient shows the fact that there are 129 Emacs packages in one Org distribution. > Org-mode should not depend on external database to manage the org > files and operations with them. Everything must be plain text! Question is what is meant by database, it can be anything. One can save LISP data. Recent files, desktop, eww bookmarks, init.el or .emacs file are also all similar databases, there is the underused EIEIO with persistent stuff that represent built-in database functionality. That Org files are not self-sufficient shows the demand that there is almost no Org user who does not have add-ons, packages, modifications, configurations. Would it be really self-sufficient there would be no development going on, right? Babel executions clearly show that Org is not self sufficient and depends on number of external software. I don't mind of philosophy, in fact I would like that philosophy is really that what it wanted to be, but that time is over. I am just pointing out that it is by many means not self sufficient. Is by default LaTeX export enabled? I think it is. How big is the LaTeX package? It is huge, and Org depends on it for export. Thus Org is far far from being self-sufficient. Almost every system has GDBM database, if Emacs would have bindings to GDBM, there would be so much less of development in general for many various packages and Emacs would be expanding faster and easier. In fact I think that author and initial developers could not predict at the time where the Org goes and that speaking of self-sufficient and "plain text" only is history. Before I found out about Org I was using back in time `hnb' in console to track various planning tasks. It works nice and simple. That is really self sufficient. Org definitely not. HNB - Hierarchical Notebook http://hnb.sourceforge.net/ In the mean time I have created database where I can store TODO, Notes, Calls, SMS, People, Invoices, Groups, Mailing Lists and so on, and made my own shell and Perl interfaces to it. And I used to manage it through: GeDaFe - PostgreSQL Generic Database Interface http://gedafe.github.io/doc/gedafe-sql.en.html and this was and is hierarchical or better graph knowledge management and relational database. Creating simple table in the database automatically helped me to manage that table. It is trivial to create NOTES table with schedule date, clock in, TODO or other conditions and tags. The interface is just there and automatic to whatever table I create the interface is there to add/modify/delete/search/refine records. That is what I would say "simple" and keeping things simple and indefinitely extensible without modification of software. The fundamental GeDaFe principle I would like to try to implement in Emacs. And same database I use for web interface I am using also within Emacs. GeDaFe principle is following: - define your data (like handling notes, TODO, or executing scripts within notes) - work and forget about any underlying functions, add/create/delete/modify/index or search, make reports with simple exports - expand with more definitions of data when necessary (like add various properties, or other data tables, like contacts, invoices, etc.) and repeat the process. Org also shows that it does not hold "Notes" only, it holds more than that, I have written average book size technical documents with it. Only just one part of the document is printed from one single node that belongs to single project among many. People use such documents on the ground. My use case is not for simple notes. A node in a tree can be anything, and Org enhancements develop by that principle. For example there is org-contacts where nodes are meant to be "People". Such development is so much hard coded. Simpler would be to define the type of nodes and work by their types. One could need just one type table and one node table and that is about all. The type table could say: - this node is heading - or, this node is text under heading - or, this node is paragraph among many others - or, this node is hyperlink to WWW URI - or, this node is hyperlink to file - this node is movie to play - this type is PDF to be opened on page 3 - this one is really note - this one is note but with action assigned like TODO, etc. Nodes could have its properties defined like for anything. Nodes can reference its parent nodes. Node can be parent to any heading. Once such 2 tables are defined magic starts to take place, it becomes its subtree with all kinds of node types including Babel execution and similar. Meta data is meta, it can be updated but need not be visible. Computer is handling meta data. Node can be a page in a subtree that becomes a website. Node can be object for person or company, or just anything. I am currently using my nodes to quickly research various subjects by using that type of dynamic knowledge repository. Org file is dynamic knowledge repository. About Dynamic Knowledge Repositories (DKR) https://www.dougengelbart.org/content/view/190/163/ Then around 2015 I have discovered Cherrytree and have made bunch of notes with it, and that is self-sufficient one program that keeps simple things simple as it is much more simpler than Org. It offers a visual interface to all features related to the knowledge management Cherrytree - hierarchical note taking application with rich text and syntax highlighting https://www.giuspen.com/cherrytree/ TAGS in Cherrytree are automatic if I remember well, TAG is simply name of the node. If I call node TODO, then nodes under are simply TODO items. Later I discovered there is something similar in Emacs so I started with Org and use it mostly for instruction writing and project management. And I find many options over kill for me. On the other hand my usage of Org would be overkill for somebody, so it depends of viewpoints. In general I was always using headings and subheadings, trees, lists, notes by using text. Somewhere 2004 I started using Markdown one among first as I found it simpler than ASCIIDOC and M4, but not as perfect. 2020 and 2021 I like to raise the level of dynamic knowledge journaling to the meta level where I think less about underlying functions in software. That experience also tells that Org is definitely not simple. Among hnb, GeDaFe database approach, Cherrytree and Org mode, for "keeping things simple" like note taking and TODO items, project management, Cherrytree was the best for me. Among all those for keeping complex things simple the database approach is the best. Of course that I use database within Emacs and it is not visible to user what it is. Database allows simultaneous operation by several people on distance and that is the groupware feature as envisioned by Doug Engelbart. For document writing and nice formatting with LaTeX export, Org mode is the best personally. For notes, database based notes are best as only so I have relations between the note and other objects. With 200,000 contacts in the database which I can quickly access and assign notes to them, how would that work with Org? Hardly. Notes are related to people, to projects, plans, opportunities, research subjects, companies and so on. There is no simple way in Org mode to relate one note to multiple other related subjects. And people try to do exactly that, people are developing Org in the manner to relate objects in Org file to anything else. And they do hard work as they do it manually. Relational database speeds up and does not tell user to manually hyperlink various relations. I have sent thousands of tasks to people by using this function below. And I had to define for each Org file who are "assignees" for that Org file. And I have like 50 assignees, so I need to copy and paste their nick names or identifiers into the Org file. There it comes the attribute of being "self-sufficient", it becomes self-sufficient because I had to define all assignees into that specific file, but I find it tedious and not useful. (defun rcd/org-send-assigned-task () "Sends assigned task to designated individual as Org" (interactive) (let* ((member-data (rcd-org-extract-assigned-member-email-data)) (id (if member-data (first member-data) nil)) (signature (if (equal (type-of (symbol-value (fifth member-data))) 'cons) (third (symbol-value (fifth member-data))) "")) (file (rcd-org-subtree-to-file signature)) (subject (rcd/org-find-headline)) (esubject (escape-% subject)) (ask (unless id (y-or-n-p "No assignment found. Do you want to send it by email?"))) (name (if id (third member-data))) ;; (name (if ask (read-from-minibuffer "Name:") name)) (voice (format "The task '%s' is being sent to '%s'" subject name)) (email (if id (if (equal (type-of (fourth member-data)) 'cons) (car (fourth member-data)) (fourth member-data)))) (email (if ask (cf-search-email (read-from-minibuffer "Search for email: ")) email)) (really (y-or-n-p (format "Do you really want to send it to: %s?" (if ask email name))))) (if (and really (or id ask)) (if (string-match "@" email) (progn ;; (message (escape-% subject)) (speak voice) (mutt-send-file name email esubject file)) (message "No email specified")) (message "Aborted sending.")))) > Moreover, some devs are even reluctant to hide metadata (like unique > ID implemented in org-id module) from users (which is possible and > also partially implemented). I hope that I have demonstrated that one who develops could review several various paradigms and learn more. I am fine with any decision of design for Org mode as I use it as it is and I have for me other ways of managing data. I am not sure how much those features have been discussed to say that hiding meta data is good or not good. It is better to discuss and find what is more useful. What I can see is that people complain for speed and they build extensions that help with it. Extensions are external while built-in Org does not keep up with the dynamic how people expect it to be. For example I would expect Org to be very simple, very very simple, I would rewind it back to its roots. Somebody else would like complexities. Currently I can see that Org is not made for complexities. It is better to unwind the development and make Org in core very basic and speedy and let people enhance it with external packages. In general it is better to remain simple. Even that may not be possible any more. I see hard work by many people who try to enhance Org as hierarchical knowledge of data because people want to have feature X, but group of those enhancements in reality belong to relational databases and not to text files. Developers wish to accommodate various people and yet by doing so they develop it into complex software. (129 .el packages for one Emacs mode!?) Among those one shall read the Doug Engelbart's paradigms, then especially if one is developer of system of data management that many people use, one shall explore other paradigms, including various approaches to data handling. And overall one shall keep it simple as it is main fundament of Org to be simple, while practical fact remains that it is not anymore simple, not at all. I remember back in time with BASIC programming language that it had also something like a built-in database that one could put on the bottom of the file. Then there is also Perl's __DATA__ that is placed straight into the file and one could also place images and other stuff. Maybe the meta data could be kept this way in just one heading named Meta data, and this heading would not be exported, it could be hidden from the user and it could contain the database necessary for single Org file. By pressing key to show properties one could see properties or simply make them hidden. By making a copy of subtree the metadata would also copy as usual. By exporting, one could make Org without meta data, and I like using this information as I like sending Org headings without personal meta data to third parties.