* source blocks DAG evaluation @ 2021-03-22 2:00 c4t0 2021-03-22 2:44 ` Thomas S. Dye 0 siblings, 1 reply; 3+ messages in thread From: c4t0 @ 2021-03-22 2:00 UTC (permalink / raw) To: emacs-orgmode Hi, Is it possible to have a dependency hierarchy of source blocks? i.e.: in C, if you use libA from a compilation unit and that library needs libB, you don't need to include it in main.c -> main.c #include "libB.h" -> libB.c #include "libA.h" you don't need to: -> main.c #include "libB.h" #include "libA.h" because library headers are closed under inclusion. I haven't succeeded in doing the same in org-mode. Even after populating org-babel-library-of-babel. Using #+call just doesn't work. Using :var is better, evaluates all, but there appears to lack session support, it doesn't check for cycles and it feels a little hacky With #+call I need to do it like this: #+name: libA #+begin_src scheme :results none (define hi "hello") #+end_src #+name: libB #+begin_src scheme :results none (define greetings (string-append hi ", " "to all the people!")) #+end_src here is my "main" I need to C-c C-c in each #+call line and write the :session that the code block uses in each one, and do it in the correct order. If I C-c C-c in libB first it won't eval because 'hi' is not defined. #+call: libB[:session example] #+call: libA[:session example] #+begin_src scheme :session example :results output (display greetings) #+end_src source blocks can be #+call(ed) but aren't closed under #+call (a source block can be called but then the callee won't) instead I would like to : #+name: libA #+begin_src scheme :results none (define hi "hello") #+end_src #+call: libA #+name: libB #+begin_src scheme :results none (define greetings (string-append hi ", " "to all the people!")) #+end_src #+call: libB #+begin_src scheme :session example :results output (display greetings) #+end_src - there shouldn't be needed to C-c C-c in the #+call line, evaluating the source block alone should suffice. - there shouldn't be a need to write the :session - it should use the session of the user evaled block, unless specified otherwise In the other hand, using :var with a dummy variable: #+name: libA #+begin_src scheme :results none (define hi "hello") #+end_src #+name: libB #+begin_src scheme :results none :var _=libA (define greetings (string-append hi ", " "to all the people!")) #+end_src #+HEADER: :var _=libB #+begin_src scheme :session example :results output (display greetings) #+end_src It evals libA then libB and finally the (display greetings) code. But it fails, because the :session example is ignored. Even if I add a :session example to every source block (which would be really bad, sessión must be decided by the consumer) it doesn't work. I think that is because :var expects a value, so it just opens a new session to evaluate code every time. Besides if there are any dependency cycles, it just fails with: Lisp nesting exceeds ‘max-lisp-eval-depth’ So if I'm right and there is not an implemented way to do this, how can we do it? Adding session support for :var? constructing a DAG of #+calls and then evaluating in order? maybe using a new header? COD. ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: source blocks DAG evaluation 2021-03-22 2:00 source blocks DAG evaluation c4t0 @ 2021-03-22 2:44 ` Thomas S. Dye 2021-03-22 13:41 ` c4t0 0 siblings, 1 reply; 3+ messages in thread From: Thomas S. Dye @ 2021-03-22 2:44 UTC (permalink / raw) To: emacs-orgmode Aloha c4to, I would be tempted to use noweb expansion here. #+name: libB #+begin_src scheme :results none :noweb yes <<libA>> (define greetings (string-append hi ", " "to all the people!")) #+end_src #+begin_src scheme :session example :results output :noweb yes <<libB>> (display greetings) #+end_src Does this do what you want? All the best, Tom c4t0 writes: > Hi, > > Is it possible to have a dependency hierarchy of source blocks? > > i.e.: in C, if you use libA from a compilation unit and that > library needs libB, you don't need to include it in main.c > > -> main.c > #include "libB.h" > -> libB.c > #include "libA.h" > > you don't need to: > -> main.c > #include "libB.h" > #include "libA.h" > > because library headers are closed under inclusion. > > I haven't succeeded in doing the same in org-mode. Even after > populating org-babel-library-of-babel. > > Using #+call just doesn't work. Using :var is better, evaluates > all, but there appears to lack session support, it doesn't check > for cycles and it feels a little hacky > > With #+call I need to do it like this: > > #+name: libA > #+begin_src scheme :results none > (define hi "hello") > #+end_src > > #+name: libB > #+begin_src scheme :results none > (define greetings (string-append hi ", " "to all the people!")) > #+end_src > > here is my "main" I need to C-c C-c in each #+call line and > write the :session that the code block uses in each one, and do > it in the correct order. If I C-c C-c in libB first it won't > eval because 'hi' is not defined. > > #+call: libB[:session example] > #+call: libA[:session example] > #+begin_src scheme :session example :results output > (display greetings) > #+end_src > > source blocks can be #+call(ed) but aren't closed under #+call > (a source block can be called but then the callee won't) > > instead I would like to : > > #+name: libA > #+begin_src scheme :results none > (define hi "hello") > #+end_src > > #+call: libA > #+name: libB > #+begin_src scheme :results none > (define greetings (string-append hi ", " "to all the people!")) > #+end_src > > #+call: libB > #+begin_src scheme :session example :results output > (display greetings) > #+end_src > > - there shouldn't be needed to C-c C-c in the #+call line, > evaluating the source block alone should suffice. > - there shouldn't be a need to write the :session > - it should use the session of the user evaled block, unless > specified otherwise > > In the other hand, using :var with a dummy variable: > > #+name: libA > #+begin_src scheme :results none > (define hi "hello") > #+end_src > > #+name: libB > #+begin_src scheme :results none :var _=libA > (define greetings (string-append hi ", " "to all the people!")) > #+end_src > > #+HEADER: :var _=libB > #+begin_src scheme :session example :results output > (display greetings) > #+end_src > > It evals libA then libB and finally the (display greetings) > code. > But it fails, because the :session example is ignored. Even if I > add a :session example to every source block (which would be > really bad, sessión must be decided by the consumer) it doesn't > work. I think that is because :var expects a value, so it just > opens a new session to evaluate code every time. > > Besides if there are any dependency cycles, it just fails with: > Lisp nesting exceeds ‘max-lisp-eval-depth’ > > So if I'm right and there is not an implemented way to do this, > how can we do it? Adding session support for :var? constructing > a DAG of #+calls and then evaluating in order? maybe using a new > header? > > COD. -- Thomas S. Dye https://tsdye.online/tsdye ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: source blocks DAG evaluation 2021-03-22 2:44 ` Thomas S. Dye @ 2021-03-22 13:41 ` c4t0 0 siblings, 0 replies; 3+ messages in thread From: c4t0 @ 2021-03-22 13:41 UTC (permalink / raw) To: Thomas S. Dye; +Cc: emacs-orgmode Hello Thomas! That is fairly nice, thanks! I can use it for the moment. I see a minor problem, if you export it, the last block will show all the code and that may be redundant or undesired. But that can be solved fiddling with exporting options or code I guess. btw, using :noweb no-export, just leaves the <<lib*>> sintax, so it's not what I want. In this sense it also doesn't do any cycle resolving, when a there is one it just fails with "org-at-heading-p: Variable binding depth exceeds max-specpdl-size" That may be ok, you shouldn't have cycles. But there is a more realist problem, you *can* have, diamonds dependencies. Say libA -> libB -> libD -> main \> libC / If you <<libD>> from main, you will get two copies of libA source code. That may be a problem in some languages and it's definitely bad at export time. The good about noweb expansion is that you can ignore the problem with the sessions all together and it will work with things that doesn't have sessions at all (now I remember a home work in assembler that I did in org-mode with noweb expansion ...) So, maybe if I try to improve anything, to address this issue, maybe is the code for noweb expansion? Maybe some options to conditional output the template expansion or just remove the <<*>> calls. I don't know if the semantics of what I want go along a template expansion ... I have the feeling that we might need something particular for this, like a (require ...) for org-mode source blocks. thanks! COD. "Thomas S. Dye" <tsd@tsdye.online> writes: > Aloha c4to, > > I would be tempted to use noweb expansion here. > > #+name: libB > #+begin_src scheme :results none :noweb yes > > <<libA>> > (define greetings (string-append hi ", " "to all the people!")) > #+end_src > > #+begin_src scheme :session example :results output :noweb yes > <<libB>> > (display greetings) > #+end_src > > Does this do what you want? > > All the best, > Tom > > c4t0 writes: > >> Hi, >> >> Is it possible to have a dependency hierarchy of source blocks? >> >> i.e.: in C, if you use libA from a compilation unit and that >> library needs libB, you don't need to include it in main.c >> >> -> main.c >> #include "libB.h" >> -> libB.c >> #include "libA.h" >> >> you don't need to: >> -> main.c >> #include "libB.h" >> #include "libA.h" >> >> because library headers are closed under inclusion. >> >> I haven't succeeded in doing the same in org-mode. Even after >> populating org-babel-library-of-babel. >> >> Using #+call just doesn't work. Using :var is better, evaluates >> all, but there appears to lack session support, it doesn't check >> for cycles and it feels a little hacky >> >> With #+call I need to do it like this: >> >> #+name: libA >> #+begin_src scheme :results none >> (define hi "hello") >> #+end_src >> >> #+name: libB >> #+begin_src scheme :results none >> (define greetings (string-append hi ", " "to all the people!")) >> #+end_src >> >> here is my "main" I need to C-c C-c in each #+call line and >> write the :session that the code block uses in each one, and do >> it in the correct order. If I C-c C-c in libB first it won't >> eval because 'hi' is not defined. >> >> #+call: libB[:session example] >> #+call: libA[:session example] >> #+begin_src scheme :session example :results output >> (display greetings) >> #+end_src >> >> source blocks can be #+call(ed) but aren't closed under #+call >> (a source block can be called but then the callee won't) >> >> instead I would like to : >> >> #+name: libA >> #+begin_src scheme :results none >> (define hi "hello") >> #+end_src >> >> #+call: libA >> #+name: libB >> #+begin_src scheme :results none >> (define greetings (string-append hi ", " "to all the people!")) >> #+end_src >> >> #+call: libB >> #+begin_src scheme :session example :results output >> (display greetings) >> #+end_src >> >> - there shouldn't be needed to C-c C-c in the #+call line, >> evaluating the source block alone should suffice. >> - there shouldn't be a need to write the :session >> - it should use the session of the user evaled block, unless >> specified otherwise >> >> In the other hand, using :var with a dummy variable: >> >> #+name: libA >> #+begin_src scheme :results none >> (define hi "hello") >> #+end_src >> >> #+name: libB >> #+begin_src scheme :results none :var _=libA >> (define greetings (string-append hi ", " "to all the people!")) >> #+end_src >> >> #+HEADER: :var _=libB >> #+begin_src scheme :session example :results output >> (display greetings) >> #+end_src >> >> It evals libA then libB and finally the (display greetings) >> code. >> But it fails, because the :session example is ignored. Even if I >> add a :session example to every source block (which would be >> really bad, sessión must be decided by the consumer) it doesn't >> work. I think that is because :var expects a value, so it just >> opens a new session to evaluate code every time. >> >> Besides if there are any dependency cycles, it just fails with: >> Lisp nesting exceeds ‘max-lisp-eval-depth’ >> >> So if I'm right and there is not an implemented way to do this, >> how can we do it? Adding session support for :var? constructing >> a DAG of #+calls and then evaluating in order? maybe using a new >> header? >> >> COD. > > > -- > Thomas S. Dye > https://tsdye.online/tsdye ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-03-22 13:59 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-03-22 2:00 source blocks DAG evaluation c4t0 2021-03-22 2:44 ` Thomas S. Dye 2021-03-22 13:41 ` c4t0
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).