* bug#50514: 28.0.50; org inline call to python src block req C-g to break [not found] ` <87mtbsgxwh.fsf@gnus.org> @ 2022-08-26 13:30 ` Ihor Radchenko 2022-08-27 13:16 ` Lars Ingebrigtsen 0 siblings, 1 reply; 9+ messages in thread From: Ihor Radchenko @ 2022-08-26 13:30 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Barton, Mark, 50514@debbugs.gnu.org, Augusto Stoffel Lars Ingebrigtsen <larsi@gnus.org> writes: > The bug on the Emacs side was fixed by Augusto's patch, I think, but > there's a remaining problem in ob-python, so I'm reassigning this bug > report to org-mode. Could you please elaborate what the problem is? There have been multiple issues in this thread and the one previously reported to Org ML is solved on Emacs side. -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#50514: 28.0.50; org inline call to python src block req C-g to break 2022-08-26 13:30 ` bug#50514: 28.0.50; org inline call to python src block req C-g to break Ihor Radchenko @ 2022-08-27 13:16 ` Lars Ingebrigtsen 2022-08-27 14:11 ` Ihor Radchenko 0 siblings, 1 reply; 9+ messages in thread From: Lars Ingebrigtsen @ 2022-08-27 13:16 UTC (permalink / raw) To: Ihor Radchenko; +Cc: Barton, Mark, 50514@debbugs.gnu.org, Augusto Stoffel Ihor Radchenko <yantar92@gmail.com> writes: > Lars Ingebrigtsen <larsi@gnus.org> writes: > >> The bug on the Emacs side was fixed by Augusto's patch, I think, but >> there's a remaining problem in ob-python, so I'm reassigning this bug >> report to org-mode. > > Could you please elaborate what the problem is? > There have been multiple issues in this thread and the one previously > reported to Org ML is solved on Emacs side. It's this bit: > So I found if I extend the sleep-for in ob-python from 10 ms to 90 ms, > my initialization problem is “fixed”. The sleep is still 10ms in the in-tree ob-python, at least. ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#50514: 28.0.50; org inline call to python src block req C-g to break 2022-08-27 13:16 ` Lars Ingebrigtsen @ 2022-08-27 14:11 ` Ihor Radchenko 2022-08-27 18:56 ` Barton, Mark 0 siblings, 1 reply; 9+ messages in thread From: Ihor Radchenko @ 2022-08-27 14:11 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Barton, Mark, 50514@debbugs.gnu.org, Augusto Stoffel [-- Attachment #1: Type: text/plain, Size: 859 bytes --] Lars Ingebrigtsen <larsi@gnus.org> writes: >> Could you please elaborate what the problem is? >> There have been multiple issues in this thread and the one previously >> reported to Org ML is solved on Emacs side. > > It's this bit: > >> So I found if I extend the sleep-for in ob-python from 10 ms to 90 ms, >> my initialization problem is “fixed”. > > The sleep is still 10ms in the in-tree ob-python, at least. 10ms magic constant is indeed not great. However, I am unable to reproduce the original bug on my system. I tried to draft an alternative patch. See the attached. Mark, would you mind testing the patch? -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-ob-python-Fix-Emacs-bug-50514-when-Python-loads-slow.patch --] [-- Type: text/x-patch, Size: 1820 bytes --] From d5347c688d2fb8037116d82fa7987e9e6690ae66 Mon Sep 17 00:00:00 2001 Message-Id: <d5347c688d2fb8037116d82fa7987e9e6690ae66.1661609451.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Sat, 27 Aug 2022 22:06:11 +0800 Subject: [PATCH] ob-python: Fix Emacs bug#50514 when Python loads slowly * lisp/ob-python.el (org-babel-python-initiate-session): Wait until python process is initialized. (org-babel-python-initiate-session-by-key): Do not rely on 10ms delay to initialize Python. --- lisp/ob-python.el | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lisp/ob-python.el b/lisp/ob-python.el index 932aca08e..2ad907bd8 100644 --- a/lisp/ob-python.el +++ b/lisp/ob-python.el @@ -197,8 +197,7 @@ (defun org-babel-python-initiate-session-by-key (&optional session) (setq py-buffer (org-babel-python-with-earmuffs session))) (let ((python-shell-buffer-name (org-babel-python-without-earmuffs py-buffer))) - (run-python cmd) - (sleep-for 0 10))) + (run-python cmd))) ((and (eq 'python-mode org-babel-python-mode) (fboundp 'py-shell)) ; python-mode.el (require 'python-mode) @@ -225,8 +224,9 @@ (defun org-babel-python-initiate-session-by-key (&optional session) (defun org-babel-python-initiate-session (&optional session _params) "Create a session named SESSION according to PARAMS." (unless (string= session "none") - (org-babel-python-session-buffer - (org-babel-python-initiate-session-by-key session)))) + (let ((session (org-babel-python-session-buffer + (org-babel-python-initiate-session-by-key session))))) + (org-babel-comint-wait-for-output session))) (defvar org-babel-python-eoe-indicator "org_babel_python_eoe" "A string to indicate that evaluation has completed.") -- 2.35.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* bug#50514: 28.0.50; org inline call to python src block req C-g to break 2022-08-27 14:11 ` Ihor Radchenko @ 2022-08-27 18:56 ` Barton, Mark 2022-08-27 21:31 ` Mark Barton 0 siblings, 1 reply; 9+ messages in thread From: Barton, Mark @ 2022-08-27 18:56 UTC (permalink / raw) To: Ihor Radchenko; +Cc: 50514@debbugs.gnu.org, Lars Ingebrigtsen, Augusto Stoffel [-- Attachment #1: Type: text/plain, Size: 1819 bytes --] On Aug 27, 2022, at 7:11 AM, Ihor Radchenko <yantar92@gmail.com<mailto:yantar92@gmail.com>> wrote: --- lisp/ob-python.el | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lisp/ob-python.el b/lisp/ob-python.el index 932aca08e..2ad907bd8 100644 --- a/lisp/ob-python.el +++ b/lisp/ob-python.el @@ -197,8 +197,7 @@ (defun org-babel-python-initiate-session-by-key (&optional session) (setq py-buffer (org-babel-python-with-earmuffs session))) (let ((python-shell-buffer-name (org-babel-python-without-earmuffs py-buffer))) - (run-python cmd) - (sleep-for 0 10))) + (run-python cmd))) ((and (eq 'python-mode org-babel-python-mode) (fboundp 'py-shell)) ; python-mode.el (require 'python-mode) @@ -225,8 +224,9 @@ (defun org-babel-python-initiate-session-by-key (&optional session) (defun org-babel-python-initiate-session (&optional session _params) "Create a session named SESSION according to PARAMS." (unless (string= session "none") - (org-babel-python-session-buffer - (org-babel-python-initiate-session-by-key session)))) + (let ((session (org-babel-python-session-buffer + (org-babel-python-initiate-session-by-key session))))) + (org-babel-comint-wait-for-output session))) (defvar org-babel-python-eoe-indicator "org_babel_python_eoe" "A string to indicate that evaluation has completed.") -- 2.35.1 I get the following in the Messages: org-babel-comint-wait-for-output: Buffer python-chain does not exist or has no process Note that python-chain is the name of my session passed to the python blocks as shown in the following line within my org file. #+PROPERTY: header-args:python :session python-chain :exports results :results raw file :file-ext table Thanks, Mark [-- Attachment #2: Type: text/html, Size: 29021 bytes --] ^ permalink raw reply related [flat|nested] 9+ messages in thread
* bug#50514: 28.0.50; org inline call to python src block req C-g to break 2022-08-27 18:56 ` Barton, Mark @ 2022-08-27 21:31 ` Mark Barton 2022-08-28 0:38 ` Barton, Mark 0 siblings, 1 reply; 9+ messages in thread From: Mark Barton @ 2022-08-27 21:31 UTC (permalink / raw) To: Barton, Mark; +Cc: 50514, Lars Ingebrigtsen, Augusto Stoffel, Ihor Radchenko [-- Attachment #1: Type: text/plain, Size: 2386 bytes --] > On Aug 27, 2022, at 11:59 AM, Barton, Mark <Mark.Barton@disney.com> wrote: > > > >> On Aug 27, 2022, at 7:11 AM, Ihor Radchenko <yantar92@gmail.com> wrote: >> >> --- >> lisp/ob-python.el | 8 ++++---- >> 1 file changed, 4 insertions(+), 4 deletions(-) >> >> diff --git a/lisp/ob-python.el b/lisp/ob-python.el >> index 932aca08e..2ad907bd8 100644 >> --- a/lisp/ob-python.el >> +++ b/lisp/ob-python.el >> @@ -197,8 +197,7 @@ (defun org-babel-python-initiate-session-by-key (&optional session) >> (setq py-buffer (org-babel-python-with-earmuffs session))) >> (let ((python-shell-buffer-name >> (org-babel-python-without-earmuffs py-buffer))) >> - (run-python cmd) >> - (sleep-for 0 10))) >> + (run-python cmd))) >> ((and (eq 'python-mode org-babel-python-mode) >> (fboundp 'py-shell)) ; python-mode.el >> (require 'python-mode) >> @@ -225,8 +224,9 @@ (defun org-babel-python-initiate-session-by-key (&optional session) >> (defun org-babel-python-initiate-session (&optional session _params) >> "Create a session named SESSION according to PARAMS." >> (unless (string= session "none") >> - (org-babel-python-session-buffer >> - (org-babel-python-initiate-session-by-key session)))) >> + (let ((session (org-babel-python-session-buffer >> + (org-babel-python-initiate-session-by-key session))))) >> + (org-babel-comint-wait-for-output session))) >> >> (defvar org-babel-python-eoe-indicator "org_babel_python_eoe" >> "A string to indicate that evaluation has completed.") >> -- >> 2.35.1 > > I get the following in the Messages: > org-babel-comint-wait-for-output: Buffer python-chain does not exist or has no process > > Note that python-chain is the name of my session passed to the python blocks as shown in the following line within my org file. > > #+PROPERTY: header-args:python :session python-chain :exports results :results raw file :file-ext table > > Thanks, > > Mark > I will work on getting more details this weekend. It looked like the buffer passed to org-babel-comint-wait-for-output was missing the "earmuffs". I tried adding the earmuffs with the org function for that but although I got passed that error, I messed up my python session and the following python blocks were not aware of the imports in the first block. Mark [-- Attachment #2: Type: text/html, Size: 29460 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#50514: 28.0.50; org inline call to python src block req C-g to break 2022-08-27 21:31 ` Mark Barton @ 2022-08-28 0:38 ` Barton, Mark 2022-08-28 1:30 ` Ihor Radchenko 0 siblings, 1 reply; 9+ messages in thread From: Barton, Mark @ 2022-08-28 0:38 UTC (permalink / raw) To: Ihor Radchenko Cc: 50514@debbugs.gnu.org, Mark Barton, Lars Ingebrigtsen, Augusto Stoffel [-- Attachment #1: Type: text/plain, Size: 1416 bytes --] I will work on getting more details this weekend. It looked like the buffer passed to org-babel-comint-wait-for-output was missing the "earmuffs". I tried adding the earmuffs with the org function for that but although I got passed that error, I messed up my python session and the following python blocks were not aware of the imports in the first block. Mark I tried something that was suggested last September 14th by Augusto Stoffel. I had not tried it because changing the sleep from 10 to 90 was my workaround. One thing that might work is to block until the shell is ready with something like (run-python) (with-current-buffer *the-shell-buffer* (while (not python-shell--first-prompt-received) (accept-process-output (get-buffer-process (current-buffer))))) The patch that is working for me now is: modified lisp/org/ob-python.el @@ -198,7 +198,10 @@ org-babel-python-initiate-session-by-key (let ((python-shell-buffer-name (org-babel-python-without-earmuffs py-buffer))) (run-python cmd) - (sleep-for 0 10))) + (with-current-buffer py-buffer + (while (not python-shell--first-prompt-received) + (accept-process-output (get-buffer-process (current-buffer))))) + )) ((and (eq 'python-mode org-babel-python-mode) (fboundp 'py-shell)) ; python-mode.el (require 'python-mode) [-- Attachment #2: Type: text/html, Size: 5009 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#50514: 28.0.50; org inline call to python src block req C-g to break 2022-08-28 0:38 ` Barton, Mark @ 2022-08-28 1:30 ` Ihor Radchenko 2022-08-28 1:56 ` Barton, Mark 0 siblings, 1 reply; 9+ messages in thread From: Ihor Radchenko @ 2022-08-28 1:30 UTC (permalink / raw) To: Barton, Mark Cc: 50514@debbugs.gnu.org, Mark Barton, Lars Ingebrigtsen, Augusto Stoffel [-- Attachment #1: Type: text/plain, Size: 967 bytes --] "Barton, Mark" <Mark.Barton@disney.com> writes: > I tried something that was suggested last September 14th by Augusto Stoffel. I had not tried it because changing the sleep from 10 to 90 was my workaround. > One thing that might work is to block until the shell is ready with something like > > (run-python) > (with-current-buffer *the-shell-buffer* > (while (not python-shell--first-prompt-received) > (accept-process-output (get-buffer-process (current-buffer))))) Thanks, but it is not a good solution. Org cannot rely on internal variables inside python.el (python-shell--first-prompt-received is an internal variable). Internal variables are a subject of change without notice. May you instead try the attached alternative patch? -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: v2-0001-ob-python-Fix-Emacs-bug-50514-when-Python-loads-s.patch --] [-- Type: text/x-patch, Size: 1642 bytes --] From 06373a64387330d5bbdcbb3417cea7c2638713df Mon Sep 17 00:00:00 2001 Message-Id: <06373a64387330d5bbdcbb3417cea7c2638713df.1661650102.git.yantar92@gmail.com> From: Ihor Radchenko <yantar92@gmail.com> Date: Sat, 27 Aug 2022 22:06:11 +0800 Subject: [PATCH v2] ob-python: Fix Emacs bug#50514 when Python loads slowly * lisp/ob-python.el (org-babel-python-initiate-session-by-key): Do not rely on 10ms delay to initialize Python. Wait until python process is initialized using `org-babel-comint-wait-for-output'. --- lisp/ob-python.el | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lisp/ob-python.el b/lisp/ob-python.el index 932aca08e..e50a7c2e2 100644 --- a/lisp/ob-python.el +++ b/lisp/ob-python.el @@ -197,8 +197,7 @@ (defun org-babel-python-initiate-session-by-key (&optional session) (setq py-buffer (org-babel-python-with-earmuffs session))) (let ((python-shell-buffer-name (org-babel-python-without-earmuffs py-buffer))) - (run-python cmd) - (sleep-for 0 10))) + (run-python cmd))) ((and (eq 'python-mode org-babel-python-mode) (fboundp 'py-shell)) ; python-mode.el (require 'python-mode) @@ -217,6 +216,8 @@ (defun org-babel-python-initiate-session-by-key (&optional session) (py-shell nil nil t org-babel-python-command py-buffer nil nil t nil))) (t (error "No function available for running an inferior Python"))) + ;; Wait until Python initializes. + (org-babel-comint-wait-for-output py-buffer) (setq org-babel-python-buffers (cons (cons session py-buffer) (assq-delete-all session org-babel-python-buffers))) -- 2.35.1 ^ permalink raw reply related [flat|nested] 9+ messages in thread
* bug#50514: 28.0.50; org inline call to python src block req C-g to break 2022-08-28 1:30 ` Ihor Radchenko @ 2022-08-28 1:56 ` Barton, Mark 2022-08-28 2:55 ` Ihor Radchenko 0 siblings, 1 reply; 9+ messages in thread From: Barton, Mark @ 2022-08-28 1:56 UTC (permalink / raw) To: Ihor Radchenko; +Cc: 50514@debbugs.gnu.org, Lars Ingebrigtsen, Augusto Stoffel [-- Attachment #1: Type: text/plain, Size: 1958 bytes --] On Aug 27, 2022, at 6:30 PM, Ihor Radchenko <yantar92@gmail.com<mailto:yantar92@gmail.com>> wrote: * lisp/ob-python.el (org-babel-python-initiate-session-by-key): Do not rely on 10ms delay to initialize Python. Wait until python process is initialized using `org-babel-comint-wait-for-output'. --- lisp/ob-python.el | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lisp/ob-python.el b/lisp/ob-python.el index 932aca08e..e50a7c2e2 100644 --- a/lisp/ob-python.el +++ b/lisp/ob-python.el @@ -197,8 +197,7 @@ (defun org-babel-python-initiate-session-by-key (&optional session) (setq py-buffer (org-babel-python-with-earmuffs session))) (let ((python-shell-buffer-name (org-babel-python-without-earmuffs py-buffer))) - (run-python cmd) - (sleep-for 0 10))) + (run-python cmd))) ((and (eq 'python-mode org-babel-python-mode) (fboundp 'py-shell)) ; python-mode.el (require 'python-mode) @@ -217,6 +216,8 @@ (defun org-babel-python-initiate-session-by-key (&optional session) (py-shell nil nil t org-babel-python-command py-buffer nil nil t nil))) (t (error "No function available for running an inferior Python"))) + ;; Wait until Python initializes. + (org-babel-comint-wait-for-output py-buffer) (setq org-babel-python-buffers (cons (cons session py-buffer) (assq-delete-all session org-babel-python-buffers))) -- 2.35.1 That works for me. I'm using a series of python blocks to create org tables to be inserted in specific places in the document. So each block is named for that reason. Do we need to worry about the note about not appropriate for large blocks of code or does that not apply in this case with the way you implemented it. (org-babel-comint-wait-for-output BUFFER) Wait until output arrives from BUFFER. Note: this is only safe when waiting for the result of a single statement (not large blocks of code). Mark [-- Attachment #2: Type: text/html, Size: 31975 bytes --] ^ permalink raw reply related [flat|nested] 9+ messages in thread
* bug#50514: 28.0.50; org inline call to python src block req C-g to break 2022-08-28 1:56 ` Barton, Mark @ 2022-08-28 2:55 ` Ihor Radchenko 0 siblings, 0 replies; 9+ messages in thread From: Ihor Radchenko @ 2022-08-28 2:55 UTC (permalink / raw) To: Barton, Mark; +Cc: 50514@debbugs.gnu.org, Lars Ingebrigtsen, Augusto Stoffel "Barton, Mark" <Mark.Barton@disney.com> writes: > Do we need to worry about the note about not appropriate for large blocks of code or does that not apply in this case with the way you implemented it. > (org-babel-comint-wait-for-output BUFFER) > > Wait until output arrives from BUFFER. > Note: this is only safe when waiting for the result of a single > statement (not large blocks of code). It should not be a problem. org-babel-python-initiate-session-by-key is creating a new python session---no actual code blocks are passed to python; just the python command is executed. Fixed on main via 06373a643. https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=06373a64387330d5bbdcbb3417cea7c2638713df -- Ihor Radchenko, Org mode contributor, Learn more about Org mode at https://orgmode.org/. Support Org development at https://liberapay.com/org-mode, or support my work at https://liberapay.com/yantar92 ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2022-08-28 2:55 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <24B9F777-2B69-45FD-ADBB-11E7A18F7A9A@disney.com>
[not found] ` <87wnnnjlav.fsf@gmail.com>
[not found] ` <AEC70ED1-96FB-4FC9-9971-EA0C5421BC8A@disney.com>
[not found] ` <871r5u1e3m.fsf@gmail.com>
[not found] ` <61CB9513-0EEC-4F46-90D6-1D208BBADFA1@disney.com>
[not found] ` <87pmteb1ef.fsf@gmail.com>
[not found] ` <21C76818-63E5-4DC9-A60D-2BEF33813A18@disney.com>
[not found] ` <73383502-159B-4BD9-8836-6C8A534A7063@disney.com>
[not found] ` <87r1dry7x0.fsf@gmail.com>
[not found] ` <32AF1DCB-F81A-4FB7-8EB2-31B483E5E468@disney.com>
[not found] ` <87sfy7qadr.fsf@gmail.com>
[not found] ` <073CC0D1-9900-4613-87E8-D69EA1C11623@disney.com>
[not found] ` <87k0jjq79k.fsf@gmail.com>
[not found] ` <87k0jiuuzc.fsf@gnus.org>
[not found] ` <87mtbsgxwh.fsf@gnus.org>
2022-08-26 13:30 ` bug#50514: 28.0.50; org inline call to python src block req C-g to break Ihor Radchenko
2022-08-27 13:16 ` Lars Ingebrigtsen
2022-08-27 14:11 ` Ihor Radchenko
2022-08-27 18:56 ` Barton, Mark
2022-08-27 21:31 ` Mark Barton
2022-08-28 0:38 ` Barton, Mark
2022-08-28 1:30 ` Ihor Radchenko
2022-08-28 1:56 ` Barton, Mark
2022-08-28 2:55 ` Ihor Radchenko
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).