Change behavior of "org-babel-tangle-publish" from org 9.6 to 9.7 -- deleting source files and not tangling properly

Change behavior of "org-babel-tangle-publish" from org 9.6 to 9.7 -- deleting source files and not tangling properly

[Gilles Marait]

[-- Attachment #1: Type: text/plain, Size: 2942 bytes --]

Hello everyone,

I encountered a change of behavior since org version 9.7 than removes
my source files instead of tangling them at the correct location, using
"org-babel-tangle-publish".

I have not found in the changelog any description of something that
could explain this change, so I report it here.

I'll try to show a minimal example of this issue.

* Description

I have the following file structure.

.
├── org
│   └── Code.org
├── publish.el
└── src
    └── code.cpp

In the "org" directory, I have my org file that describes my code in
literate programming. It's content is:

#+begin_src c++ :tangle ../src/code.cpp 
int a = 3;
#+end_src

The expected behavior is to generate the file "code.cpp" in the src
directory with the tangle mechanism. In this example code.cpp only
contains the line:

int a = 3;

To generate the sources from the org files, I use org-publish. The
code of publish.el is:

  (require 'package)
  (package-initialize)
  (require 'org)
  (require 'ox-publish)
  (setq org-publish-project-alist
	(list
	 (list "generate-source-code"
	       :base-directory "./org"
	       :base-extension "org"
	       :recursive t
	       :publishing-function '(org-babel-tangle-publish)
	       :publishing-directory "."
	       :auto-sitemap nil)))
  (provide 'publish)
  ;;; publish.el ends here

To use the publish from a script I use the following command:

  emacs --batch --load publish.el --eval '(org-publish "generate-source-code")'

* With org-mode 9.6

It works as expected, the code in "Code.cpp" is tangled into "src/code.cpp".

* With org-mode 9.7

I can see "code.cpp" is tangled at the root directory "./code.cpp",
but the file "src/code.cpp" that existed previously is deleted.

* Additional information

emacs version: "29.4"

org-version:
- Test with version 9.7.10 (commit 5ebd973e09d7a46d056f1cd9b5efc44e878cce9c)
- Test with version 9.6.30 (commit 071c6e986c424d2e496be7d0815d6e9cd83ae4e6)

If I use directly "org-babel-tangle-file" I don't have the problem,
the file in tangled to the right location:

emacs --batch --eval "(require 'org)" --eval '(org-babel-tangle-file "./org/Code.org")'

* Bonus question

As a guix user, I use contained environments as much as possible. In
the case described above, I generate an environment without git. I get
the following warning:

Error: (file-missing "Searching for program" "No such file or directory" "git")

It does not have any consequence on the tangling process, but I was
wondering if it was expected that my org-publish process depends on
git ? Would there be any benefit to add it to my environment ?

* Repository

I created a repository with some scripts to reproduce the error, using
guix to control the environment.

https://gitlab.inria.fr/gmarait/debug-org-9.7 


Thank you and best regards, 
Gilles Marait 

[-- Attachment #2: Type: text/html, Size: 3104 bytes --]

Re: Change behavior of "org-babel-tangle-publish" from org 9.6 to 9.7 -- deleting source files and not tangling properly

[Ihor Radchenko]

Gilles Marait <gilles.marait@inria.fr> writes:

> I encountered a change of behavior since org version 9.7 than removes
> my source files instead of tangling them at the correct location, using
> "org-babel-tangle-publish".
>
> I have not found in the changelog any description of something that
> could explain this change, so I report it here.
> ...

I suspect that the regression is after
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=478576749d

May I know if you end up with _two_ code.cpp files in Org 9.6 - one in
"." directory and one in src/ directory?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Change behavior of "org-babel-tangle-publish" from org 9.6 to 9.7 -- deleting source files and not tangling properly

[Gilles Marait]

Hi Igor,

Thanks for your answer. 
Yes, with org 9.6, I end up with two code.cpp as your said, one in "." and one in "src".

I think the line you're pointing to is indeed the problem

----- Mail original -----
> De: "Ihor Radchenko" <yantar92@posteo.net>
> À: "Gilles Marait" <gilles.marait@inria.fr>
> Cc: "emacs-orgmode" <emacs-orgmode@gnu.org>, "emmanuel agullo" <emmanuel.agullo@inria.fr>
> Envoyé: Lundi 9 Septembre 2024 19:40:27
> Objet: Re: Change behavior of "org-babel-tangle-publish" from org 9.6 to 9.7 -- deleting source files and not tangling
> properly

> Gilles Marait <gilles.marait@inria.fr> writes:
> 
>> I encountered a change of behavior since org version 9.7 than removes
>> my source files instead of tangling them at the correct location, using
>> "org-babel-tangle-publish".
>>
>> I have not found in the changelog any description of something that
>> could explain this change, so I report it here.
>> ...
> 
> I suspect that the regression is after
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=478576749d
> 
> May I know if you end up with _two_ code.cpp files in Org 9.6 - one in
> "." directory and one in src/ directory?
> 
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>

Re: Change behavior of "org-babel-tangle-publish" from org 9.6 to 9.7 -- deleting source files and not tangling properly

[Ihor Radchenko]

Gilles Marait <gilles.marait@inria.fr> writes:

> Hi Igor,
>
> Thanks for your answer. 
> Yes, with org 9.6, I end up with two code.cpp as your said, one in "." and one in "src".
>
> I think the line you're pointing to is indeed the problem

This is a bit tricky.

The new behavior is not exactly a bug - you just relied upon
undocumented behavior of `org-babel-tangle-publish' not cleaning up the
tangled files.

On the other hand, I can see tangled files being removed during
publishing as unexpected - what if the same set of files is used for
actual tangling (e.g. of config files). In such scenario, the configs
will be moved, potentially breaking workflows.

IMHO, it makes more sense to preserve _and also document_ (in the
docstring) the old behavior. We just need to make sure that the bug
fixed in 478576749d does not re-surface - when publishing directory is
"." the old code failed trying to copy tangled file into self.

I think that we can simply call `org-publish-attachment' on every
tangled file in `org-babel-tangle-publish' - `org-publish-attachment'
takes care about the situation when the tangled file is already in the
publishing dir.

Would you be interested to submit a patch fixing the issue as I described?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>