RE: org-babel not finding executables when using direnv [Was: Re: ob-sql is not finding psql when using direnv+guix]

["Cook, Malcolm" <MEC@stowers.org>]

> > > > >I am using Guix with direnv.
> > > > 
> > > > What is your shell?
> > > > 
> > > 
> > > My shell is bash, originally I was using zsh and I thought that was the problem, so I switched to bash and still not working.
> > > 
> > > > How/When do you "hook direnv into your shell" (https://direnv.net/)?
> > > > 
> > > 
> > > In the .profile file, at login.
> > > 
> > > > > In an specific folder I am installing and using psql and postgresql using direnv+guix as follows:
> > > > >use guix --manifest=cdpp-manifest.scm
> > > > >
> > > > >export PGUSER=food_user
> > > > >export PGPASSWORD=some_password
> > > > >export PGDATABASE=food
> > > > >
> > > > >layout postgres
> > > > 
> > > > When are you doing this?
> > > > 
> > > > a) in the "specific folder"'s .envrc file?
> > > > b) in an org-mode shell block that you execute prior to the sql block?
> > > > c) ??
> > > 
> > > Option (a)
> > > 
> > > 
> > > > You seem to be following [Per\-project 
> > > > Postgres](https://jamey.thesharps.us/2019/05/29/per-project-postgres/) 
> > > > but with guix instead of nix. Nice.
> > > > 
> > > 
> > > Yep, great post. direnv + guix change the way I develop software, do data science and write lectures and papers.
> > > 
> > > > If you enter the directory and then call emacs, earthing should just 
> > > > work, no changes neede.
> > > > 
> > > 
> > > It works for almost everything (sql-buffers, python buffers, etc), except for org-babel sql blocks.
> > > 
> > > I am using EXWM, so emacs is always on.
> > > 
> > > 
> > > > If you want to tell emacs to adopt the environment established by a 
> > > > .direnv, you probably want [direnv integration for 
> > > > emacs\.](https://github.com/wbolster/emacs-direnv)
> > > > 
> > > 
> > > emacs-direnv was my first choice, but then I changed to envrc (https://github.com/purcell/envrc) . In both I got the error.
> > > 
> > 
> > I see. Envrc looks superior. Good to know.
> > 
> > You've covered all my bases. Shooting in the dark, I would 
> > confirm/check the following:
> > 
> 
> I have news:
> 
> It fails for python too. Using the same files as before and this block:
> 
> #+begin_src python 
> import pandas as pd
> import matplotlib.pyplot as plt
> #+end_src
> 
> It finds matplotlib, but fails to find pandas. I tried the same trick as before (but with the python executable, no psql), added the line
> 
> (org-babel-python-command . ".direnv/.guix-profile/bin/python3")
> 
> but is not working. 
> 
> But if I am in shell, or in a python buffer or in a inferior-python process Emacs is finding all the libraries and executables.
> 
> So, I am assumming that the problem lies in how org-babel searchs the path ...
> 
> Answers to your questions below:
> 
> 
> > How do you know that envrc is working otherwise?
> > 
> > When you switch to a direnv controlled buffer do you get any 
> > informative diagnostics in *Messages*?
> > 
> 
> Yes
> 
> > Do you get any more if you first `(setq envrc-debug t)`?
> > 
> Didn't try, Will do later
> 
> > Is this buffer in the same directory as the .direnvrc file?
> > 
> 
> Yes
> 
> > Presumably TRAMP is not in the mix in any way.
> >
> 
> No
> 
> > Variable sql-postgres-program is unchanged from default value of psql
> > 
> > What is result of
> > 
> > m-x shell-command
> > which psql
> > 
> 
> Is empty when I am not in a direnv controlled directory, the correct path when I enter to those directories.
> 
> > and does it change as expected/required depending on the current active 
> > .direnvrc?
> > 
> > What happens if you eval (getenv 'psql) while in buffer governed by 
> > direnv, or alternatively
> > 
> > m-x getenv
> > psql

Ok - all this suggest that purcess/envrc processing is correctly resetting PATH in the emacs environment, as desired, but it is not being picked up in the environment which emacs consults when resolving the 'psql' to its full path.

A little sleuthing leads to `comint-exec-1 `function which sets exec-path and calls > start-file-process > start-process  which is documented to search exec-path and not PATH.

So, apparently exec-path is not being set/reset by emacs correctly (or at all).

Following up on this, it looks like it is trying, looking at these 2 code results in purcell/envrc https://github.com/purcell/envrc/search?q=exec-path

But, let's follow up on this, tell me, what does emacs reply when you, both inside & outside your direnv controlled directory, eval  `(describe-variable "exec-path")`, or alternatively:

	c-h v exec-path

I think this will focus us on root cause....

~Malcolm

> > 
> > If you `m-x shell`, does normal direnv work as expected in the created shell?
> > 
> 
> Yes, also works in vterm
> > > > >
> > > > >
> > > > >Where cdpp-manifest.scm contains the following:
> > > > >
> > > > >(specifications->manifest
> > > > >'("python"
> > > > >   "python-pandas"
> > > > >   "python-numpy"
> > > > >   "python-flask"
> > > > >   "python-graphene"
> > > > >   "postgresql"
> > > > >   "jupyter"))
> > > > >
> > > > >I am able to use sql-mode and run queries against the database, in order to achieve that I am using the following .dir-locals.el
> > > > >
> > > > >;;; Directory Local Variables
> > > > >;;; For more information see (info "(emacs) Directory Variables")
> > > > >
> > > > >
> > > > >((nil .
> > > > >      ((projectile-project-test-cmd . "pytest --color=no --failed-first --maxfail=5")))
> > > > >(python-mode .
> > > > >              ((python-shell-buffer-name . "Python [CDPP-Inspecciones]")))
> > > > >
> > > > >(org-mode . (
> > > > >              (indent-tabs-mode . nil)
> > > > >              (org-src-preserve-indentation . t)
> > > > >              (org-footnote-auto-adjust . t)
> > > > >              (org-footnote-auto-label . t)
> > > > >              (ispell-local-dictionary . "spanish")
> > > > >              (org-export-allow-bind-keywords . t)
> > > > >              (org-footnote-define-inline . nil)
> > > > >              (org-footnote-section . "Footnotes")))
> > > > >
> > > > >(sql-mode . ((sql-connection-alist . ((mydb
> > > > >                                        (sql-product 'postgres)
> > > > >                                        (sql-database "mydb")
> > > > >                                        (sql-user "db_user")
> > > > >                                        (sql-server (expand-file-name ".direnv/postgres"))
> > > > >                                        (sql-port 5432)
> > > > >                                        )
> > > > >                                      )))))
> > > > >
> > > > >But If I try to use an sql org-babel block
> > > > >
> > > > >#+begin_src sql
> > > > >select 1;
> > > > >#+end_src
> > > > >
> > > > >(I am setting the connection variables in a PROPERTY)
> > > > >
> > > > >I get the error: `psql is not found`
> > > > >
> > > > >
> > > > >I was reading about the variable sql-postgres-program, so if I set the following in dir-locals.el
> > > > >
> > > > >(sql-postgres-program . "/gnu/store/f2v92bkx2vfzmkl14qxj3hlmby4dy9x0-profile/bin/psql")
> > > > >
> > > > >It works (note that psql ONLY lives inside the profile defined by direnv+guix), but I don't like the idea of hardcode the path.
> > > > >
> > > > >Is there a better way?
> > > > >
> > > > >Ideally I will expect that the org block will read it from the environment, but is not working. 
> > > > >
> > > > >Thanks in advance
> > > > >
> > > > 
> > > 
> > 
>