ob-sql is not finding psql when using direnv+guix

ob-sql is not finding psql when using direnv+guix

[Adolfo De Unanue]

[-- Attachment #1: Type: text/plain, Size: 2372 bytes --]

Hi

I am using Guix with direnv. In an specific folder I am installing and using psql and postgresql using direnv+guix as follows:


use guix --manifest=cdpp-manifest.scm

export PGUSER=food_user
export PGPASSWORD=some_password
export PGDATABASE=food

layout postgres


Where cdpp-manifest.scm contains the following:

(specifications->manifest
'("python"
   "python-pandas"
   "python-numpy"
   "python-flask"
   "python-graphene"
   "postgresql"
   "jupyter"))

I am able to use sql-mode and run queries against the database, in order to achieve that I am using the following .dir-locals.el

;;; Directory Local Variables
;;; For more information see (info "(emacs) Directory Variables")


((nil .
      ((projectile-project-test-cmd . "pytest --color=no --failed-first --maxfail=5")))
(python-mode .
              ((python-shell-buffer-name . "Python [CDPP-Inspecciones]")))

(org-mode . (
              (indent-tabs-mode . nil)
              (org-src-preserve-indentation . t)
              (org-footnote-auto-adjust . t)
              (org-footnote-auto-label . t)
              (ispell-local-dictionary . "spanish")
              (org-export-allow-bind-keywords . t)
              (org-footnote-define-inline . nil)
              (org-footnote-section . "Footnotes")))

(sql-mode . ((sql-connection-alist . ((mydb
                                        (sql-product 'postgres)
                                        (sql-database "mydb")
                                        (sql-user "db_user")
                                        (sql-server (expand-file-name ".direnv/postgres"))
                                        (sql-port 5432)
                                        )
                                      )))))

But If I try to use an sql org-babel block

#+begin_src sql
select 1;
#+end_src

(I am setting the connection variables in a PROPERTY)

I get the error: `psql is not found`


I was reading about the variable sql-postgres-program, so if I set the following in dir-locals.el

(sql-postgres-program . "/gnu/store/f2v92bkx2vfzmkl14qxj3hlmby4dy9x0-profile/bin/psql")

It works (note that psql ONLY lives inside the profile defined by direnv+guix), but I don't like the idea of hardcode the path.

Is there a better way?

Ideally I will expect that the org block will read it from the environment, but is not working. 

Thanks in advance

[-- Attachment #2: Type: text/html, Size: 5686 bytes --]

RE: ob-sql is not finding psql when using direnv+guix

[Cook, Malcolm]

>I am using Guix with direnv.

What is your shell?

How/When do you "hook direnv into your shell" (https://direnv.net/)?

> In an specific folder I am installing and using psql and postgresql using direnv+guix as follows:
>use guix --manifest=cdpp-manifest.scm
>
>export PGUSER=food_user
>export PGPASSWORD=some_password
>export PGDATABASE=food
>
>layout postgres

When are you doing this?

 a) in the "specific folder"'s .envrc file?
 b) in an org-mode shell block that you execute prior to the sql block?
 c) ??


You seem to be following [Per\-project Postgres](https://jamey.thesharps.us/2019/05/29/per-project-postgres/) but with guix instead of nix.  Nice.

If you enter the directory and then call emacs, earthing should just work, no changes neede.

If you want to tell emacs to adopt the environment established by a .direnv, you probably want [direnv integration for emacs\.](https://github.com/wbolster/emacs-direnv)

I've got to try this!

Cheers

>
>
>Where cdpp-manifest.scm contains the following:
>
>(specifications->manifest
>'("python"
>   "python-pandas"
>   "python-numpy"
>   "python-flask"
>   "python-graphene"
>   "postgresql"
>   "jupyter"))
>
>I am able to use sql-mode and run queries against the database, in order to achieve that I am using the following .dir-locals.el
>
>;;; Directory Local Variables
>;;; For more information see (info "(emacs) Directory Variables")
>
>
>((nil .
>      ((projectile-project-test-cmd . "pytest --color=no --failed-first --maxfail=5")))
>(python-mode .
>              ((python-shell-buffer-name . "Python [CDPP-Inspecciones]")))
>
>(org-mode . (
>              (indent-tabs-mode . nil)
>              (org-src-preserve-indentation . t)
>              (org-footnote-auto-adjust . t)
>              (org-footnote-auto-label . t)
>              (ispell-local-dictionary . "spanish")
>              (org-export-allow-bind-keywords . t)
>              (org-footnote-define-inline . nil)
>              (org-footnote-section . "Footnotes")))
>
>(sql-mode . ((sql-connection-alist . ((mydb
>                                        (sql-product 'postgres)
>                                        (sql-database "mydb")
>                                        (sql-user "db_user")
>                                        (sql-server (expand-file-name ".direnv/postgres"))
>                                        (sql-port 5432)
>                                        )
>                                      )))))
>
>But If I try to use an sql org-babel block
>
>#+begin_src sql
>select 1;
>#+end_src
>
>(I am setting the connection variables in a PROPERTY)
>
>I get the error: `psql is not found`
>
>
>I was reading about the variable sql-postgres-program, so if I set the following in dir-locals.el
>
>(sql-postgres-program . "/gnu/store/f2v92bkx2vfzmkl14qxj3hlmby4dy9x0-profile/bin/psql")
>
>It works (note that psql ONLY lives inside the profile defined by direnv+guix), but I don't like the idea of hardcode the path.
>
>Is there a better way?
>
>Ideally I will expect that the org block will read it from the environment, but is not working. 
>
>Thanks in advance
>

Re: ob-sql is not finding psql when using direnv+guix

[Adolfo De Unanue]

Dear Malcom

On Mon, May 10, 2021, at 13:14, Cook, Malcolm wrote:
> >I am using Guix with direnv.
> 
> What is your shell?
> 

My shell is bash, originally I was using zsh and I thought that was the problem, so I switched to bash and still not working.

> How/When do you "hook direnv into your shell" (https://direnv.net/)?
> 

In the .profile file, at login.

> > In an specific folder I am installing and using psql and postgresql using direnv+guix as follows:
> >use guix --manifest=cdpp-manifest.scm
> >
> >export PGUSER=food_user
> >export PGPASSWORD=some_password
> >export PGDATABASE=food
> >
> >layout postgres
> 
> When are you doing this?
> 
>  a) in the "specific folder"'s .envrc file?
>  b) in an org-mode shell block that you execute prior to the sql block?
>  c) ??
> 
>

Option (a)

 
> You seem to be following [Per\-project 
> Postgres](https://jamey.thesharps.us/2019/05/29/per-project-postgres/) 
> but with guix instead of nix.  Nice.
> 

Yep, great post. direnv + guix change the way I develop software, do data science and write lectures and papers.

> If you enter the directory and then call emacs, earthing should just 
> work, no changes neede.
> 

It works for almost everything (sql-buffers, python buffers, etc), except for org-babel sql blocks.

I am using EXWM, so emacs is always on.


> If you want to tell emacs to adopt the environment established by a 
> .direnv, you probably want [direnv integration for 
> emacs\.](https://github.com/wbolster/emacs-direnv)
> 

emacs-direnv was my first choice, but then I changed to envrc (https://github.com/purcell/envrc) . In both I got the error.

> I've got to try this!
> 
> Cheers
> 

Please tell me how that works for you

Thanks

- A


> >
> >
> >Where cdpp-manifest.scm contains the following:
> >
> >(specifications->manifest
> >'("python"
> >   "python-pandas"
> >   "python-numpy"
> >   "python-flask"
> >   "python-graphene"
> >   "postgresql"
> >   "jupyter"))
> >
> >I am able to use sql-mode and run queries against the database, in order to achieve that I am using the following .dir-locals.el
> >
> >;;; Directory Local Variables
> >;;; For more information see (info "(emacs) Directory Variables")
> >
> >
> >((nil .
> >      ((projectile-project-test-cmd . "pytest --color=no --failed-first --maxfail=5")))
> >(python-mode .
> >              ((python-shell-buffer-name . "Python [CDPP-Inspecciones]")))
> >
> >(org-mode . (
> >              (indent-tabs-mode . nil)
> >              (org-src-preserve-indentation . t)
> >              (org-footnote-auto-adjust . t)
> >              (org-footnote-auto-label . t)
> >              (ispell-local-dictionary . "spanish")
> >              (org-export-allow-bind-keywords . t)
> >              (org-footnote-define-inline . nil)
> >              (org-footnote-section . "Footnotes")))
> >
> >(sql-mode . ((sql-connection-alist . ((mydb
> >                                        (sql-product 'postgres)
> >                                        (sql-database "mydb")
> >                                        (sql-user "db_user")
> >                                        (sql-server (expand-file-name ".direnv/postgres"))
> >                                        (sql-port 5432)
> >                                        )
> >                                      )))))
> >
> >But If I try to use an sql org-babel block
> >
> >#+begin_src sql
> >select 1;
> >#+end_src
> >
> >(I am setting the connection variables in a PROPERTY)
> >
> >I get the error: `psql is not found`
> >
> >
> >I was reading about the variable sql-postgres-program, so if I set the following in dir-locals.el
> >
> >(sql-postgres-program . "/gnu/store/f2v92bkx2vfzmkl14qxj3hlmby4dy9x0-profile/bin/psql")
> >
> >It works (note that psql ONLY lives inside the profile defined by direnv+guix), but I don't like the idea of hardcode the path.
> >
> >Is there a better way?
> >
> >Ideally I will expect that the org block will read it from the environment, but is not working. 
> >
> >Thanks in advance
> >
> 

RE: ob-sql is not finding psql when using direnv+guix

[Cook, Malcolm]

> > >I am using Guix with direnv.
> > 
> > What is your shell?
> > 
> 
> My shell is bash, originally I was using zsh and I thought that was the problem, so I switched to bash and still not working.
> 
> > How/When do you "hook direnv into your shell" (https://direnv.net/)?
> > 
> 
> In the .profile file, at login.
> 
> > > In an specific folder I am installing and using psql and postgresql using direnv+guix as follows:
> > >use guix --manifest=cdpp-manifest.scm
> > >
> > >export PGUSER=food_user
> > >export PGPASSWORD=some_password
> > >export PGDATABASE=food
> > >
> > >layout postgres
> > 
> > When are you doing this?
> > 
> > a) in the "specific folder"'s .envrc file?
> > b) in an org-mode shell block that you execute prior to the sql block?
> > c) ??
> 
> Option (a)
> 
> 
> > You seem to be following [Per\-project 
> > Postgres](https://jamey.thesharps.us/2019/05/29/per-project-postgres/) 
> > but with guix instead of nix. Nice.
> > 
> 
> Yep, great post. direnv + guix change the way I develop software, do data science and write lectures and papers.
> 
> > If you enter the directory and then call emacs, earthing should just 
> > work, no changes neede.
> > 
> 
> It works for almost everything (sql-buffers, python buffers, etc), except for org-babel sql blocks.
> 
> I am using EXWM, so emacs is always on.
> 
> 
> > If you want to tell emacs to adopt the environment established by a 
> > .direnv, you probably want [direnv integration for 
> > emacs\.](https://github.com/wbolster/emacs-direnv)
> > 
> 
> emacs-direnv was my first choice, but then I changed to envrc (https://github.com/purcell/envrc) . In both I got the error.
> 

I see.  Envrc looks superior.  Good to know.

You've covered all my bases.  Shooting in the dark,  I would confirm/check the following:

How do you know that envrc is working otherwise?

When you switch to a direnv controlled buffer do you get any informative diagnostics in *Messages*?

Do you get any more if you first `(setq envrc-debug t)`?

Is this buffer in the same directory as the .direnvrc file?

Presumably TRAMP is not in the mix in any way.

Variable sql-postgres-program is unchanged from default value of psql

What is result of

	m-x shell-command
	which psql

and does it change as expected/required depending on the current active .direnvrc?

What happens if you eval (getenv 'psql) while in buffer governed by direnv, or alternatively

	m-x getenv
	psql

If you `m-x shell`, does normal direnv work as expected in the created shell?

> > I've got to try this!
> > 
> > Cheers
> > 
> 
> Please tell me how that works for you

Gotta get guix rolling again.  Hope to do this RSN.

Cheers,

~Malcolm

> 
> Thanks
> 
> - A
> 
> 
> > >
> > >
> > >Where cdpp-manifest.scm contains the following:
> > >
> > >(specifications->manifest
> > >'("python"
> > >   "python-pandas"
> > >   "python-numpy"
> > >   "python-flask"
> > >   "python-graphene"
> > >   "postgresql"
> > >   "jupyter"))
> > >
> > >I am able to use sql-mode and run queries against the database, in order to achieve that I am using the following .dir-locals.el
> > >
> > >;;; Directory Local Variables
> > >;;; For more information see (info "(emacs) Directory Variables")
> > >
> > >
> > >((nil .
> > >      ((projectile-project-test-cmd . "pytest --color=no --failed-first --maxfail=5")))
> > >(python-mode .
> > >              ((python-shell-buffer-name . "Python [CDPP-Inspecciones]")))
> > >
> > >(org-mode . (
> > >              (indent-tabs-mode . nil)
> > >              (org-src-preserve-indentation . t)
> > >              (org-footnote-auto-adjust . t)
> > >              (org-footnote-auto-label . t)
> > >              (ispell-local-dictionary . "spanish")
> > >              (org-export-allow-bind-keywords . t)
> > >              (org-footnote-define-inline . nil)
> > >              (org-footnote-section . "Footnotes")))
> > >
> > >(sql-mode . ((sql-connection-alist . ((mydb
> > >                                        (sql-product 'postgres)
> > >                                        (sql-database "mydb")
> > >                                        (sql-user "db_user")
> > >                                        (sql-server (expand-file-name ".direnv/postgres"))
> > >                                        (sql-port 5432)
> > >                                        )
> > >                                      )))))
> > >
> > >But If I try to use an sql org-babel block
> > >
> > >#+begin_src sql
> > >select 1;
> > >#+end_src
> > >
> > >(I am setting the connection variables in a PROPERTY)
> > >
> > >I get the error: `psql is not found`
> > >
> > >
> > >I was reading about the variable sql-postgres-program, so if I set the following in dir-locals.el
> > >
> > >(sql-postgres-program . "/gnu/store/f2v92bkx2vfzmkl14qxj3hlmby4dy9x0-profile/bin/psql")
> > >
> > >It works (note that psql ONLY lives inside the profile defined by direnv+guix), but I don't like the idea of hardcode the path.
> > >
> > >Is there a better way?
> > >
> > >Ideally I will expect that the org block will read it from the environment, but is not working. 
> > >
> > >Thanks in advance
> > >
> > 
> 

org-babel not finding executables when using direnv [Was: Re: ob-sql is not finding psql when using direnv+guix]

[Adolfo De Unanue]

On Wed, May 12, 2021, at 08:49, Cook, Malcolm wrote:
> > > >I am using Guix with direnv.
> > > 
> > > What is your shell?
> > > 
> > 
> > My shell is bash, originally I was using zsh and I thought that was the problem, so I switched to bash and still not working.
> > 
> > > How/When do you "hook direnv into your shell" (https://direnv.net/)?
> > > 
> > 
> > In the .profile file, at login.
> > 
> > > > In an specific folder I am installing and using psql and postgresql using direnv+guix as follows:
> > > >use guix --manifest=cdpp-manifest.scm
> > > >
> > > >export PGUSER=food_user
> > > >export PGPASSWORD=some_password
> > > >export PGDATABASE=food
> > > >
> > > >layout postgres
> > > 
> > > When are you doing this?
> > > 
> > > a) in the "specific folder"'s .envrc file?
> > > b) in an org-mode shell block that you execute prior to the sql block?
> > > c) ??
> > 
> > Option (a)
> > 
> > 
> > > You seem to be following [Per\-project 
> > > Postgres](https://jamey.thesharps.us/2019/05/29/per-project-postgres/) 
> > > but with guix instead of nix. Nice.
> > > 
> > 
> > Yep, great post. direnv + guix change the way I develop software, do data science and write lectures and papers.
> > 
> > > If you enter the directory and then call emacs, earthing should just 
> > > work, no changes neede.
> > > 
> > 
> > It works for almost everything (sql-buffers, python buffers, etc), except for org-babel sql blocks.
> > 
> > I am using EXWM, so emacs is always on.
> > 
> > 
> > > If you want to tell emacs to adopt the environment established by a 
> > > .direnv, you probably want [direnv integration for 
> > > emacs\.](https://github.com/wbolster/emacs-direnv)
> > > 
> > 
> > emacs-direnv was my first choice, but then I changed to envrc (https://github.com/purcell/envrc) . In both I got the error.
> > 
> 
> I see.  Envrc looks superior.  Good to know.
> 
> You've covered all my bases.  Shooting in the dark,  I would 
> confirm/check the following:
> 

I have news:

It fails for python too. Using the same files as before and this block:

#+begin_src python 
import pandas as pd
import matplotlib.pyplot as plt
#+end_src

It finds matplotlib, but fails to find pandas. I tried the same trick as before (but with the python executable, no psql), added the line

    (org-babel-python-command . ".direnv/.guix-profile/bin/python3")

but is not working. 

But if I am in shell, or in a python buffer or in a inferior-python process Emacs is finding all the libraries and executables.

So, I am assumming that the problem lies in how org-babel searchs the path ...

Answers to your questions below:


> How do you know that envrc is working otherwise?
> 
> When you switch to a direnv controlled buffer do you get any 
> informative diagnostics in *Messages*?
> 

Yes

> Do you get any more if you first `(setq envrc-debug t)`?
> 
Didn't try, Will do later

> Is this buffer in the same directory as the .direnvrc file?
> 

Yes

> Presumably TRAMP is not in the mix in any way.
>

No
 
> Variable sql-postgres-program is unchanged from default value of psql
> 
> What is result of
> 
> 	m-x shell-command
> 	which psql
> 

Is empty when I am not in a direnv controlled directory, the correct path when I enter to those directories.

> and does it change as expected/required depending on the current active 
> .direnvrc?
> 
> What happens if you eval (getenv 'psql) while in buffer governed by 
> direnv, or alternatively
> 
> 	m-x getenv
> 	psql
> 
> If you `m-x shell`, does normal direnv work as expected in the created shell?
> 

Yes, also works in vterm

> > > I've got to try this!
> > > 
> > > Cheers
> > > 
> > 
> > Please tell me how that works for you
> 
> Gotta get guix rolling again.  Hope to do this RSN.
> 
> Cheers,
> 
> ~Malcolm
> 
> > 
> > Thanks
> > 
> > - A
> > 
> > 
> > > >
> > > >
> > > >Where cdpp-manifest.scm contains the following:
> > > >
> > > >(specifications->manifest
> > > >'("python"
> > > >   "python-pandas"
> > > >   "python-numpy"
> > > >   "python-flask"
> > > >   "python-graphene"
> > > >   "postgresql"
> > > >   "jupyter"))
> > > >
> > > >I am able to use sql-mode and run queries against the database, in order to achieve that I am using the following .dir-locals.el
> > > >
> > > >;;; Directory Local Variables
> > > >;;; For more information see (info "(emacs) Directory Variables")
> > > >
> > > >
> > > >((nil .
> > > >      ((projectile-project-test-cmd . "pytest --color=no --failed-first --maxfail=5")))
> > > >(python-mode .
> > > >              ((python-shell-buffer-name . "Python [CDPP-Inspecciones]")))
> > > >
> > > >(org-mode . (
> > > >              (indent-tabs-mode . nil)
> > > >              (org-src-preserve-indentation . t)
> > > >              (org-footnote-auto-adjust . t)
> > > >              (org-footnote-auto-label . t)
> > > >              (ispell-local-dictionary . "spanish")
> > > >              (org-export-allow-bind-keywords . t)
> > > >              (org-footnote-define-inline . nil)
> > > >              (org-footnote-section . "Footnotes")))
> > > >
> > > >(sql-mode . ((sql-connection-alist . ((mydb
> > > >                                        (sql-product 'postgres)
> > > >                                        (sql-database "mydb")
> > > >                                        (sql-user "db_user")
> > > >                                        (sql-server (expand-file-name ".direnv/postgres"))
> > > >                                        (sql-port 5432)
> > > >                                        )
> > > >                                      )))))
> > > >
> > > >But If I try to use an sql org-babel block
> > > >
> > > >#+begin_src sql
> > > >select 1;
> > > >#+end_src
> > > >
> > > >(I am setting the connection variables in a PROPERTY)
> > > >
> > > >I get the error: `psql is not found`
> > > >
> > > >
> > > >I was reading about the variable sql-postgres-program, so if I set the following in dir-locals.el
> > > >
> > > >(sql-postgres-program . "/gnu/store/f2v92bkx2vfzmkl14qxj3hlmby4dy9x0-profile/bin/psql")
> > > >
> > > >It works (note that psql ONLY lives inside the profile defined by direnv+guix), but I don't like the idea of hardcode the path.
> > > >
> > > >Is there a better way?
> > > >
> > > >Ideally I will expect that the org block will read it from the environment, but is not working. 
> > > >
> > > >Thanks in advance
> > > >
> > > 
> > 
> 

RE: org-babel not finding executables when using direnv [Was: Re: ob-sql is not finding psql when using direnv+guix]

[Cook, Malcolm]

> > > > >I am using Guix with direnv.
> > > > 
> > > > What is your shell?
> > > > 
> > > 
> > > My shell is bash, originally I was using zsh and I thought that was the problem, so I switched to bash and still not working.
> > > 
> > > > How/When do you "hook direnv into your shell" (https://direnv.net/)?
> > > > 
> > > 
> > > In the .profile file, at login.
> > > 
> > > > > In an specific folder I am installing and using psql and postgresql using direnv+guix as follows:
> > > > >use guix --manifest=cdpp-manifest.scm
> > > > >
> > > > >export PGUSER=food_user
> > > > >export PGPASSWORD=some_password
> > > > >export PGDATABASE=food
> > > > >
> > > > >layout postgres
> > > > 
> > > > When are you doing this?
> > > > 
> > > > a) in the "specific folder"'s .envrc file?
> > > > b) in an org-mode shell block that you execute prior to the sql block?
> > > > c) ??
> > > 
> > > Option (a)
> > > 
> > > 
> > > > You seem to be following [Per\-project 
> > > > Postgres](https://jamey.thesharps.us/2019/05/29/per-project-postgres/) 
> > > > but with guix instead of nix. Nice.
> > > > 
> > > 
> > > Yep, great post. direnv + guix change the way I develop software, do data science and write lectures and papers.
> > > 
> > > > If you enter the directory and then call emacs, earthing should just 
> > > > work, no changes neede.
> > > > 
> > > 
> > > It works for almost everything (sql-buffers, python buffers, etc), except for org-babel sql blocks.
> > > 
> > > I am using EXWM, so emacs is always on.
> > > 
> > > 
> > > > If you want to tell emacs to adopt the environment established by a 
> > > > .direnv, you probably want [direnv integration for 
> > > > emacs\.](https://github.com/wbolster/emacs-direnv)
> > > > 
> > > 
> > > emacs-direnv was my first choice, but then I changed to envrc (https://github.com/purcell/envrc) . In both I got the error.
> > > 
> > 
> > I see. Envrc looks superior. Good to know.
> > 
> > You've covered all my bases. Shooting in the dark, I would 
> > confirm/check the following:
> > 
> 
> I have news:
> 
> It fails for python too. Using the same files as before and this block:
> 
> #+begin_src python 
> import pandas as pd
> import matplotlib.pyplot as plt
> #+end_src
> 
> It finds matplotlib, but fails to find pandas. I tried the same trick as before (but with the python executable, no psql), added the line
> 
> (org-babel-python-command . ".direnv/.guix-profile/bin/python3")
> 
> but is not working. 
> 
> But if I am in shell, or in a python buffer or in a inferior-python process Emacs is finding all the libraries and executables.
> 
> So, I am assumming that the problem lies in how org-babel searchs the path ...
> 
> Answers to your questions below:
> 
> 
> > How do you know that envrc is working otherwise?
> > 
> > When you switch to a direnv controlled buffer do you get any 
> > informative diagnostics in *Messages*?
> > 
> 
> Yes
> 
> > Do you get any more if you first `(setq envrc-debug t)`?
> > 
> Didn't try, Will do later
> 
> > Is this buffer in the same directory as the .direnvrc file?
> > 
> 
> Yes
> 
> > Presumably TRAMP is not in the mix in any way.
> >
> 
> No
> 
> > Variable sql-postgres-program is unchanged from default value of psql
> > 
> > What is result of
> > 
> > m-x shell-command
> > which psql
> > 
> 
> Is empty when I am not in a direnv controlled directory, the correct path when I enter to those directories.
> 
> > and does it change as expected/required depending on the current active 
> > .direnvrc?
> > 
> > What happens if you eval (getenv 'psql) while in buffer governed by 
> > direnv, or alternatively
> > 
> > m-x getenv
> > psql

Ok - all this suggest that purcess/envrc processing is correctly resetting PATH in the emacs environment, as desired, but it is not being picked up in the environment which emacs consults when resolving the 'psql' to its full path.

A little sleuthing leads to `comint-exec-1 `function which sets exec-path and calls > start-file-process > start-process  which is documented to search exec-path and not PATH.

So, apparently exec-path is not being set/reset by emacs correctly (or at all).

Following up on this, it looks like it is trying, looking at these 2 code results in purcell/envrc https://github.com/purcell/envrc/search?q=exec-path

But, let's follow up on this, tell me, what does emacs reply when you, both inside & outside your direnv controlled directory, eval  `(describe-variable "exec-path")`, or alternatively:

	c-h v exec-path

I think this will focus us on root cause....

~Malcolm

> > 
> > If you `m-x shell`, does normal direnv work as expected in the created shell?
> > 
> 
> Yes, also works in vterm
> > > > >
> > > > >
> > > > >Where cdpp-manifest.scm contains the following:
> > > > >
> > > > >(specifications->manifest
> > > > >'("python"
> > > > >   "python-pandas"
> > > > >   "python-numpy"
> > > > >   "python-flask"
> > > > >   "python-graphene"
> > > > >   "postgresql"
> > > > >   "jupyter"))
> > > > >
> > > > >I am able to use sql-mode and run queries against the database, in order to achieve that I am using the following .dir-locals.el
> > > > >
> > > > >;;; Directory Local Variables
> > > > >;;; For more information see (info "(emacs) Directory Variables")
> > > > >
> > > > >
> > > > >((nil .
> > > > >      ((projectile-project-test-cmd . "pytest --color=no --failed-first --maxfail=5")))
> > > > >(python-mode .
> > > > >              ((python-shell-buffer-name . "Python [CDPP-Inspecciones]")))
> > > > >
> > > > >(org-mode . (
> > > > >              (indent-tabs-mode . nil)
> > > > >              (org-src-preserve-indentation . t)
> > > > >              (org-footnote-auto-adjust . t)
> > > > >              (org-footnote-auto-label . t)
> > > > >              (ispell-local-dictionary . "spanish")
> > > > >              (org-export-allow-bind-keywords . t)
> > > > >              (org-footnote-define-inline . nil)
> > > > >              (org-footnote-section . "Footnotes")))
> > > > >
> > > > >(sql-mode . ((sql-connection-alist . ((mydb
> > > > >                                        (sql-product 'postgres)
> > > > >                                        (sql-database "mydb")
> > > > >                                        (sql-user "db_user")
> > > > >                                        (sql-server (expand-file-name ".direnv/postgres"))
> > > > >                                        (sql-port 5432)
> > > > >                                        )
> > > > >                                      )))))
> > > > >
> > > > >But If I try to use an sql org-babel block
> > > > >
> > > > >#+begin_src sql
> > > > >select 1;
> > > > >#+end_src
> > > > >
> > > > >(I am setting the connection variables in a PROPERTY)
> > > > >
> > > > >I get the error: `psql is not found`
> > > > >
> > > > >
> > > > >I was reading about the variable sql-postgres-program, so if I set the following in dir-locals.el
> > > > >
> > > > >(sql-postgres-program . "/gnu/store/f2v92bkx2vfzmkl14qxj3hlmby4dy9x0-profile/bin/psql")
> > > > >
> > > > >It works (note that psql ONLY lives inside the profile defined by direnv+guix), but I don't like the idea of hardcode the path.
> > > > >
> > > > >Is there a better way?
> > > > >
> > > > >Ideally I will expect that the org block will read it from the environment, but is not working. 
> > > > >
> > > > >Thanks in advance
> > > > >
> > > > 
> > > 
> > 
> 

Re: org-babel not finding executables when using direnv [Was: Re: ob-sql is not finding psql when using direnv+guix]

[Tim Cross]

"Adolfo De Unanue" <adolfo+org@unanue.mx> writes:

> On Wed, May 12, 2021, at 08:49, Cook, Malcolm wrote:
>> > > >I am using Guix with direnv.
>> > > 
>> > > What is your shell?
>> > > 
>> > 
>> > My shell is bash, originally I was using zsh and I thought that was the
>> > problem, so I switched to bash and still not working.
>> > 
>> > > How/When do you "hook direnv into your shell" (https://direnv.net/)?
>> > > 
>> > 
>> > In the .profile file, at login.
>> > 
>> > > > In an specific folder I am installing and using psql and postgresql using direnv+guix as follows:
>> > > >use guix --manifest=cdpp-manifest.scm
>> > > >
>> > > >export PGUSER=food_user
>> > > >export PGPASSWORD=some_password
>> > > >export PGDATABASE=food
>> > > >
>> > > >layout postgres
>> > > 
>> > > When are you doing this?
>> > > 
>> > > a) in the "specific folder"'s .envrc file?
>> > > b) in an org-mode shell block that you execute prior to the sql block?
>> > > c) ??
>> > 
>> > Option (a)
>> > 
>> > 
>> > > You seem to be following [Per\-project 
>> > > Postgres](https://jamey.thesharps.us/2019/05/29/per-project-postgres/) 
>> > > but with guix instead of nix. Nice.
>> > > 
>> > 
>> > Yep, great post. direnv + guix change the way I develop software, do data science and write lectures and papers.
>> > 
>> > > If you enter the directory and then call emacs, earthing should just 
>> > > work, no changes neede.
>> > > 
>> > 
>> > It works for almost everything (sql-buffers, python buffers, etc), except for org-babel sql blocks.
>> > 
>> > I am using EXWM, so emacs is always on.
>> > 
>> > 
>> > > If you want to tell emacs to adopt the environment established by a 
>> > > .direnv, you probably want [direnv integration for 
>> > > emacs\.](https://github.com/wbolster/emacs-direnv)
>> > > 
>> > 
>> > emacs-direnv was my first choice, but then I changed to envrc
>> > (https://github.com/purcell/envrc) . In both I got the error.
>> > 
>> 
>> I see.  Envrc looks superior.  Good to know.
>> 
>> You've covered all my bases.  Shooting in the dark,  I would 
>> confirm/check the following:
>> 
>
> I have news:
>
> It fails for python too. Using the same files as before and this block:
>
> #+begin_src python 
> import pandas as pd
> import matplotlib.pyplot as plt
> #+end_src
>
> It finds matplotlib, but fails to find pandas. I tried the same trick as before
> (but with the python executable, no psql), added the line
>
>     (org-babel-python-command . ".direnv/.guix-profile/bin/python3")
>
> but is not working. 
>
> But if I am in shell, or in a python buffer or in a inferior-python process
> Emacs is finding all the libraries and executables.
>
> So, I am assumming that the problem lies in how org-babel searchs the path ...

This is very unlikely. If there were issues in this area, we would be
seeing many more bug reports. More likely is that the PATH variable
doesn't contain what you think it does. First thing I would do is try
running the following source block

+being_src emacs-lisp
  (getenv "PATH")
+end_src

and verify the direnv bin directory is actually in your path.

Emacs will inherit the path from the shell running when emacs is
started. Changes made in your shell after Emacs has started will have no
effect on the PATH variable for emacs.

You mentioned at one point your running exwm. If you are not running
exwm in a login shell and your not sourcing your ~/.profile prior to
starting Emacs/exwm (assuming the direnv settings and path are setup
there), then the path variable is not going to have the direnv bin
directory. Either you need to ensure the appropriate direnv bin
directory is in your path before starting exwm or you need to add a
configuration step (e.g. possibly a hook function) which will add the
direnv bin directory using setenv.



-- 
Tim Cross