From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:c151::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id wBz0KbehT2DNfwAA0tVLHw (envelope-from ) for ; Mon, 15 Mar 2021 18:04:39 +0000 Received: from aspmx2.migadu.com ([2001:41d0:2:c151::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 4KSvJbehT2BgZQAAB5/wlQ (envelope-from ) for ; Mon, 15 Mar 2021 18:04:39 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx2.migadu.com (Postfix) with ESMTPS id 9DA8B22095 for ; Mon, 15 Mar 2021 19:04:38 +0100 (CET) Received: from localhost ([::1]:32922 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLra0-0001Px-SQ for larch@yhetil.org; Mon, 15 Mar 2021 14:04:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57420) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLrM0-0004Qn-Cp for emacs-orgmode@gnu.org; Mon, 15 Mar 2021 13:50:08 -0400 Received: from mail-ot1-x333.google.com ([2607:f8b0:4864:20::333]:36886) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lLrLw-00034k-56 for emacs-orgmode@gnu.org; Mon, 15 Mar 2021 13:50:06 -0400 Received: by mail-ot1-x333.google.com with SMTP id 75so7414554otn.4 for ; Mon, 15 Mar 2021 10:50:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to; bh=IElduPoY/xZGlV27DklXxKGX66SXR6nP+iQVGMb/R04=; b=PaRlMxyil/JXLDTQ2VDP+pROawW7NWiodHAfMk+BX3VWx0q28DThowPzpS5BS7Rme2 FBlZu4+pOZ1e+WvWgOxJMNhKCNnvghZ91y14RVWPJ7+bby2EjwUDAlWpPhJtshVaUQtr 20c2xjiELMX737YDLRgzwM6V/PNEGi9LzV0ls118n2lfjfbB2nxc0/+zKseL3Fajq7iN Y3K+/p0wwHUn7x1UJbcHQ5/mA3xClkmgFN62BAsLRif6QBQI3q7XY6AtZ1QA9uqYdatg nrG+ZzRnQoeDZKUbyJxMS5BDdfCW8TiCNlbRNJs2CnvAvBXAwpUJExsCjtooc+a7MGvz IALQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to; bh=IElduPoY/xZGlV27DklXxKGX66SXR6nP+iQVGMb/R04=; b=etfaz+GGGU0CFZl6j5rTRDAFrIUozyYGS9cC46qGwV/IkPa50C+z4DoNbftM9/++2c vgfEowlNelfCMwRflHAtIC15+WLWkuaFXeDaPN2LruFpSoxZQPm7WAMYPhxgIgw4o3b9 uB56QSTZVWsLNVEJ4aCplDEvMGwCXXuy6B5C9pMyM+O5kP3Lo1mOEyafyfQLKb78jm7s lsl+nLj825pqw12ga8CAcNhOQo6GZ8ZEuoXJpUqE488EFLXxNuSFmNcK6D6GGANsEy6E 5La/feEFEM0uQcv8IXOU+O6c8Ro8vqMNLnwvJcIb1JMaT7nRSgrsmDXG4luhjzts/2oJ hkDg== X-Gm-Message-State: AOAM531pZzJGauSWGOgWrAc5z1W4JDLvM7R4U4XKwzg3cPHxcn5b1WzN lkN9ZzFJwMKiFaJ8ZznOd/mXVH13peZufcKgASEvKlpwSwG+HQ== X-Google-Smtp-Source: ABdhPJxHaKSnYiNaonC/CT/JilSkV6Wp8I/DtJZLXEnN35srZz4lWYk8sm5R28vQxe6U/RDcFkAmz3ANF80xbaNVSME= X-Received: by 2002:a9d:6d8a:: with SMTP id x10mr209691otp.339.1615830602497; Mon, 15 Mar 2021 10:50:02 -0700 (PDT) MIME-Version: 1.0 References: <87a6r5l5bh.fsf@gmail.com> <87ft0x2hbe.fsf@gmail.com> In-Reply-To: <87ft0x2hbe.fsf@gmail.com> From: George Mauer Date: Mon, 15 Mar 2021 12:49:51 -0500 Message-ID: Subject: Re: How to get shell source blocks to read my profile? To: emacs-orgmode Content-Type: multipart/alternative; boundary="00000000000014410c05bd96e158" Received-SPF: pass client-ip=2607:f8b0:4864:20::333; envelope-from=gmauer@gmail.com; helo=mail-ot1-x333.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: gmauer@gmail.com Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1615831479; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=IElduPoY/xZGlV27DklXxKGX66SXR6nP+iQVGMb/R04=; b=apJA6VqWlhL6wSTO5J7dfYIm7SpniA5C6VerSr/c2DuyYuyJrDiwNifdtZg9VldSeOt+fg SW+iz3sjVsgUUwJ8UTxnOONXoZXBvGbUr4BbzM0Ye4qX3rKX0Tkq4gR7S81L37dF37sa7w vlqGFyxq/X8GUC2hFVjqfgZ/dIQBH9wdV3l8T8QSd44U3fNI6VBm6e5lTsWoDMDH+Eh0v2 wSJ5yegGG8W6EqiPoK4ZmWEXna/JNJIRdGGO8uH4qU220A8mzuKPuGI9RmPswq/jf9s6wz 23aPvKMEDBa/eb818j6Ge9tPsGl+RY+WV6QJ+Z/nLws0kNAUgsaIkqdLdGLdcw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1615831479; a=rsa-sha256; cv=none; b=A4GURLpmKpSIcH87KkLSV5hIX8fWgBaEluo0YPQ4JtxKH1qo8Jin/NIVFTskOinFD3uM32 AVbdPpF75xXsoZm1ksxcGkRU0VIvZ0GUuB8hQWtvxPULnLJzGt40VxvMsOdzy10QUKRjhV ax3Vycgis8/qiQQh7P0+b5KOAkmzdbeR087fVYnzvAq8lgS2RcbFDD9NQg9A1qPclqyWAf YB28t9zRjdnPaRYGDtCZS3w6iLTXkfESseziX6c5lkM6O9MPpXzlomDzXDppeyYjKQ6pXV uYJmQvIekcMrxiTTdBGPX+Dce3psq/WEedZ26jZMbuirEVqdvgkeJY5p2+9nIw== ARC-Authentication-Results: i=1; aspmx2.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=PaRlMxyi; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx2.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Spam-Score: -2.10 Authentication-Results: aspmx2.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=PaRlMxyi; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx2.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: 9DA8B22095 X-Spam-Score: -2.10 X-Migadu-Scanner: scn0.migadu.com X-TUID: y4muSqNpqC+v --00000000000014410c05bd96e158 Content-Type: text/plain; charset="UTF-8" Thanks a lot! The interactive/non-interactive was indeed the core issue. Extra frustrating because it seems like supplying `--rcfile` does nothing if you *do* use `-c` but *don't* use `-i`...ah ad-hoc cli design. As a general solution, I've found that adding this block in my org buffer will make shell (which is zsh in my case) run in interactive mode for all of these * Local Variables :noexport: :PROPERTIES: :VISIBILITY: folded :END: local variables: shell-file-name: "/bin/zsh -i" end: On Sun, Mar 14, 2021 at 11:04 PM Tim Cross wrote: > > comments in-line ... > > George Mauer writes: > > > Hey Tim, thanks for helping out. I commented inline to your response > below but I'll sum up and ask the outstanding questions more directly here > as well. > > > > I think you might have misread what I was doing - I had 3 different > variables set in 3 different places precisely because I want to dissect > things and figure > > out which profile/rc scripts were run. > > > > Yes, this is emacs running in GUI mode so yes, it doesn't have a shell > as a parent. But that only explains why these environment variables aren't > in > > emacs itself, not the spawned bash or zsh processes > > > > To clarify, the GUI mode is unrelated. You can run in GUI mode from a > terminal and the emacs process will inherit the environment in the > terminal. If that terminal is a login terminal (i.e. terminal shell was > run as a login shell), the 'profile' file will be sourced and the 'rc' > file will have been sourced. If not a login shell, only the rc file will > have been sourced. > > If on the other hand, Emacs is started from the dock, it is started in a > completely different process chain and is not a child of a login > process. This is why, if you want your emacs to have /usr/local/bin in > the path, you have to add it to /etc/paths or /etc/path.d whereas if you > start emacs from within a terminal (GUI or text mode), you only need to > add /usr/local/bin to your PATH setting in .profile (just an example to > highlight the difference between running from in a terminal and running > from the dock). > > > So to sum up the specific questions. My ~shell-file-name~ is ~/bin/zsh~ > which seems to be what ~ob-shell~ is passing down to be run. So why on earth > > does my ~GIM_ZSHRC~ variable not show up here? > > > > #+begin_src shell > > env > > #+end_src > > > > When I actually directly call bash > > > > #+begin_src shell > > bash -c env > > #+end_src > > > > Have a closer look on the section in the bash manual on the difference > between interactive and non-interactive shells. (also holds for zsh). > Basically, the 'rc' files are not sourced for non-interactive shells. > > > Why would the output not include ~GIM_BASHRC~ - it should have been run, > right? > > > > No. Adding the -c means it is a non-interactive shell, so no .bashrc > sourcing. > > > > What about when I call this? Even with explicitly selecting the rc file > to run, it seems to not > > > > #+begin_src shell > > bash --rcfile ~/.bashrc -c env > > #+end_src > > > > Same issue here. --rcfile only has effect in interactive shells. > > > Finally, the outstanding question about ~ob-shell~ is if there is any > way to force it to run the shell processes' rc-script? I really would have > expected it to > > be run in the above situations already... > > > > You could try sourcing it e.g. > > #+begin_src shell > source ~/.bashrc > env > #+end_src > > or use 'shorthand' dot > > #+begin_src shell > . ~/.bashrc > env > #+end_src > > there are also some options you can add at the 'shebang' line i.e. > > #!/bin/bash -l > > or > > #!/bin/bash -i > > which will change the behaviour. > > There is a lot of 'meat' in the bash man page and there is a lot of > additional information in the bash info pages. However, both can be a > bit terse and because the info is very 'dense', it is very easy to miss > key points. > > In order to have environment variables available inside your org source > blocks, you really need to > > - have them in the environment inherited by emacs when it starts. This > will depend how you start Emacs (i.e from within an interactive shell > vs from the dock). Note that you typically don't see this issue under > Linux because in most Linux setups, the X environment is started > inside a login shell. So everything started as part of the X session, > like a dock, is a child of the login process and therefore inherits > the login environment. On a mac, the dock is not part of the login > shell, so it only inherits what is in the system wide bash profile > file. > > - Ensure your source block run as interactive and/or login shells using > shebang options like -i or -l > > - explicitly source the .profile or .bashrc file using a source call > > - pass the environment variable in on the command line e.g. > > #+begin_src shell > FRED=barney env > #+end_src > > will result in > > FRED=barney > > being in the output from 'env'. > > > > ----------- > > > > Is this perhaps on a Mac where Emacs is started from the dock? > > > > Yup like I said, its in GUI mode so I understand why those environment > variables aren't within emads itself > > > > It is actually a little more subtle. It isn't because your running in > GUI mode those variables are not there. It is because your running from > the dock. Run it in GUI mode from within a login terminal and all those > variables will be 'in' emacs. > > > > > It is certainly confusing but I think I got a handle on it mostly. If > it's a login shell you'll run a profile, if it's not you'll run the default > rc file unless one of the > > options were specified. I think each is named specific to each shell > name but oftentimes you chain them together in practice. > > > > There is also a difference between interactive and non-interactive > shells. I suspect this is the root cause of your issue. The source block > being run are non-interactive. > > -- > Tim Cross > > --00000000000014410c05bd96e158 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks a lot! The interactive/non-interactive was indeed t= he core issue. Extra frustrating because it seems like supplying `--rcfile`= does nothing if you *do* use `-c` but *don't* use `-i`...ah ad-hoc cli= design.

As a general solution, I've found that addi= ng this block in my org buffer will make shell (which is zsh in my case) ru= n in interactive mode for all of these

* Local Var= iables :noexport:
=C2=A0 :PROPERTIES:
=C2=A0 :VISIBILITY: folded
= =C2=A0 :END:
=C2=A0 local variables:
=C2=A0 shell-file-name: "/b= in/zsh -i"
=C2=A0 end:

On Sun, Mar 14, 2021 at 11:04 PM Ti= m Cross <theophilusx@gmail.com<= /a>> wrote:
<= br> comments in-line ...

George Mauer <gmau= er@gmail.com> writes:

> Hey Tim, thanks for helping out. I commented inline to your response b= elow but I'll sum up and ask the outstanding questions more directly he= re as well.
>
> I think you might have misread what I was doing - I had 3 different va= riables set in 3 different places precisely because I want to dissect thing= s and figure
> out which profile/rc scripts were run.
>
> Yes, this is emacs running in GUI mode so yes, it doesn't have a s= hell as a parent. But that only explains why these environment variables ar= en't in
> emacs itself, not the spawned bash or zsh processes
>

To clarify, the GUI mode is unrelated. You can run in GUI mode from a
terminal and the emacs process will inherit the environment in the
terminal. If that terminal is a login terminal (i.e. terminal shell was
run as a login shell), the 'profile' file will be sourced and the &= #39;rc'
file will have been sourced. If not a login shell, only the rc file will have been sourced.

If on the other hand, Emacs is started from the dock, it is started in a completely different process chain and is not a child of a login
process. This is why, if you want your emacs to have /usr/local/bin in
the path, you have to add it to /etc/paths or /etc/path.d whereas if you start emacs from within a terminal (GUI or text mode), you only need to
add /usr/local/bin to your PATH setting in .profile (just an example to
highlight the difference between running from in a terminal and running
from the dock).

> So to sum up the specific questions. My ~shell-file-name~ is ~/bin/zsh= ~ which seems to be what ~ob-shell~ is passing down to be run. So why on ea= rth
> does my ~GIM_ZSHRC~ variable not show up here?
>
>=C2=A0 =C2=A0#+begin_src shell
>=C2=A0 =C2=A0 =C2=A0env
>=C2=A0 =C2=A0#+end_src
>
> When I actually directly call bash
>
>=C2=A0 #+begin_src shell
>=C2=A0 =C2=A0 =C2=A0bash -c env
>=C2=A0 =C2=A0#+end_src
>

Have a closer look on the section in the bash manual on the difference
between interactive and non-interactive shells. (also holds for zsh).
Basically, the 'rc' files are not sourced for non-interactive shell= s.

> Why would the output not include ~GIM_BASHRC~ - it should have been ru= n, right?
>

No. Adding the -c means it is a non-interactive shell, so no .bashrc
sourcing.


> What about when I call this? Even with explicitly selecting the rc fil= e to run, it seems to not
>
>=C2=A0 =C2=A0#+begin_src shell
>=C2=A0 =C2=A0 =C2=A0bash --rcfile ~/.bashrc -c env
>=C2=A0 =C2=A0#+end_src
>

Same issue here. --rcfile only has effect in interactive shells.

> Finally, the outstanding question about ~ob-shell~ is if there is any = way to force it to run the shell processes' rc-script? I really would h= ave expected it to
> be run in the above situations already...
>

You could try sourcing it e.g.

#+begin_src shell
source ~/.bashrc
env
#+end_src

or use 'shorthand' dot

#+begin_src shell
. ~/.bashrc
env
#+end_src

there are also some options you can add at the 'shebang' line i.e.<= br>
#!/bin/bash -l

or

#!/bin/bash -i

which will change the behaviour.

There is a lot of 'meat' in the bash man page and there is a lot of=
additional information in the bash info pages. However, both can be a
bit terse and because the info is very 'dense', it is very easy to = miss
key points.

In order to have environment variables available inside your org source
blocks, you really need to

- have them in the environment inherited by emacs when it starts. This
=C2=A0 will depend how you start Emacs (i.e from within an interactive shel= l
=C2=A0 vs from the dock). Note that you typically don't see this issue = under
=C2=A0 Linux because in most Linux setups, the X environment is started
=C2=A0 inside a login shell. So everything started as part of the X session= ,
=C2=A0 like a dock, is a child of the login process and therefore inherits<= br> =C2=A0 the login environment. On a mac, the dock is not part of the login =C2=A0 shell, so it only inherits what is in the system wide bash profile =C2=A0 file.

- Ensure your source block run as interactive and/or login shells using
=C2=A0 shebang options like -i or -l

- explicitly source the .profile or .bashrc file using a source call

- pass the environment variable in on the command line e.g.

#+begin_src shell
FRED=3Dbarney env
#+end_src

will result in

FRED=3Dbarney

being in the output from 'env'.


> -----------
>
>=C2=A0 Is this perhaps on a Mac where Emacs is started from the dock? >
> Yup like I said, its in GUI mode so I understand why those environment= variables aren't within emads itself
>

It is actually a little more subtle. It isn't because your running in GUI mode those variables are not there. It is because your running from
the dock. Run it in GUI mode from within a login terminal and all those
variables will be 'in' emacs.

<snip>

> It is certainly confusing but I think I got a handle on it mostly. If = it's a login shell you'll run a profile, if it's not you'll= run the default rc file unless one of the
> options were specified. I think each is named specific to each shell n= ame but oftentimes you chain them together in practice.
>

There is also a difference between interactive and non-interactive
shells. I suspect this is the root cause of your issue. The source block being run are non-interactive.

--
Tim Cross

--00000000000014410c05bd96e158--