From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 8CaLOhvITmCLRAAA0tVLHw (envelope-from ) for ; Mon, 15 Mar 2021 02:36:11 +0000 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 6BsvNhvITmBqUgAAbx9fmQ (envelope-from ) for ; Mon, 15 Mar 2021 02:36:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0B40729699 for ; Mon, 15 Mar 2021 03:36:11 +0100 (CET) Received: from localhost ([::1]:55768 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLd5V-0002Km-SF for larch@yhetil.org; Sun, 14 Mar 2021 22:36:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42528) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLd4p-0002JW-CK for emacs-orgmode@gnu.org; Sun, 14 Mar 2021 22:35:27 -0400 Received: from mail-ot1-x32b.google.com ([2607:f8b0:4864:20::32b]:36234) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lLd4m-00024A-Pn for emacs-orgmode@gnu.org; Sun, 14 Mar 2021 22:35:27 -0400 Received: by mail-ot1-x32b.google.com with SMTP id t16so6377781ott.3 for ; Sun, 14 Mar 2021 19:35:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:cc; bh=kQo4vWCXCSX+Tgx1UYu5zXvTvxyrUfC28WNuZPIoazk=; b=PNImcVk1PJ9fRFmAZpTrjJXu+q6AWNsn3FqRjQfdUDzPp9Z5JmH56JURQWBD6z0MaV YosCFkknM5qS2yZxQNW+79LAR16MWF3XZSLVzZ0z7+IuT7aB6O0wtl2mkAJYRGZZo/Bs CIaP+XIVD61nIL41fCz30uKEF730avZ4+djRcEHJf/SdGW1Q2qg1UDl7Xg8OBFqA1IYK gDF/i5xEl1qYAyn4ohTmQUUb57BRBCDxxoJls0bCypA/Ezivrog68ujZ7JBcx4EqXWes IW/YchzclWaP7N/SXmJD7iviL1i3MrOsCnHJZN7YrN2Rf9/CP2hifbvyOyv3hKpDw7ya 5zxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:cc; bh=kQo4vWCXCSX+Tgx1UYu5zXvTvxyrUfC28WNuZPIoazk=; b=ERB8msBSxSHbl3O5s1FA0HcpcW/q2yInrP0BN0GKyH96iuedp08e8+9BwijUq29TKz /JZyJS6QuUNLl+TwoR8SO8WprxXUdks8QtVcL9zsMOsrUNnQuInzlnp4gENzQ1yUpZKq GY/Uwzx6W9A/ryRRXmN0XqCoGWWcVFwZD5bviH/X28GNwO5lsKxDu5sWrkS50pOAuOau plmCDLB2ZCS5dx8kvr3xG936eNztmfg14qsEB9gbiAUv3K9PoQ/XvhLF1FYPzP2qxMg/ PxXTIBd980XG76rzBV5oMQyfTVUulTxaCz4W+1c1VUswMDtTuJcX2Dah4spAkPB6hNe1 JBVQ== X-Gm-Message-State: AOAM531xCwp8hMkRobtrtL/GQEG5PC063ONrZt8XUvUCm6RggjTIIVGb x8Btlh3+fVDritoIxXg1kzowk7d8jYX7PSBdfiS6MJ5GRTce5Q== X-Google-Smtp-Source: ABdhPJxR8LZiI3YOpz9TDzCNcsmdVVCK4zxy85O78MaJUVMM8/uU4jZXsUoh5oxJIw4gRdfaOFWd5Y9bUNLEy6ygYks= X-Received: by 2002:a05:6830:101a:: with SMTP id a26mr12285785otp.68.1615775723237; Sun, 14 Mar 2021 19:35:23 -0700 (PDT) MIME-Version: 1.0 References: <87a6r5l5bh.fsf@gmail.com> In-Reply-To: <87a6r5l5bh.fsf@gmail.com> From: George Mauer Date: Sun, 14 Mar 2021 21:35:12 -0500 Message-ID: Subject: Re: How to get shell source blocks to read my profile? Cc: emacs-orgmode Content-Type: multipart/alternative; boundary="00000000000005368505bd8a1a92" Received-SPF: pass client-ip=2607:f8b0:4864:20::32b; envelope-from=gmauer@gmail.com; helo=mail-ot1-x32b.google.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, MALFORMED_FREEMAIL=0.001, MISSING_HEADERS=1.021, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: gmauer@gmail.com Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1615775771; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=kQo4vWCXCSX+Tgx1UYu5zXvTvxyrUfC28WNuZPIoazk=; b=iNgUhYIYNVkv2d+tYdxfP0fA2FOUH58HFp6KM1x8VERxmQEsQ7dtJMQCk/2RU2Olgv8Or8 JXIuZVIsyvk+YrwzibijPBXSm8QFBRzCFGhrRdlwq698MI48Y3wdsXWwaHEu7CGtbF47EW e+XGl5jjzBl2Av5Bhf3yMTPI0uMLN+mB2CQBoVQKZ4EcLkhWyvWjJOUQ3uQeHmpwkKU/I+ z3KjtK4hllqXJrky+fRIXW3N/TSR93IuqlbixvJu8nBGllxGAaHQ/P0O/ZdoQ0NOUQk+rz d0L8lorxkW5kSXvKpctw2z8i+T0NsQVL72o11Nr+z3YhEPL8OWzPZkMv7oQxXA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1615775771; a=rsa-sha256; cv=none; b=Xsv+t+Eg3qmMXjDLO6gRzFE3hHULowjymYLVWVdUF4jpXeFFrsvkhmWSUBp/5gQGmCfyVO AEyp2odAK9kbuCBG1KSFx6V34Ouv9q+ChVNAKOA7FAXOzwia/O3HSCnxeLToZQlBQvw5SC KsK45RVcsdT48dF+B8MXKzHTBrAsoaZ5ixGyTeHze/kzh0g9EvdsEnAGB/h+3J7DD4uV27 08uGaOv4BZGYC5B0rOtSWyG55nVvhG+BL/zdjzo83fwC1X/TaO2DW94/Kth48JIRg7Q496 bJspjDrv3w92hSmRIp7JxTj5lJzKCo4lUIcxJqFWSItmyMVEW6RMDUeIiHW4yg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=PNImcVk1; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Spam-Score: -0.10 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=PNImcVk1; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: 0B40729699 X-Spam-Score: -0.10 X-Migadu-Scanner: scn0.migadu.com X-TUID: Nv0sfe3MFimT --00000000000005368505bd8a1a92 Content-Type: text/plain; charset="UTF-8" Hey Tim, thanks for helping out. I commented inline to your response below but I'll sum up and ask the outstanding questions more directly here as well. I think you might have misread what I was doing - I had 3 different variables set in 3 different places precisely because I want to dissect things and figure out which profile/rc scripts were run. Yes, this is emacs running in GUI mode so yes, it doesn't have a shell as a parent. But that only explains why these environment variables aren't in emacs itself, not the spawned bash or zsh processes So to sum up the specific questions. My ~shell-file-name~ is ~/bin/zsh~ which seems to be what ~ob-shell~ is passing down to be run. So why on earth does my ~GIM_ZSHRC~ variable not show up here? #+begin_src shell env #+end_src When I actually directly call bash #+begin_src shell bash -c env #+end_src Why would the output not include ~GIM_BASHRC~ - it should have been run, right? What about when I call this? Even with explicitly selecting the rc file to run, it seems to not #+begin_src shell bash --rcfile ~/.bashrc -c env #+end_src Finally, the outstanding question about ~ob-shell~ is if there is any way to force it to run the shell processes' rc-script? I really would have expected it to be run in the above situations already... ----------- > > Is this perhaps on a Mac where Emacs is started from the dock? Yup like I said, its in GUI mode so I understand why those environment variables aren't within emads itself > If so, it > could be because on the mac, applications started from the dock are not > executed inside a login shell. Do you get different results if you start > emacs from within a terminal? Yup I do, but thats why I move on to the test where I run `bash` directly > I am also a little confused by your examples. Some of them are > attempting to source .bash_profile, others .bashrc and others .zshrc. > I created the three environment variables I indicated and placed one in each file so that I could test which profile file ran > Which shell are you using and which files do you have? I get the feeling > that in your frustration, you have taken a 'shotgun' approach and added > the variable everywhere. That's exactly what I'm trying to avoid here. I *did* add variables everywhere, but I added *different* variables everywhere so that I can see exactly which profile ran. > For some reason, there seems to be some confusion these days about the > difference between .profile, .bash_profile, .bashrc, .zsh_profile and > .zshrc. In particular, I frequently see incorrect/wrong advice regarding > the use of 'profile' and 'rc' files. Most of it can > be blamed on wrong advice in forums like stack overflow! > It is certainly confusing but I think I got a handle on it mostly. If it's a login shell you'll run a profile, if it's not you'll run the default rc file unless one of the options were specified. I think each is named specific to each shell name but oftentimes you chain them together in practice. > I would also experiment with just dumping out the output of 'env' and > not run it through grep so that you can see what is being defined. I did, that just isn't conducive to posting on an email list ;) iI also think it is a good dea to be explicit about the shell so that you > have more > control/understanding over what is being run e.g. put the line > Yeah, I suppose I could do a prologue with a hashbang, but I debugged into the source and it's just ~shell-file-name~ which in my case is ~/bin/zsh~ In very broad terms (glossing over some of the subtleties, which is > likely the cause of some of your issues), the 'profile' files are only > sourced for login shells and the 'rc' files are sourced whenever a new > shell process is run. Note that these processes run in a hierarchy i.e. > shell processes have a parent process. Some things have to be set in > your 'profile' (or are better set there) while other things must be set > in your 'rc' file. For example, setting PATH in your 'profile' is normal > because you typically just want that set once and then 'exported' to all > child processes. Setting a dynamic prompt which changes depending on the > directory your in (or because it has some other dynamic data, like the > time) need to be set in your 'rc' file because these are sourced before > each shell process is executed. I the 'old days' it was important to get > these distinctions correct because sourcing the files could have a > performance impact - not as big an issue these days. > Great history, thanks, that's mostly what I was thinking but gives me useful background These days, things seem to have become even more confusing in an attempt > to make things easier. I often see people with a .profile, a > .bash_profile, a .bashrc, a .zsh_profile and a .zshrc with variables set > and exported mixed in across all these files. Don't forget that many tools from homebrew to pyenv install scripts to ansible will start modifying these for you > This is a source of > terrible confusion and unexpected results. Some 'canned' shell > configurations, like 'oh-my-zsh' also try to be extra helpful by > sourcing files which are not normally sourced by that shell (to make > smoother migration from bash shells for example). Sometimes, it can be > helpful to put a line like the following into each of these files > > echo "Executing at `date`" >> ~/profile.log > > just to see what is getting executed when (you won't want to leave this > there - just for diagnostic and experimentation to help understand when > things are being sourced. You could even dump out the output of 'env' so > that you can see how the environment is being changed as each of these > files executes. > > Yup, thanks. > George Mauer writes: > > > I am confused why no matter how I try to run shell commands they seem to > be missing variables exported in profiles. > > > > I have added 3 variables to various startup scripts > > > > - ~./bash-profile~ :: ~export GIM_BASH_PROFILE="yes"~ > > - ~./bashrc~ :: ~export GIM_BASHRC="yes"~ > > - ~./zshrc~ :: ~export GIM_ZSHRC="yes"~ > > > > I am running emacs in GUI mode so I get why none of these are available > *directly* in emacs, so then I try > > > > #+begin_src shell :results list > > env | grep GIM > > #+end_src > > > > #+RESULTS: > > > > Ok that's kinda surprising, but I suppose it could be running ~/bin/zsh~ > (that's my ~shell-file-name~) directly > > > > what about > > > > #+begin_src shell :results list > > bash -c env | grep GIM > > #+end_src > > > > #+RESULTS: > > > > That's pretty surprising. I would have expected running it directly to > actually run my profile. > > > > Shockingly > > > > #+begin_src shell :results list > > bash --rcfile ~/.bashrc -c env | grep GIM > > #+end_src > > > > #+RESULTS: > > > > That is *still* nothing! > > > > Sanity is restored slightly when I run > > > > #+begin_src shell :results list > > bash --login -c env | grep GIM > > #+end_src > > > > which *does* indeed visit ~.bash_profile~ but only slightly. > > > > What is going on, and is there a straightforward way in which I can get > shell block to read from a profile? > > > -- > Tim Cross > > --00000000000005368505bd8a1a92 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hey Tim, thanks for helpin= g out. I commented inline to your response below but I'll sum up and as= k the outstanding questions more=C2=A0directly here as well.

I thin= k you might have misread what I was doing - I had 3 different variables set= in 3 different places precisely because I want to dissect things and figur= e out which profile/rc scripts were run.

Yes, this is emacs running = in GUI mode so yes, it doesn't have a shell as a parent. But that only = explains why these environment=C2=A0variables aren't in emacs itself, n= ot the spawned bash or zsh processes

So to sum up the specific quest= ions. My ~shell-file-name~ is ~/bin/zsh~ which seems to be what ~ob-shell~ = is passing down to be run. So why on earth does my ~GIM_ZSHRC~ variable not= show up here?

=C2=A0 #+begin_src shell=C2=A0
=C2=A0 = =C2=A0 env=C2=A0
=C2=A0 #+end_src
=C2=A0
When I actuall= y directly call bash

=C2=A0#+begin_src shell
=C2=A0 =C2=A0 bash -= c env
=C2=A0 #+end_src

Why would the output not include ~GIM_BASH= RC~ - it should have been run, right?

What about when I call this? E= ven with explicitly selecting the rc file to run, it seems to not

= =C2=A0 #+begin_src shell=C2=A0
=C2=A0 =C2=A0 bash --rcfile ~/.bashrc -c = env
=C2=A0 #+end_src

Finally, the outstanding question about ~ob-= shell~ is if there is any way to force it to run the shell processes' r= c-script? I really would have expected it to be run in the above situations= already...

-----------

Is this perhaps on a Mac where Emacs is started from the dock?

Yup like I said, its in GUI mode so I understand why t= hose environment variables aren't within emads itself
=C2=A0<= /div>
If so, it
could be because on the mac, applications started from the dock are not
executed inside a login shell. Do you get different results if you start emacs from within a terminal?

Yup I do, bu= t thats why I move on to the test where I run `bash` directly
=C2= =A0
I am also a litt= le confused by your examples. Some of them are
attempting to source .bash_profile, others .bashrc and others .zshrc.

I created the three environment variables=C2=A0I indica= ted and placed one in each file so that I could test which profile file ran=
=C2=A0
Which shell are you using and which files do you have? I get the feeling that in your frustration, you have taken a 'shotgun' approach and a= dded
the variable everywhere.

That's exactl= y what I'm trying to avoid here. I *did* add variables everywhere, but = I added *different* variables everywhere so that I can see exactly which pr= ofile ran.
=C2=A0
For some reason, there seems to be some confusion these days about = the
difference between .profile, .bash_profile, .bashrc, .zsh_profile and
.zshrc. In particular, I frequently see incorrect/wrong advice regarding the use of 'profile' and 'rc' files. Most of it can
be blamed on wrong advice in forums like stack overflow!

It is certainly confusing but I think I got a handle on i= t mostly. If it's=C2=A0a login shell you'll run a profile, if it= 9;s not you'll run the default rc file unless one of the options were s= pecified. I think each is named specific to each shell name but oftentimes = you chain them together in practice.
=C2=A0
I would also experiment with just dumping out the output of 'env' a= nd
not run it through grep so that you can see what is being defined.

I did, that just isn't conducive to posting on= an email list ;)

iI also think it is a good dea to be explicit about the shell so that you h= ave more
control/understanding over what is being run e.g. put the line

Yeah, I suppose I could do a prologue with a hashbang, but I d= ebugged into=C2=A0 the source and it's just ~shell-file-name~ which in = my case is ~/bin/zsh~

In very broad terms (glossing over some of the subtleties, which is
likely the cause of some of your issues), the 'profile' files are o= nly
sourced for login shells and the 'rc' files are sourced whenever a = new
shell process is run. Note that these processes run in a hierarchy i.e.
shell processes have a parent process. Some things have to be set in
your 'profile' (or are better set there) while other things must be= set
in your 'rc' file. For example, setting PATH in your 'profile&#= 39; is normal
because you typically just want that set once and then 'exported' t= o all
child processes. Setting a dynamic prompt which changes depending on the directory your in (or because it has some other dynamic data, like the
time) need to be set in your 'rc' file because these are sourced be= fore
each shell process is executed. I the 'old days' it was important t= o get
these distinctions correct because sourcing the files could have a
performance impact - not as big an issue these days.
<= br>
Great history, thanks, that's mostly what I was thinking = but gives me useful background=C2=A0

These days, things seem to have become even more confusing in an attempt to make things easier. I often see people with a .profile, a
.bash_profile, a .bashrc, a .zsh_profile and a .zshrc with variables set and exported mixed in across all these files.

Don'= ;t forget that many tools from homebrew to pyenv install scripts to ansible= will start modifying these for you
=C2=A0
This is a source of
terrible confusion and unexpected results. Some 'canned' shell
configurations, like 'oh-my-zsh' also try to be extra helpful by sourcing files which are not normally sourced by that shell (to make
smoother migration from bash shells for example). Sometimes, it can be
helpful to put a line like the following into each of these files

echo "Executing <filename> at `date`" >> ~/profile.lo= g

just to see what is getting executed when (you won't want to leave this=
there - just for diagnostic and experimentation to help understand when
things are being sourced. You could even dump out the output of 'env= 9; so
that you can see how the environment is being changed as each of these
files executes.


Yup, thanks.
<= div>=C2=A0
George Mauer <gmau= er@gmail.com> writes:

> I am confused why no matter how I try to run shell commands they seem = to be missing variables exported in profiles.
>
> I have added 3 variables to various startup scripts
>
>=C2=A0 =C2=A0- ~./bash-profile~ :: ~export GIM_BASH_PROFILE=3D"yes= "~
>=C2=A0 =C2=A0- ~./bashrc~ :: ~export GIM_BASHRC=3D"yes"~
>=C2=A0 =C2=A0- ~./zshrc~ :: ~export GIM_ZSHRC=3D"yes"~
>
> I am running emacs in GUI mode so I get why none of these are availabl= e *directly* in emacs, so then I try
>
>=C2=A0 =C2=A0#+begin_src shell :results list
>=C2=A0 =C2=A0 =C2=A0env | grep GIM
>=C2=A0 =C2=A0#+end_src
>
>=C2=A0 =C2=A0#+RESULTS:
>
> Ok that's kinda surprising, but I suppose it could be running ~/bi= n/zsh~ (that's my ~shell-file-name~) directly
>
> what about
>
>=C2=A0 =C2=A0#+begin_src shell :results list
>=C2=A0 =C2=A0 =C2=A0bash -c env | grep GIM
>=C2=A0 =C2=A0#+end_src
>
>=C2=A0 =C2=A0#+RESULTS:
>
> That's pretty surprising. I would have expected running it directl= y to actually run my profile.
>
> Shockingly
>
>=C2=A0 =C2=A0#+begin_src shell :results list
>=C2=A0 =C2=A0 =C2=A0bash --rcfile ~/.bashrc -c env | grep GIM
>=C2=A0 =C2=A0#+end_src
>
>=C2=A0 =C2=A0#+RESULTS:
>
> That is *still* nothing!
>
> Sanity is restored slightly when I run
>
>=C2=A0 =C2=A0#+begin_src shell :results list
>=C2=A0 =C2=A0 =C2=A0bash --login -c env | grep GIM
>=C2=A0 =C2=A0#+end_src
>
> which *does* indeed visit ~.bash_profile~ but only slightly.
>
> What is going on, and is there a straightforward way in which I can ge= t shell block to read from a profile?


--
Tim Cross

--00000000000005368505bd8a1a92--