* Why Emacs shows ^M at the end of some lines in sh code blocks that use curl?
@ 2024-10-17 15:05 Rodrigo Morales
2024-10-17 18:14 ` Bruno Barbier
0 siblings, 1 reply; 3+ messages in thread
From: Rodrigo Morales @ 2024-10-17 15:05 UTC (permalink / raw)
To: emacs-orgmode
[-- Attachment #1: Type: text/plain, Size: 3871 bytes --]
I saved the contents of the code block below to the file =/tmp/a.org=.
#+HEADER: :tangle /tmp/a.org
#+BEGIN_SRC org
,#+HEADER: :tangle no
,#+HEADER: :results verbatim
,#+BEGIN_SRC sh
curl --silent --request GET 'http://localhost:4891/posts'
,#+END_SRC
,#+HEADER: :tangle no
,#+HEADER: :results verbatim
,#+BEGIN_SRC sh
curl --include --silent --request GET 'http://localhost:4891/posts'
,#+END_SRC
,#+HEADER: :tangle no
,#+HEADER: :results verbatim
,#+BEGIN_SRC sh
seq 1 10
,#+END_SRC
#+END_SRC
I started Emacs using the command shown below.
#+BEGIN_SRC sh
emacs -Q --eval "(require 'ob-shell)" /tmp/a.org
#+END_SRC
I evaluated the three =sh= code blocks in Emacs. Here's an screenshot
that shows the content of the buffer =/tmp/a.org= after the three code
blocks have been evaluated: https://archive.is/RnAD6 (edited for
clarity: https://archive.is/IJzuc ) Note that the character "^M" is
shown at the end of each line that contain response headers shown by
curl (in curl, response headers are shown using the flag
--include). Those who can't open the screenshot can refer to the code
bock below, I have manually replaced the character '^M' with the string
"^M" just so that you see what I am seeing in my computer. The character
^M is only shown for lines showing the response headers, not for lines
showing the response body.
#+BEGIN_SRC org
,#+HEADER: :tangle no
,#+HEADER: :results verbatim
,#+BEGIN_SRC sh
curl --include --silent --request GET 'http://localhost:4891/posts'
,#+END_SRC
,#+RESULTS:
,#+begin_example
HTTP/1.1 200 OK^M
Server: Werkzeug/3.0.4 Python/3.10.12^M
Date: Thu, 17 Oct 2024 14:57:28 GMT^M
Content-Type: application/json^M
Content-Length: 123^M
Connection: close^M
^M
[
{
"title": "Title of post 1"
},
{
"title": "Title of post 2"
},
{
"title": "Title of post 3"
}
]
,#+end_example
#+END_SRC
I then pressed =M-x shell RET= and executed the same commands from the
sh code blocks. Here's an screenshot that shows the buffer =*shell*=
after the three commands have been executed: https://archive.is/XDyIg
(edited for clarity: https://archive.is/A8dkZ ) Note that the character
"^M" is not shown at the end of each line, as happened when the same
command was evaluated in an Org Mode code block.
I wish those ^M were not shown in the sh code blocks in the Org Mode
buffer. Does anyone know how to accomplish this behavior?
You can find the file =/tmp/a.org= (after the three code blocks have
been evaluated) attached in this message.
** Additional information
Below you can find a simple Python application that you can use as an
API to reproduce this behavior. The first code block below shows the
command that you can run to initialize the virtual environment. The
second code block shows the Flask application. The third code block
below shows the command to run the Flask application.
#+HEADER: :results silent
#+HEADER: :tangle no
#+BEGIN_SRC bash
rm -rf /tmp/python \
&& mkdir /tmp/python \
&& cd /tmp/python \
&& virtualenv venv \
&& source venv/bin/activate \
&& pip install flask
#+END_SRC
#+HEADER: :tangle /tmp/python/blog.py
#+BEGIN_SRC python
import flask
app = flask.Flask(__name__)
@app.route('/posts', methods = ['GET'])
def get_posts():
return [
{'title': 'Title of post 1'},
{'title': 'Title of post 2'},
{'title': 'Title of post 3'}
], 200
@app.route('/categories', methods = ['GET'])
def get_categories():
return [
{'name': 'Category 1'},
{'name': 'Category 2'},
{'name': 'Category 3'}
], 200
#+END_SRC
#+HEADER: :tangle no
#+BEGIN_SRC bash
cd /tmp/python && source venv/bin/activate && python -m flask --app blog --debug run --port 4891
#+END_SRC
** System information
Emacs version: GNU Emacs 29.4 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.33, cairo version 1.16.0) of 2024-10-12
OS: Ubuntu 22.04.5 LTS
Org mode version: 9.6.15
[-- Attachment #2: a.org --]
[-- Type: text/plain, Size: 912 bytes --]
#+HEADER: :tangle no
#+HEADER: :results verbatim
#+BEGIN_SRC sh
curl --silent --request GET 'http://localhost:4891/posts'
#+END_SRC
#+RESULTS:
#+begin_example
[
{
"title": "Title of post 1"
},
{
"title": "Title of post 2"
},
{
"title": "Title of post 3"
}
]
#+end_example
#+HEADER: :tangle no
#+HEADER: :results verbatim
#+BEGIN_SRC sh
curl --include --silent --request GET 'http://localhost:4891/posts'
#+END_SRC
#+RESULTS:
#+begin_example
HTTP/1.1 200 OK
Server: Werkzeug/3.0.4 Python/3.10.12
Date: Thu, 17 Oct 2024 15:02:54 GMT
Content-Type: application/json
Content-Length: 123
Connection: close
[
{
"title": "Title of post 1"
},
{
"title": "Title of post 2"
},
{
"title": "Title of post 3"
}
]
#+end_example
#+HEADER: :tangle no
#+HEADER: :results verbatim
#+BEGIN_SRC sh
seq 1 10
#+END_SRC
#+RESULTS:
#+begin_example
1
2
3
4
5
6
7
8
9
10
#+end_example
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Why Emacs shows ^M at the end of some lines in sh code blocks that use curl?
2024-10-17 15:05 Why Emacs shows ^M at the end of some lines in sh code blocks that use curl? Rodrigo Morales
@ 2024-10-17 18:14 ` Bruno Barbier
2024-10-18 23:44 ` Rodrigo Morales
0 siblings, 1 reply; 3+ messages in thread
From: Bruno Barbier @ 2024-10-17 18:14 UTC (permalink / raw)
To: Rodrigo Morales, emacs-orgmode
Hi Rodrigo,
Rodrigo Morales <rodrigo-mailing-lists@morales.pe> writes:
> I saved the contents of the code block below to the file =/tmp/a.org=.
>
>
> I wish those ^M were not shown in the sh code blocks in the Org Mode
> buffer. Does anyone know how to accomplish this behavior?
>
Your result is mixing different "end-of-line" conventions. If you
redirect the result into a file and open it with Emacs, you will get the
same result: it's not related to Org (HTTP headers use '\r\n', your JSON
'\n').
You could include a translation step. For example, using 'tr':
#+begin_src shell :results raw
curl --include --silent https://filesampleshub.com/download/code/json/sample1.json | tr -d '\r'
#+end_src
If you are fetching arbitrary data, you do not want to do that though.
You may ask curl to dump the headers in a file, and translate only
that file (using --dump-header).
Note that getting any JSON file out there should give the same result;
i.e. no need to install Flask :)
HTH,
Bruno
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Why Emacs shows ^M at the end of some lines in sh code blocks that use curl?
2024-10-17 18:14 ` Bruno Barbier
@ 2024-10-18 23:44 ` Rodrigo Morales
0 siblings, 0 replies; 3+ messages in thread
From: Rodrigo Morales @ 2024-10-18 23:44 UTC (permalink / raw)
To: Bruno Barbier, emacs-orgmode
Bruno Barbier <brubar.cs@gmail.com> writes:
> it's not related to Org (HTTP headers use '\r\n', your JSON
Thanks for pointing me in the right direction. I searched related
information on the Internet and I found out there is an specific section
in Request for Comments (RFC2616) that explicitly states that '\r\n'
should be used as the line break for HTTP headers. I learned this from
https://stackoverflow.com/a/5757349
> You could include a translation step. For example, using 'tr':
>
> #+begin_src shell :results raw
> curl --include --silent https://filesampleshub.com/download/code/json/sample1.json | tr -d '\r'
> #+end_src
>
> If you are fetching arbitrary data, you do not want to do that though.
> You may ask curl to dump the headers in a file, and translate only
> that file (using --dump-header).
I agree with you on not using tr -d '\r' for both the response body and
the response headers. Starting from now, when I feel the need to show
the response headers in my notes, I'll dump the headers to a file and
then show pass that file to 'tr' as shown below:
#+begin_src sh
curl --dump-header /tmp/curl-header.txt --silent --request GET 'https://filesampleshub.com/download/code/json/sample1.json'
echo '----'
cat /tmp/curl-header.txt | tr -d '\r'
#+end_src
#+RESULTS:
#+begin_example
{
"fear": "quiet",
"chest": -1477429467,
"how": false,
"graph": false,
"camp": 929234312,
"plural": "settle"
}----
HTTP/2 200
accept-ranges: bytes
access-control-allow-origin: *
age: 3641498
cache-control: public, max-age=0, must-revalidate
content-disposition: inline; filename="sample1.json"
content-type: application/json; charset=utf-8
date: Fri, 18 Oct 2024 22:43:39 GMT
etag: "c4b4a9669841a9ab437f5dce6c9c9230"
last-modified: Fri, 06 Sep 2024 19:12:01 GMT
server: Vercel
strict-transport-security: max-age=63072000
x-matched-path: /download/code/json/sample1.json
x-vercel-cache: HIT
x-vercel-id: gru1::9cdc6-1729291419743-bcee78204a79
content-length: 136
#+end_example
Sometimes when I want to focus on the behavior of an API and don't write
such long command due to that technical detail, I could simply use
restclient. See example below (in my system, the header lines shown by
restclient don't show the ^M character)
#+begin_src restclient
GET https://filesampleshub.com/download/code/json/sample1.json
#+end_src
#+RESULTS:
#+BEGIN_SRC js
{
"fear": "quiet",
"chest": -1477429467,
"how": false,
"graph": false,
"camp": 929234312,
"plural": "settle"
}
// GET https://filesampleshub.com/download/code/json/sample1.json
// HTTP/1.1 200 OK
// Accept-Ranges: bytes
// Access-Control-Allow-Origin: *
// Age: 3641582
// Cache-Control: public, max-age=0, must-revalidate
// Content-Disposition: inline; filename="sample1.json"
// Content-Length: 136
// Content-Type: application/json; charset=utf-8
// Date: Fri, 18 Oct 2024 22:45:03 GMT
// Etag: "c4b4a9669841a9ab437f5dce6c9c9230"
// Last-Modified: Fri, 06 Sep 2024 19:12:01 GMT
// Server: Vercel
// Strict-Transport-Security: max-age=63072000
// X-Matched-Path: /download/code/json/sample1.json
// X-Vercel-Cache: HIT
// X-Vercel-Id: gru1::4ktmb-1729291503557-396fe6c0ce31
// Request duration: 0.258368s
#+END_SRC
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2024-10-18 23:46 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-10-17 15:05 Why Emacs shows ^M at the end of some lines in sh code blocks that use curl? Rodrigo Morales
2024-10-17 18:14 ` Bruno Barbier
2024-10-18 23:44 ` Rodrigo Morales
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).