From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id EGEUAshCEmPE1wAAbAwnHQ (envelope-from ) for ; Fri, 02 Sep 2022 19:52:08 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id ALIAAshCEmMdcgAAauVa8A (envelope-from ) for ; Fri, 02 Sep 2022 19:52:08 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 93D211475B for ; Fri, 2 Sep 2022 19:52:07 +0200 (CEST) Received: from localhost ([::1]:47462 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oUApq-00085B-28 for larch@yhetil.org; Fri, 02 Sep 2022 13:52:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53082) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oUAYs-0005Lk-KV for emacs-orgmode@gnu.org; Fri, 02 Sep 2022 13:34:34 -0400 Received: from smtp4-g21.free.fr ([212.27.42.4]:6592) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oUAYq-0007AV-Hx for emacs-orgmode@gnu.org; Fri, 02 Sep 2022 13:34:34 -0400 Received: from [IPv6:2a01:e35:39f3:4610:8057:6f64:f7cf:4745] (unknown [IPv6:2a01:e35:39f3:4610:8057:6f64:f7cf:4745]) (Authenticated sender: tbanelwebmin@free.fr) by smtp4-g21.free.fr (Postfix) with ESMTPSA id 1795D19F738 for ; Fri, 2 Sep 2022 19:34:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=free.fr; s=smtp-20201208; t=1662140065; bh=5zBRvyt1fWJ15kDRWlUWekpUpqgDZ7uQnBBf+Y/PmHc=; h=Subject:To:References:From:Date:In-Reply-To:From; b=tDoekJLmTdOLxHjkUfbWN3B6eRjk3BAIWXpXKKB9hN3GoUu6N2KAAq/pZRnJ2DbTU wb1KVrtPlqqSEuQSkF/CIK6N9Vfsse1Vbeu7QK7yAXScA7umxKyrMDuuzAjSFR7UnT ZB3UBepo49IpaZ/kMVOB6lqqb6zO3eSAltWn/G7vimqkIFG9QJZs8n832TcWiImlDM M5Jj4bIRKHicsu2DcaaYZUavVBI08t/rPchMus6+ehFl4uBpefFoyZbCysJclI379w ZtHPD5EZHItcw0x9WoTbw4GqTSwn1BalmZf49LTXg0BJ7wVIZDP3ddi+MLVTjdsjd7 78m9xrWY3DGTw== Subject: Re: Babel C-mode corrupts double-quoted strings in output To: emacs-orgmode@gnu.org References: <34af937c-f80e-9664-b29e-a767f62ba89e@free.fr> From: tbanelwebmin Message-ID: <89b0e8a5-907c-4028-68d6-ec48c0dc98fd@free.fr> Date: Fri, 2 Sep 2022 19:34:24 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <34af937c-f80e-9664-b29e-a767f62ba89e@free.fr> Content-Type: text/html; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=212.27.42.4; envelope-from=tbanelwebmin@free.fr; helo=smtp4-g21.free.fr X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.1, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1662141127; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=Tti71FqS3jUTbc7aIvsxo0ON7l6kyHESR6jch/8Is3g=; b=BOlgdzayO2Ait+/EsKOYlhhETUMEygMaLRn6YiYtiT0AZqU1XKE9KtdfxL3OtClwDEY9C0 CZejoBzq+77015w4kjjqcOlGK0S3oqBgFRQd3e5ZFZ4uiLIPQa6fIUdg5SvWOWTYnjk6L7 0/5960d7RO8aCVuMDCdRilGUkGFZZ2wLZJPO4nwnDqj0mFX3/X2gyvijg3cvb7UI6oEP71 pPj8Gwp8E9E1pduHiuN/UScOfXAfzNt97uLgJeLmoN+sBIr0gVJg0DUmkzWvY0MxfXKbfP 9m1kk7OybNHjYEokGp3MUWGQBmZJTGNpheUKOntq8a0hqVZq4dMaeoeqwdLdGw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1662141127; a=rsa-sha256; cv=none; b=ujplKPgtAvNMMFFXZFQFLvs+ZsgdCYttFeGcLkOKHejCBAs21WIo8kk5RRtYrZYblHFHy3 qvAEMUhlkKLEGSUJ1CjuxvlTzl/He5YUEUqK8f8KA/q4NNsXKadAQ1RWhChL5/tjLROUUE cUZa2nIYj/IvgmqAFCeGBAjTxtNpcSJWzWwcgPM7/QNQedN+Z/8ff5wpq1L9IJPwtbbmfk nXlIfl01ZwI1D1I866+KlcoxdR7Grf/nTaWhA8eyWqEzMAzDi7iMaUAqx1GL5HGMVObzEq kuNxwOVgcSi198EZqVYlRoOTOz+KU56/hgP53JNjecTuq8ak/Ey/+ek6wO6ncQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=free.fr header.s=smtp-20201208 header.b=tDoekJLm; dmarc=pass (policy=none) header.from=free.fr; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.27 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=free.fr header.s=smtp-20201208 header.b=tDoekJLm; dmarc=pass (policy=none) header.from=free.fr; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 93D211475B X-Spam-Score: -3.27 X-Migadu-Scanner: scn0.migadu.com X-TUID: AMeb+tN5w6+a This fix in ob-C.el passes all unit tests, as well as Martin's example, and some other examples.

If someone get a regression, please tell me!
In a few days, I will commit.

The fix: in ob-C.el line 185, change:
  (org-babel-read results t)
to:
  results

Have fun
Le 02/09/2022 à 13:02, tbanelwebmin a écrit :
This looks like a bug in ob-C.el

Around line 196 we should replace
  (org-babel-read results t)
with
  results

In this way, ob-C.el will look more like ob-shell.el

Let me see what are the consequences with such a fix.

Thanks Martin for investigating deep in the sources!

Regards
Le 31/08/2022 à 18:35, Martin Jerabek a écrit :
Hi!

I recently started to use Org Babel for C++ programs. One of the programs outputs several lines with double-quoted strings, similar to this:

#+NAME: doublequotes_cpp
#+begin_src cpp :includes <iostream> :results output verbatim raw
std::cout << "\"line 1\"\n";
std::cout << "\"line 2\"\n";
std::cout << "\"line 3\"\n";
#+end_src

#+RESULTS: doublequotes_cpp
line 1

As you can see, only the first line is copied to the RESULTS block, and it is stripped of the double quotes.

I tracked down the problem to org-babel-read (in ob-core.el). org-babel-C-execute (in ob-C.el) calls this function with the output of the C++ program. The problem is the following line:

((eq (string-to-char cell) ?\") (read cell))

i.e. if the output of the program starts with a double quote, it is passed to read which reads only the first string and also removes the double quotes, resulting in the observed output.

The original version of this piece of code was added with the following commit:

commit 60a8ba556d682849eafb0f84e689967cd2965549
Author: Eric Schulte <schulte.eric@gmail.com>
Date:   Wed Mar 2 07:55:39 2011 -0700

    ob: read string variable values wrapped in double quotes, removing the quotes
    
    * lisp/ob.el (org-babel-read): Read string variable values wrapped in
      double quotes, removing the quotes.

AFAICT this modification was done in response to the email thread "[Orgmode] org-babel-read should have option NOT to interpret as elisp" started on 2011-02-27, more specifically the email on "Wed, 02 Mar 2011 07:56:45 -0700" from Eric Schulte. This was obviously done for parsing variables in the header line, not for the program output, but the Babel C mode uses org-babel-read also for the output.

I assumed that ":results output verbatim raw" would prevent any postprocessing of the output but this is not the case for C mode.

I am not sure how to fix this without breaking backward compatibility. I assume it should be fixed directly in org-babel-C-execute, not in a central function like org-babel-read to minimize the impact. Surprisingly (for me) the equivalent shell script works as expected:

#+NAME: doublequotes
#+begin_src shell :results output verbatim raw
echo '"line 1"'
echo '"line 2"'
echo '"line 3"'
#+end_src

#+RESULTS: doublequotes
"line 1"
"line 2"
"line 3"

because org-babel-execute:shell does not process the output with org-babel-read. I do not know if languages other than the C family (C, C++, D) are affected.

At the very least, the documentation of org-babel-read should be expanded to document the fact that if the CELL parameter starts with a double quote, it is processed by the read function.

Best regards
Martin Jerabek