From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id wE83E8ixkWAuTAAAgWs5BA (envelope-from ) for ; Tue, 04 May 2021 22:42:48 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id mP3EDsixkWBxPQAAbx9fmQ (envelope-from ) for ; Tue, 04 May 2021 20:42:48 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8C70D1B0B0 for ; Tue, 4 May 2021 22:42:47 +0200 (CEST) Received: from localhost ([::1]:59194 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1le1sU-000731-BQ for larch@yhetil.org; Tue, 04 May 2021 16:42:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36878) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1le1rx-00072W-CV for emacs-orgmode@gnu.org; Tue, 04 May 2021 16:42:14 -0400 Received: from out1-smtp.messagingengine.com ([66.111.4.25]:51199) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1le1rs-0000Y8-Tx for emacs-orgmode@gnu.org; Tue, 04 May 2021 16:42:13 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 1864C5C0189 for ; Tue, 4 May 2021 16:42:07 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Tue, 04 May 2021 16:42:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=3QcxoY aQ4vQ5SI9mujMJweRdlG6DMktm6ek/1WmCx0o=; b=GVU3DIrw3zgBkOSUkl4XQn fhp/bIyB5Jora52jV2MhCTKyA/0c7GaOudU2TQ6yPyIk9W7VjHjGZqAo0n+AH1fm zEvjSrgoNk0OLPpYIuGBHNhE9cjVbPZybZsdoHMgDqYAdwVu+SsFcr4SmvyoeopX G5DGBYDvXZuTlUCY8OviYvExRaPlutRxKJ6Um+CbaBdjXQOr/Sya0USd5OQcJZr3 LhNsl+4hZGqxJsXvhtVI1IYE+dDUcIk9is0ASh66Bqa4iH3UpfzpT6XkLZfidUe4 LVaMEFo7LzcfYLSQHK3Qe+jdKaTvopWlvmGmO6SugeF6FaIQ0k7wmw3gVYf+DqJg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvdefiedgudehfecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepuffvfhfhkffffgggjggtsehmtd erredtfeejnecuhfhrohhmpefpihgtkhcuufgrvhgrghgvuceonhhitghksehnihgtkhhs rghvrghgvgdrtggrqeenucggtffrrghtthgvrhhnpeetgeekgeeiueetudfhueejgfdtie ehffetjeeukeevvedvgeduudehudetgfeitdenucfkphepudejgedrleefrdduieehrddu feenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehnih gtkhesnhhitghkshgrvhgrghgvrdgtrg X-ME-Proxy: Received: from [192.168.0.140] (bras-base-aylmpq0104w-grc-46-174-93-165-13.dsl.bell.ca [174.93.165.13]) by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 4 May 2021 16:42:06 -0400 (EDT) Subject: [PATCH] Fix truncated output in ob-sqlite.el (Was: Bug: sqlite output truncated?) To: emacs-orgmode@gnu.org References: From: Nick Savage Message-ID: <3cf8beb3-bd45-ddc8-780a-79b32fc413eb@nicksavage.ca> Date: Tue, 4 May 2021 16:42:06 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------8237B5FC005068B583041558" Content-Language: en-US Received-SPF: none client-ip=66.111.4.25; envelope-from=nick@nicksavage.ca; helo=out1-smtp.messagingengine.com X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1620160967; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=3QcxoYaQ4vQ5SI9mujMJweRdlG6DMktm6ek/1WmCx0o=; b=Z2mI/4pSQOvIx4TRbkO8I5ycmfty1BWAwg4+M/+/kJrei5PhLYOhfMrEH/Ictaxk8TM1Tu mz2IZzUmTkizshOTdqeI/oj6P+b6mdaKs9NoGSpbvFfjsTpVm5GKiDaRyzhL+3Ragaa54W zbV6I45iZUPQu3gInc3PWyIr1qX1X68DdqV6+W7QfaN+4E+DM5aAXkuDmLWRkxeb60JkRs fPz6sx2vNSCPUe75JwaLiWek/kb25s+CeQU3Pq9dIFvwxKB/zpCLzkjh3wWgxVhEvcb4rD 2lM5mRwWiXQWa+927aPfv1XHPSHuwdmo+yYmEWq+Y1T7umDO2oA+yiqSryB3Rg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1620160967; a=rsa-sha256; cv=none; b=fklE2OW3BnoGxD6FL3kJU8mqWKHVpn3lxcVSAFu8wK0TyxlpbThm7ufCYbcqTmFHnJQFEH BL3VwfnrRDUZ9CTyOS+iVg08hkyyWH2GcEAdHidGXcZTiGSHF9i0tYrNCKPxDYZKNUAVDK Jusp4jsFuDfb6TZaU0XLiGz2p0/kx6/C1gJ24KPNTa9nsUYX0JzXRlB+XlTmTklhHm46/b h1blqMmlkhCEOoJXKxlIwzv5A2HVbXVzKc6lo3EnTk9w7tJLYEN9oYtReau61nR63jbX+T Z2fr5fPYf/5w2RQH/XCWwmPvEsPobetellPeJ5Gnwb7FPiaafFm860jFJg2Drg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=GVU3DIrw; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Spam-Score: -1.16 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=GVU3DIrw; dmarc=none; spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Migadu-Queue-Id: 8C70D1B0B0 X-Spam-Score: -1.16 X-Migadu-Scanner: scn0.migadu.com X-TUID: hFHLR+QAzpdY This is a multi-part message in MIME format. --------------8237B5FC005068B583041558 Content-Type: multipart/alternative; boundary="------------044F05B6578748FF59D8CD9F" --------------044F05B6578748FF59D8CD9F Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit I've attached a patch to address this. The issue is that ob-sqlite.el uses org-babel-string-read, which purposefully removes double-quotes. I think this is unintended behaviour, and it only seems to be used with ob-sqlite.el. I added a minor function to bypass the part of org-babel-string-read that was stripping out the double-quotes and use the rest of the function. I tried ob-sql with the mysql engine and didn't experience the same bug, so I'm pretty confident that this is an isolated issue. On 5/3/21 6:22 PM, learn orchids wrote: > I am using Org mode version 9.4.5 (9.4.5-73-g4c7696-elpaplus and I have the following code snippet. Values in the 'sql' column of the second row is truncated. Am I missing something? > > #+begin_src sqlite :db /tmp/rip.db  :colnames yes >   drop table if exists testtable; >   create table testtable(id int, sql varchar); >   insert into testtable values (1, "Select id from foo"); >   insert into testtable values (2, 'Select "id" from foo'); >   select* from testtable; > #+end_src > > #+RESULTS: testsql > | id | sql                | > |----+--------------------| > |  1 | Select id from foo | > |  2 | id                 | --------------044F05B6578748FF59D8CD9F Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

I've attached a patch to address this.

The issue is that ob-sqlite.el uses org-babel-string-read, which purposefully removes double-quotes. I think this is unintended behaviour, and it only seems to be used with ob-sqlite.el. I added a minor function to bypass the part of org-babel-string-read that was stripping out the double-quotes and use the rest of the function.

I tried ob-sql with the mysql engine and didn't experience the same bug, so I'm pretty confident that this is an isolated issue.


On 5/3/21 6:22 PM, learn orchids wrote:
I am using Org mode version 9.4.5 (9.4.5-73-g4c7696-elpaplus and I have the following code snippet. Values in the 'sql' column of the second row is truncated. Am I missing something?

#+begin_src sqlite :db /tmp/rip.db  :colnames yes
  drop table if exists testtable;
  create table testtable(id int, sql varchar);
  insert into testtable values (1, "Select id from foo");
  insert into testtable values (2, 'Select "id" from foo');
  select* from testtable;
#+end_src

#+RESULTS: testsql
| id | sql                |
|----+--------------------|
|  1 | Select id from foo |
|  2 | id                 |
--------------044F05B6578748FF59D8CD9F-- --------------8237B5FC005068B583041558 Content-Type: text/x-patch; charset=UTF-8; name="0001-lisp-ob-sqlite.el-Prevent-output-from-being-incorrec.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename*0="0001-lisp-ob-sqlite.el-Prevent-output-from-being-incorrec.pa"; filename*1="tch" >From 3dd868bcd798e6763ec67ba0d2661834e09bb6e9 Mon Sep 17 00:00:00 2001 From: Nicholas Savage Date: Tue, 4 May 2021 11:02:20 -0400 Subject: [PATCH] lisp/ob-sqlite.el: Prevent output from being incorrectly parsed if it contains double-quotes. * lisp/ob-sqlite.el (org-babel-sqlite--read-cell): New function. lisp/ob-sqlite.el (org-babel-sqlite-table-or-scalar): Use `org-babel-sqlite--read-cell' instead of `org-babel-string-read' to prevent data from being ignored if it contains double-quotes. Reported-by: learnorchids@gmail.com Link: https://orgmode.org/list/CAK5xwica4i8h09ObzCVPx2pe-T6B0_ju_MXiduRiY7GH+PkE_A@mail.gmail.com/ --- lisp/ob-sqlite.el | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lisp/ob-sqlite.el b/lisp/ob-sqlite.el index c0c6f3c97..d227ba69d 100644 --- a/lisp/ob-sqlite.el +++ b/lisp/ob-sqlite.el @@ -125,7 +125,7 @@ This function is called by `org-babel-execute-src-block'." (mapcar (lambda (row) (if (eq 'hline row) 'hline - (mapcar #'org-babel-string-read row))) + (mapcar #'org-babel-sqlite--read-cell row))) result))) (defun org-babel-sqlite-offset-colnames (table headers-p) @@ -139,6 +139,10 @@ This function is called by `org-babel-execute-src-block'." Prepare SESSION according to the header arguments specified in PARAMS." (error "SQLite sessions not yet implemented")) +(defun org-babel-sqlite--read-cell (cell) + "Process CELL to remove unnecessary characters." + (org-babel-read cell t)) + (provide 'ob-sqlite) ;;; ob-sqlite.el ends here -- 2.20.1 --------------8237B5FC005068B583041558--