From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id EDOpIm7o3WTBEwEASxT56A (envelope-from ) for ; Thu, 17 Aug 2023 11:29:18 +0200 Received: from aspmx1.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id yIPcIm7o3WSZxwAA9RJhRA (envelope-from ) for ; Thu, 17 Aug 2023 11:29:18 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1FC804A55A for ; Thu, 17 Aug 2023 11:29:18 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=QTw6QnMf; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=key1; d=yhetil.org; t=1692264558; a=rsa-sha256; cv=none; b=Qh3Pz5sVNIs+fZH325PSx0mm3GxI4q9fGjzASTYgGIWfACWJjdS837OB8W8W5gL/3C7AbL iCJIVh3wyLh3riW6M4qj/9RtYLbAIASUbv91/10BBnsKwJVIQUM0QeXlzCTjm/5mFGw1ma cIv1nfCb+O4XLK/5QwjkmevFrFg6+zTjK+b0YnggnELvbNGHvrB8+sXU4B7BR+yoIlJa95 pskEy0PxX+IS2wzA0I/DwCUYx5jL6ef7IIDNf6mbZhFvC5BHCClf74WoZqaW3FL2slGVHc Y67y9/25hNsUexgajme4juCBxY08z1SuPLCXeCw3BMW234vc4MtB9jB/8wUnJg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=QTw6QnMf; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1692264558; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=NYtNLfxh9N5EmAdnWUpCuNPlVPKckVtz/r5CwPjpjJ0=; b=pYffabz89o8FwLN/QLVxcUfc85P5rzUyPDsTg/6Y3fcCQHIwEhndHZblKRnbeB0Vuiwiex Nk9416bGwBa/mdSv/oiR/M+7Wr9ssnAou8ArWmoc0MMnMUStfqKxSIrbghQbQPKXEEJe6y p7H7gUfufqAgky07xmgMQpOUqqhBSd46BGpNG4s9ASXUlli2EI3FTGDJNxGM3Ovv5yh5tk mWpoeqXL4t3VnvDd5FWV2F4r5YSkMGxkGQ9vJu6t2FesyjJQ6pzNk/rVeOyfS77tf2RCDR cfmPAgiTKnicb6upWQrQn0Ulyx2tntY7SRPtt14/PyEImyT3uCr/DuTrKi5SKg== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qWVfv-0002j7-Os; Thu, 17 Aug 2023 01:36:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qWVfs-0002ic-EY for emacs-orgmode@gnu.org; Thu, 17 Aug 2023 01:36:00 -0400 Received: from mail-vs1-xe32.google.com ([2607:f8b0:4864:20::e32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qWVfp-0006ie-Uj for emacs-orgmode@gnu.org; Thu, 17 Aug 2023 01:36:00 -0400 Received: by mail-vs1-xe32.google.com with SMTP id ada2fe7eead31-447a3d97d77so2908272137.1 for ; Wed, 16 Aug 2023 22:35:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692250556; x=1692855356; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=NYtNLfxh9N5EmAdnWUpCuNPlVPKckVtz/r5CwPjpjJ0=; b=QTw6QnMfdXAn2bGE6CfBzTVqRG8TBt/AcLKTxfXbEgWroAEUdXy9S/vFkCfm8XvJM5 OeY/yq3faeLJ81QZen8GuAS5K4piKKqTmbT/AhP3ZxE4lyUTVoVN7+quhu14vY8ROtJ5 vIHHXJE08w/k8Zu8H9HG62CeSa3T2RyvrlCHqC8x5HtMM9gWqJUVDXR/Li4Ynq/EjhLC WjnuwecOBcBA1N7RDUvIIsH9JgDTM0AbIlMxxaTsI9zHPwetNQL3Z7HEXA9HRLr2M2U6 MN8r56yHBxURquFhZ+CO+DjvExhwAMxNed9Tea+ofZr8aRRk9oPseo7aNtBt38i+Cl2J hvUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692250556; x=1692855356; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=NYtNLfxh9N5EmAdnWUpCuNPlVPKckVtz/r5CwPjpjJ0=; b=BpcQ+4ie9gidTBYx7SMhzBY+c6iNbcV06fxtn/eX4nsmkwJBXyzhITIG9JXmPzgFtr ymPqmBp99Jv2i8WUbklNCOevN7FikczTkz79frJNDJ3i174sIIK3RnRFyb/OautOdeXc MSt7G39KncY+Dekj+Vi3qSittioffYsq2rTKgJME8zN/se88PIzIeFM+BK52Mpgod3d/ Wo8owsfjsA2n5A0CjGys6xe+cqdOOsJn38JfL+F281xTaMFy97PKoLdNDqUoalutAQja BScKPCerfWKufTRbSdGROFC0CprMNFI97zLYJ95sPpNimuS7n11RtAgkPPXmWSA9MNLy R6ww== X-Gm-Message-State: AOJu0YxNiosJkwDdYXQvGZnAS9Oo+ys0+OnNHDgvDpzwQoi8KwaZgp9x 0ayGglHKPN+XXfp/DFw+Tg3MqqpFiUHrDsGBmdI= X-Google-Smtp-Source: AGHT+IELGnrQoqjdnH0cRcHqJ2Y2BNRsoYCB9e4EqDNr+Kwg0XcFM1NTCePY1FAOqMS01fXTUaZkWV591OUJU4H+Mmo= X-Received: by 2002:a67:f7d7:0:b0:447:68a0:a121 with SMTP id a23-20020a67f7d7000000b0044768a0a121mr3523309vsp.2.1692250556635; Wed, 16 Aug 2023 22:35:56 -0700 (PDT) MIME-Version: 1.0 References: <87a5ur6f7w.fsf@gmail.com> In-Reply-To: <87a5ur6f7w.fsf@gmail.com> From: Liu Hui Date: Thu, 17 Aug 2023 13:35:45 +0800 Message-ID: Subject: Re: [PATCH] ob-python results handling for dicts, dataframes, arrays, and plots To: Jack Kamm Cc: emacs-orgmode@gnu.org, Ihor Radchenko Content-Type: text/plain; charset="UTF-8" Received-SPF: pass client-ip=2607:f8b0:4864:20::e32; envelope-from=liuhui1610@gmail.com; helo=mail-vs1-xe32.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Queue-Id: 1FC804A55A X-Spam-Score: -9.45 X-Migadu-Scanner: mx2.migadu.com X-Migadu-Spam-Score: -9.45 X-TUID: p7dmMMljBF7P Hi, Thank you for the patch! > Next, for numpy arrays and pandas dataframes/series: these are > converted to tables, for example: > > #+begin_src python > import pandas as pd > import numpy as np > > return pd.DataFrame(np.array([[1,2,3],[4,5,6]]), > columns=['a','b','c']) > #+end_src > > #+RESULTS: > | | a | b | c | > |---+---+---+---| > | 0 | 1 | 2 | 3 | > | 1 | 4 | 5 | 6 | > > To avoid conversion, you can specify "raw", "verbatim", "scalar", or > "output" in the ":results" header argument. Do we need to limit the table/list size by default, or handle them only with relevant result type (e.g. `table/list')? Dataframe/array are often large. The following results are truncated by default previously, which can be tweaked via np.set_printoptions and pd.set_option. #+begin_src python import numpy as np return np.random.randint(10, size=(30,40)) #+end_src #+begin_src python import numpy as np return np.random.rand(20,3,4,5) #+end_src #+begin_src python import pandas as pd import numpy as np d = {'col1': np.random.rand(100), 'col2': np.random.rand(100)} return pd.DataFrame(d) #+end_src > +def __org_babel_python_format_value(result, result_file, result_params): > + with open(result_file, 'w') as f: > + if 'graphics' in result_params: > + result.savefig(result_file) > + elif 'pp' in result_params: > + import pprint > + f.write(pprint.pformat(result)) > + else: > + if not set(result_params).intersection(\ > +['scalar', 'verbatim', 'raw']): > + try: > + import pandas > + except ImportError: > + pass > + else: > + if isinstance(result, pandas.DataFrame): > + result = [[''] + list(result.columns), None] + \ Here we can use '{}'.format(df.index.name) to show the name of index > (defun org-babel-python-format-session-value > (src-file result-file result-params) > "Return Python code to evaluate SRC-FILE and write result to RESULT-FILE." > - (format "\ > + (concat org-babel-python--def-format-value > + (format " Maybe `org-babel-python--def-format-value' can be evaluated only once in the session mode? It would shorten the string sent to the python shell, where temp files are used for long strings.