Re: bug#47885: [PATCH] org-table-import: Make it more smarter for interactive use

[Utkarsh Singh <utkarsh190601@gmail.com>]

On 2021-06-02, 23:44 +0700, Maxim Nikulin <manikulin@gmail.com> wrote:

> On 02/06/2021 22:08, Utkarsh Singh wrote:
>> ;;;###autoload
>>   (defun org-table-import (file separator)
>> @@ -955,12 +971,13 @@ lines.  It can have the following values:
>>   - integer When a number, use that many spaces, or a TAB, as field separator.
>>   - regexp  When a regular expression, use it to match the separator."
>>     (interactive (list (read-file-name "Import file: ")
>> -                     (prefix-numeric-value current-prefix-arg)))
>> +                     current-prefix-arg))
>
> It seems, prefix argument works now. Let me remind that file name 
> completion was working better before your change.
>
> I have noticed a couple of error messages. Unsure what is going wrong, I 
> hope, command to launch emacs is correct (-Q -L ~/src/org-mode/lisp 
> test.org).
>
> At startup:
> Eager macro-expansion failure: (error "rx ‘not’ syntax error: (or 10 44)")
>
> In response to M-x org-table-import:
> rx-check-not: rx ‘not’ syntax error: (or 10 44)
>

Is `rx' library loading correctly? If no then you can try:

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 20144c91b..9278f91bb 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -38,6 +38,7 @@
 (require 'org-macs)
 (require 'org-compat)
 (require 'org-keys)
+(require 'rx)

Can you also share test.org so that I can test locally?

>> Currently I am trying to refactor CSV parsing by applying techniques
>> used in pcsv library (MELPA package) which I think you will also enjoy
>> to play with!
>
> I do not know opinion of Org maintainers. Personally I believe that org 
> can take advantage of Emacs core features or of other packages if they 
> are available and fallback to minimal implementation otherwise. Unsure 
> whether borrowing code from pcsv can cause license issues. Current CSV 
> parser is not perfect but it works reasonably well.

Yes, I think you are right about how Org should make use of existing
feature rather than including everyting with itself but for now I am
considering this as an opportunity to learn how Org and Emacs work and
will leave it upto the maintainers if they find anything useful to be
including into Org itself.

Now on the topic of CSV parser, I have succesfully implemented a CSV
parser in less than 65 LOC which also preserves newline character but I
am facing a dilemma on how should I represent it as a Org table.  For
ex:

Row with newline as CSV data:

6,"Cell with new
Line",6.28

Row with newline as Lisp data:

("6" "Cell with new\nLine" "6.28")

Row with newline a Org table:

| 6 | Cell with new      |       6.28 |
|   | Line               |            |

Rows in Org mode are separated using a newline-character which makes
"Cell with new" and "Line" as different cells which is different from
how Libreoffice Calc represents it.  Libreoffice Calc has a much
powerfull representation of a cell which is out of scope for any plain
text tables.  Possible solutions for this problem are:

1. Replace newline with a space character.  What will happen when
imported line as a really long line?

2. Represent every newline as distinct cell.  I was trying to implement
this method but the algorithm that I came up with requires time
complexity of O(n^2) to just print each row which is too much for large
CSV files.

(defun org-table--print-row (row)
  (let ((maxlines 1)
        list1 list2 tmp)
    ;; get maxlines
    (dolist (i row)
      (if (not (stringp i))
          (push (list i) list1)
        (push (string-lines i) list1)
        (setq tmp (length (car list1)))
        (when (< maxlines tmp)
          (setq maxlines tmp))))
    ;; normalize row
    (dolist (i list1)
      (setq tmp (length i))
      (when (< tmp maxlines)
        (setq i (append i (make-list (- maxlines tmp) ""))))
      (push i list2))
    ;; print row
    (dolist (i list2)
      (setq tmp (point))
      (dolist (j i))))) <-- Printing is still not complete

Thank you,
Utkarsh Singh

-- 
Utkarsh Singh
http://utkarshsingh.xyz