how to force org-mode to interpret number as string

how to force org-mode to interpret number as string

[Stefan Huchler]

I try to generate mail drafts from a table as template-tokens. 

The Problem is that that org-mode seems to auto-format the salary column
as a number. I don´t need to calc this numbers so saving/getting it as
string would be what I want. 

It not only butchers the string because he interprets the numbers behind
of the . as decimal point, while in germany we use the "," for that, and
the point as a way to mark thousends.

But even if he would understand that the number is 5000 and not 5, if
the outcommented replace-regexp throughs a error because its not string.

To convert it back makes no sense, the fields will always be interpreted
as replacement-strings.


#+TBLNAME: jobs
| jobname     | city          |  salary | email                 |
| Taxi-driver | New York City | 500.000 | mayor@gotham-city.com |
| Butcher     | Peking        |   5.000 | Jinping@china.cn      |

#+SRC_NAME: generate_job_mails_from_table
#+BEGIN_SRC emacs-lisp :var table=jobs header=jobs[0] data=jobs[1:-1] :results output

(setq column-length (length header))
(setq lines)
(dolist (line data)
  (setq tmp-line)
  (dotimes (i column-length)
    (add-to-list 'tmp-line (cons (nth i header) (nth i line)))
;    (print (nth i line)
  )
  (add-to-list 'lines tmp-line)
)

(message-mail)
(setq line1 (car lines))
(insert (cdr (assoc "email" line1)))
(message-goto-subject)
(insert "applying for the job: $jobname")
(dolist (var line1)
;  (replace-regexp (concat "$"(car var)) (cdr var ))
  (print (cdr var))
)
  
#+END_SRC

#+RESULTS:
: 
: "Jinping@china.cn"
: 
: 5.0
: 
: "Peking"
: 
: "Butcher"


btw, is there a easier way to generate this dict(python) like "lines" in
elisp?

Re: how to force org-mode to interpret number as string

[Bastien]

Hi Stefan,

Stefan Huchler <stefan.huchler@mail.de> writes:

> The Problem is that that org-mode seems to auto-format the salary column
> as a number.

That a problem here, but I guess this is handy in many circumstances.

Why don't use just turn the number into a string in your code?

-- 
 Bastien

Re: how to force org-mode to interpret number as string

[Stefan Huchler]

Bastien <bzg@gnu.org> writes:

> Hi Stefan,
>
> Stefan Huchler <stefan.huchler@mail.de> writes:
>
>> The Problem is that that org-mode seems to auto-format the salary column
>> as a number.
>
> That a problem here, but I guess this is handy in many circumstances.
>
> Why don't use just turn the number into a string in your code?

if I could do that, without loosing the numbers behind the point I would
be ok with that, but I dont want a fixed system for 1 table, but for a
random table. 

I wrote something similar in python:

https://github.com/spiderbit/mmailer

I just wanted to migrate it into org-mode/emacs because I have not to
deal with csv files and because I can use the email-ability from
emacs/gnus.

Its just stupid to have a seperate smtp-client-code and
smtp-auth-settings in such tool.

Its around 500 lines of code in emacs I would get that done in maybe 20
lines of code.


If that problem would not occur.


If there is something in elisp that is like that:

if is-that-a-number(column):
  column = number-to-string(give-the-complete-number(column))

I would be happy to use that ^^. But If he saves it as number I think he
will not save how many zeros were behind the point.

So basicly there is no solution to it that will satisfy me.

I really struggle to use the org-mode structures its always a little bit
off, it sounds all good but fails in small details.

Sorry whining but it is frustrating when you have 99% and you need to
get the 1% problem out of the way but it will not happen except writing
seperate 1000 lines of code in R or something like that (that was the
workaround for my last problem with my last "group-by" problem, with
org-mode tables.

That then makes the 99% just worthless because it is useless if its
static code that is not dynamic and works for this kind of problems but
only for this one example of the problem.


I dont want to repreat myself. Maybe I even could make that code a
seperate small emacs-package. Because there seems that there is no
serial-mail tool for emacs right now.

Maybe I am wrong here, but I searched long and did not find one.

Maybe you or somebody else can point me to it, then I maybe dont need
that to work anymore.


It´s just so stupid that seems for me very general problems, that you
maybe have a implizit declaration of a variable but you should still
have a way to define it manually there is even a way to do this, to
force a field to be a number but not to force it to be a string.

Something like that:

tblfm: $4 = ($4);N or was it \N why is there no \S and good.

ok its not in the information gathering and a seperate process but in
the table you have at least the full number there.


Hope there is one emacs-way to do that. That dont need learning another
programming language like R and works for any strings.

I am a bit disapointed in org-tables because they never work like I need
them too.

I think the only alterantive would be to save the org-capture data in a
csv-file so I have to write thousends of lines of code to manage paths
and projects and whatever like I have in python.

Then because csv works better in not converting without me saying it to
convert a string to a number.

and then use file-open blabla.


But nice that you did answer that fast ;)

Re: how to force org-mode to interpret number as string

[Eric Schulte]

Stefan Huchler <stefan.huchler@mail.de> writes:

> Bastien <bzg@gnu.org> writes:
>
>> Hi Stefan,
>>
>> Stefan Huchler <stefan.huchler@mail.de> writes:
>>
>>> The Problem is that that org-mode seems to auto-format the salary column
>>> as a number.
>>
>> That a problem here, but I guess this is handy in many circumstances.
>>
>> Why don't use just turn the number into a string in your code?
>
> if I could do that, without loosing the numbers behind the point I would
> be ok with that, but I dont want a fixed system for 1 table, but for a
> random table. 
>

I think that for this to work you may have to abandon using "." as your
numerical thousands separator.  Or customize string-to-number and
org-babel-number-p to fit your number syntax.

Best,

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: how to force org-mode to interpret number as string

[Stefan Huchler]

Eric Schulte <schulte.eric@gmail.com> writes:

> Stefan Huchler <stefan.huchler@mail.de> writes:
>
>> Bastien <bzg@gnu.org> writes:
>>
>>> Hi Stefan,
>>>
>>> Stefan Huchler <stefan.huchler@mail.de> writes:
>>>
>>>> The Problem is that that org-mode seems to auto-format the salary column
>>>> as a number.
>>>
>>> That a problem here, but I guess this is handy in many circumstances.
>>>
>>> Why don't use just turn the number into a string in your code?
>>
>> if I could do that, without loosing the numbers behind the point I would
>> be ok with that, but I dont want a fixed system for 1 table, but for a
>> random table. 
>>
>
> I think that for this to work you may have to abandon using "." as your
> numerical thousands separator.  Or customize string-to-number and
> org-babel-number-p to fit your number syntax.
>
> Best,

I think the way I have to go is to use export the table to csv then it
looses some of its "cleverness" and does what it should do in the first
place.

Hmm I just thought a second, maybe it does also do that
data-manipulation also before it exports it, but no org-table-export
does export it like a string.

So here you have one bug, its inconsistent.

So yes the easiest workaround is most likely to export it to a csv in /tmp/...
and then parse it back ^^.

Again I am suprised that there is no serial-mail package for emacs, then
I would not even have to write this code.


Sorry that I am maybe not so constructive but it´s stuff that should
work and now you say that I should fix that upstream with me having written 50 lines
of lisp code in my life.

And how big are the chances that it gets included, when you people do
not even consider that as a bug / needed feature.

But sorry I really flame to much, I just find some stuff really stupid,
as example why did I need to make out of a table a dict in the first
place to put it into a list with lists is really stupid, dicts are more
confortable I know in elisp maybe it has another name, but its similar
and even in python I get no list of dicts (lines).

Is it because of monstroses big Tables and performance problems?

Sorry I can´t stopp. Still there is much great stuff there. Else I would
not try to migrate my life into emacs ^^ ;)

Re: how to force org-mode to interpret number as string

[Eric Schulte]

>> I think that for this to work you may have to abandon using "." as your
>> numerical thousands separator.  Or customize string-to-number and
>> org-babel-number-p to fit your number syntax.
>>
>> Best,
>
> I think the way I have to go is to use export the table to csv then it
> looses some of its "cleverness" and does what it should do in the first
> place.
>
> Hmm I just thought a second, maybe it does also do that
> data-manipulation also before it exports it, but no org-table-export
> does export it like a string.
>
> So here you have one bug, its inconsistent.
>

Automatically recognizing numbers in tables is useful when converting
tables to code, but would make no sense for CSV export.  I don't see a
problem here.

>
> Sorry that I am maybe not so constructive but it´s stuff that should
> work and now you say that I should fix that upstream with me having
> written 50 lines of lisp code in my life.
>

I suggested that you remove "." from your numbers.  You could also wrap
your numbers in "s.

#+name: example
| "100.00000000" |

#+begin_src emacs-lisp :var example=example
(caar example)
#+end_src

#+RESULTS:
: 100.00000000

Furthermore, when I said "customize" I did not mean "fix upstream", I
meant re-define org-babel-number-p in your *personal* config file.  I
apologize if this was not clear.

>
> But sorry I really flame to much, I just find some stuff really stupid,
> as example why did I need to make out of a table a dict in the first
> place to put it into a list with lists is really stupid, dicts are more
> confortable I know in elisp maybe it has another name, but its similar
> and even in python I get no list of dicts (lines).
>
> Is it because of monstroses big Tables and performance problems?
>

I would argue that the list of lists representation is indeed more
commonly useful than converting a table to a list of dictionaries keyed
by the header row.  Maybe you will find the following helpful.

#+TBLNAME: jobs
| jobname     | city          |  salary | email                 |
| Taxi-driver | New York City | 500.000 | mayor@gotham-city.com |
| Butcher     | Peking        |   5.000 | Jinping@china.cn      |

#+name: table-w-headers-to-list-of-key-value
#+begin_src emacs-lisp :var data='()
  (let ((keys (car data))
        (vals (cdr data)))
    (mapcar (lambda (row) (map 'list #'cons keys row)) vals))
#+end_src

Example usage, pull the email from each value.
#+begin_src emacs-lisp :var data=table-w-headers-to-list-of-key-value(jobs)
(mapcar (lambda (row) (cdr (assoc "email" row))) data)
#+end_src

#+RESULTS:
| mayor@gotham-city.com | Jinping@china.cn |

-- 
Eric Schulte
https://cs.unm.edu/~eschulte
PGP: 0x614CA05D

Re: how to force org-mode to interpret number as string

[Stefan Huchler]

Eric Schulte <schulte.eric@gmail.com> writes:


> I suggested that you remove "." from your numbers.  You could also wrap
> your numbers in "s.

> #+name: example
> | "100.00000000" |
> #+begin_src emacs-lisp :var example=example
> (caar example)
> #+end_src
>
> #+RESULTS:
> : 100.00000000
>

thank you and sorry for my inpatiance.

That is a very helpful and constructive post.

And basicly that was one thing I searched that you can mark somehow
numbers as strings.

To rename the numbers that would work of course I just want a option
that works in any case, its not supposed to be a one-thing solution for
one problem, I want it to be usable for every data.

As a general serial mail tool, so renaming the numbers is no real good
solution, and the point on the thausends is pretty common here in
germany.

I dont need to use calc functionality in any way on this table, so I
don´t need proper number recognition besided if I need string its stupid
to first convert it to a number from string to convert it then back.
Especialy with all the "if (is number...)" block, and your mapping would
then also not work, because I have to check then every entry by 2 loops
like I did.

So to complete your solution, is there a way to wrap "" around of all
values in a table (or only on numbers)?

something like:

#tblfm: $3 = "$3"      (or for all columns?)

Then I think I am satisfied ;)



>
> I would argue that the list of lists representation is indeed more
> commonly useful than converting a table to a list of dictionaries keyed
> by the header row.  Maybe you will find the following helpful.
>
> #+TBLNAME: jobs
> | jobname     | city          |  salary | email                 |
> | Taxi-driver | New York City | 500.000 | mayor@gotham-city.com |
> | Butcher     | Peking        |   5.000 | Jinping@china.cn      |
>
> #+name: table-w-headers-to-list-of-key-value
> #+begin_src emacs-lisp :var data='()
>   (let ((keys (car data))
>         (vals (cdr data)))
>     (mapcar (lambda (row) (map 'list #'cons keys row)) vals))
> #+end_src
>
> Example usage, pull the email from each value.
> #+begin_src emacs-lisp :var data=table-w-headers-to-list-of-key-value(jobs)
> (mapcar (lambda (row) (cdr (assoc "email" row))) data)
> #+end_src
>
> #+RESULTS:
> | mayor@gotham-city.com | Jinping@china.cn |

thanx yes the problem is that I am not so used to the lambda construct,
and I did not want to learn that and a new language at the same time,
but yes I should get used to map methods and lambda.

It´s a bit shorter yes. thanks!

Re: how to force org-mode to interpret number as string

[Bastien]

Stefan Huchler <stefan.huchler@mail.de> writes:

> So to complete your solution, is there a way to wrap "" around of all
> values in a table (or only on numbers)?
>
> something like:
>
> #tblfm: $3 = "$3"      (or for all columns?)
>
> Then I think I am satisfied ;)

This may help:

| a  | 3 |
|----+---|
| z3 |   |
#+TBLFM: @2$1='(concat "z" "@1$2");L

The quotes around @1$2 tells to use the @1$2 reference as a string in
the sexp.  If you remove them, you get an error.

HTH,

-- 
 Bastien