* Re: Managing bibtex database using org-mode?
2014-05-07 10:29 ` Vikas Rawal
2014-05-08 10:27 ` Leonard Avery Randall
@ 2014-05-08 16:30 ` Richard Lawrence
2014-06-02 11:29 ` Leonard Avery Randall
2 siblings, 0 replies; 8+ messages in thread
From: Richard Lawrence @ 2014-05-08 16:30 UTC (permalink / raw)
To: emacs-orgmode; +Cc: Vikas Rawal
[-- Attachment #1: Type: text/plain, Size: 3371 bytes --]
Hi Vikas and all,
Vikas Rawal <vikaslists@agrarianresearch.org> writes:
>> I manage my whole bibtex database on org. It makes my workflow more
>> integrated. It allows me to keep bib info, todo states and notes all
>> in the same place, and it allows me to access it all through the
>> agenda. I just periodically run org-bibtex to make sure that I have a
>> updated bib file.
>
> This is exactly what I have in mind. Would you mind sharing an example
> file, and may be an outline of your work flow?
I also manage my whole bibtex database in Org. Here is my setup:
1) I store new readings as I come across them using a capture template.
(See the attached "reading.txt".) This template is pretty basic, but I
think it could easily be adapted to store more data, such as links to
PDFs, or automatically import Bibtex entries (over org-protocol?) using
a function from org-bibtex.
The relevant stanza from my org-capture-templates is:
#+BEGIN_SRC elisp
("dr" "Reading" entry
(file+olp ,dissertation-agenda-file "Reading list")
(file ,(concat org-template-directory "/reading.txt"))
:prepend t)
#+END_SRC
2) I use a post-capture hook to detect whether a captured entry is a
reading (it looks for the AUTHOR property) and optionally add
bibliographic data using org-bibtex-create-in-current-entry:
#+BEGIN_SRC elisp
(defun add-bibliographic-data ()
; this is a bit hacky: we detect the AUTHOR property, and create bibtex entries if
; it is present
(message "optionally adding bibliographic data")
(if (and (org-entry-get (point) "AUTHOR")
(y-or-n-p "Add bibliographic data? "))
; with prefix arg to get all fields:
(org-bibtex-create-in-current-entry 1)
nil))
(add-hook 'org-capture-before-finalize-hook (lambda () (add-bibliographic-data)))
#+END_SRC
This completes the `front-end' portion of my setup, which gets data on
new readings into Org.
The `back-end' setup, which exports the captured entries to a .bib file,
involves a Makefile that calls wrappers around org-bibtex functions.
3) All the wrapper functions do is walk over the Org tree containing my
reading list, using org-map-entries, and export each entry with a
defined CUSTOM_ID value using org-bibtex-headline to a new buffer. (It
skips those with no CUSTOM_ID, since these won't produce valid bibtex
entries.) This buffer then gets saved as the new .bib file. The code
is in bib-export.el, attached.
Note that this code makes some assumptions about my setup (mostly that
all reading entries are stored under a top-level "Reading list" heading
in a certain Org file); it is not suitable for general use without some
adaptation.
You can call the wrapper functions from within Emacs using
M-x reading-list-to-bibtex.
4) I also call the wrapper functions from a Makefile. This allows me to
get a fresh copy of my .bib file whenever it's needed, just by typing
`make bib'. Here's the relevant stanza from the Makefile, which lives
in the same directory as the file containing my reading list
(tasks.org):
#+BEGIN_SRC make
EMACS=emacs
BATCH_EMACS=$(EMACS) --batch -Q --load lib/el/org-dissertation.el
build/dissertation.bib: tasks.org lib/el/bib-export.el
$(BATCH_EMACS) --load lib/el/bib-export.el --file tasks.org \
--funcall reading-list-to-bibtex
bib: build/dissertation.bib
#+END_SRC
That's it! Hope you find that useful.
Best,
Richard
[-- Attachment #2: reading.txt --]
[-- Type: text/plain, Size: 170 bytes --]
** %^{Todo state|FIND|PRINT|READ|NOTES} [#%^{Priority|A|B|C}] %^{Description|Reading} %^g
%^{TITLE}p %^{AUTHOR}p %^{AREA}p %?
:PROPERTIES:
:Entered: %U
:END:
[-- Attachment #3: bib-export.el --]
[-- Type: application/emacs-lisp, Size: 1293 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: Managing bibtex database using org-mode?
2014-05-07 10:29 ` Vikas Rawal
2014-05-08 10:27 ` Leonard Avery Randall
2014-05-08 16:30 ` Richard Lawrence
@ 2014-06-02 11:29 ` Leonard Avery Randall
2014-06-02 17:31 ` John Kitchin
2 siblings, 1 reply; 8+ messages in thread
From: Leonard Avery Randall @ 2014-06-02 11:29 UTC (permalink / raw)
To: Vikas Rawal, Richard Lawrence; +Cc: org-mode mailing list
[-- Attachment #1.1: Type: text/plain, Size: 2047 bytes --]
Hi Vikas,
Sorry I did not reply earlier. Your original email prompted me to
develop a more functional workflow.
I have developed a few functions that make organizing research much
easier. I have functions that reorganize, rename, and add links to pdfs,
search for and add links to pdfs that have been organized by other
programs (Papers2), import notes from Skim, rename org-bibtex headlines
to a format I find more useful (e.g. Author (year) Title), and do a few
other things that I find convenient. The renaming functions also look up
crossrefs, and the headline renaming function lets you know if they are
missing. The functions are partially documented and have a few
customizable functions so they can be set up for different systems. Also
note that the pdf organizing system uses the conventions of my old
research organizing tool (Papers2) but this can be modified without too
much work. I have attached a file with the functions. The bottom of the
file also contains the variables that I have set and hooks that I have
added to make the functions integrate more smoothly into my set up. They
may serve as a guide if you choose to use them.
Additionally, here is the capture template I use for bibtex entries, it
is based largely on Richard's but it has you import more bibtex info
during the main capture process, and uses a function to help you find
crossrefs.
("r" "Reading" entry
(file+olp
"~/Google-Drive/Personal-Projects/Bib/Readinglist.org" "RLIST Inbox")
"** %^{Todo state|READ|FIND|PRINT|NOTES} [#%^{Priority|A|B|C}]
New Reading Entry %? %^{BIB_TITLE}p %^{BIB_AUTHOR}p %^{BIB_EDITOR}p
%^{BIB_YEAR}p %^{CUSTOM_ID}p %^g
:PROPERTIES:
:BIB_BTYPE: %^{Entry
type|book|article|inbook|bookinbook|incollection|suppbook|phdthesis|proceedings|inproceedings|booklet}
:ENTERED_ON: %U %(my-org-bibtex-crossref)
:END:" :prepend t)
When I have some time I will follow up to explain how the functions work
in more detail. In the mean time let me know if you have any questions.
All best,
Leonard
[-- Attachment #1.2: Type: text/html, Size: 3315 bytes --]
[-- Attachment #2: my-org-bibtex-functions.el --]
[-- Type: text/plain, Size: 40266 bytes --]
(setq-default bibtex-dialect 'biblatex)
;; setting a default bibtex-dialect makes `orb-bibtex-read' and
;; relatives work more smoothly, but I still come accross bugs.
(defcustom my-papers-directory nil
"Directory for filing papers for org-bibtex entries.
For optimal lin fuknctionality path should not contain spaces. Include trailing forward slash."
:group 'my-org-bibtex
:type '(choice
(const nil)
(string)))
(defcustom my-temp-pdf-dir nil
"Default directory used by `my-org-refile-supplements' to
search for pdfs. Include trailing forward slash."
:group 'my-org-bibtex
:type '(choice
(const nil)
(string)))
(defcustom my-miscellaneous-file-dir nil
"Directory used-by `my-org-refile-pdf' to refile supplements not associated with org-bibtex entries. Include trailing forward slash."
:group 'my-org-bibtex
:type '(choice
(const nil)
(string)))
(defcustom my-org-bibtex-rename-entry-searches-for-link t
"Determines if `my-org-bibtex-rename-entry' and `my-org-bibtex-header-replacer' look for and insert links using `my-org-bibtex-insert-link'"
:group 'my-org-bibtex
:type 'boolean)
(defcustom my-default-skim-notes-dir nil
"Default directory used-by `my-org-bibtex-capture-skim-notes'"
:group 'my-org-bibtex
:type '(choice
(const nil)
(string)))
(defcustom my-org-bibtex-standard-cite-command "\\cite"
"Citation Command used by `my-org-refile-skim-notes'"
:type 'string
:group 'my-org-bibtex)
(defun my-org-bibtex-header-replacer ()
"Replaces headers of all bibtex files in buffer using `my-org-bibtex-header-replacer'."
(interactive)
(let
((TYPE (concat org-bibtex-prefix org-bibtex-type-property-name)))
(save-excursion
(goto-char (point-max))
(while (not (or (org-before-first-heading-p) (= (point) (point-min))))
(backward-char 1)
(re-search-backward org-outline-regexp-bol nil t)
(when (org-entry-get (point) TYPE)
(my-org-bibtex-rename-entry))))))
(defun my-org-bibtex-rename-entry ()
"Replaces header of bibtex entry to the format Author (year) Title.
If `my-org-bibtex-rename-entry-searches-for-link' is non-nil it also looks for associated file and inserts link using `my-org-bibtex-insert-link'"
(interactive)
(let ((TYPE (concat org-bibtex-prefix org-bibtex-type-property-name))
(CROSSREF (concat org-bibtex-prefix "CROSSREF"))
(AUTHOR (concat org-bibtex-prefix "AUTHOR"))
(EDITOR (concat org-bibtex-prefix "EDITOR"))
(YEAR (concat org-bibtex-prefix "YEAR"))
(DATE (concat org-bibtex-prefix "DATE"))
(TITLE (concat org-bibtex-prefix "TITLE")))
(save-excursion
(unless (looking-at org-outline-regexp-bol)
(re-search-backward org-outline-regexp-bol))
(let* ((content (nth 4 (org-heading-components)))
(tags (nth 5 (org-heading-components)))
(crossref (org-entry-get (point) CROSSREF))
(creator (if (org-entry-get (point) AUTHOR)
(org-entry-get (point) AUTHOR)
(if (org-entry-get (point) EDITOR)
(concat (org-entry-get (point) EDITOR) " ed")
(if crossref
(my-org-find-crossref-entry
crossref AUTHOR EDITOR)
"Unknown"))))
(year (if (org-entry-get (point) YEAR)
(org-entry-get (point) YEAR)
(if (org-entry-get (point) DATE)
(org-entry-get (point) DATE)
(if crossref
(my-org-find-crossref-entry
crossref YEAR DATE)
"Unknown"))))
(title (org-entry-get (point) TITLE))
(nice-title (my-org-bibtex-convert-emphasis-and-quotes title))
(new-header (concat creator ". (" year ") " nice-title)))
(save-excursion
(end-of-line)
(search-backward content) (insert new-header)
;; this slightly indirect method maintains current visibility.
(let ((beg (point))) (end-of-line)
(delete-region beg (point)))
(org-set-tags-to tags)
;; check if file exists and insert
(if my-org-bibtex-rename-entry-searches-for-link
(my-org-bibtex-insert-link)))))))
(defun my-org-find-crossref-entry (crossref field &optional alternate)
(let* ((TYPE (concat org-bibtex-prefix org-bibtex-type-property-name))
(CROSSREF (concat org-bibtex-prefix "CROSSREF"))
(AUTHOR (concat org-bibtex-prefix "AUTHOR"))
(EDITOR (concat org-bibtex-prefix "EDITOR"))
(YEAR (concat org-bibtex-prefix "YEAR"))
(DATE (concat org-bibtex-prefix "DATE"))
(TITLE (concat org-bibtex-prefix "TITLE"))
(BIBKEY org-bibtex-key-property)
(ID (org-id-get nil t))
(marker (org-id-find ID t))
(cbuffer (current-buffer))
(buffer (marker-buffer marker)))
(set-buffer buffer)
(save-excursion
(goto-char (point-min))
(let ((crossrefp (concat "^[ \t]*:" BIBKEY ":[ \t]*" crossref "[ \t]*$")))
(if (re-search-forward crossrefp (point-max) t)
(if (org-entry-get (point) field)
(org-entry-get (point) field)
(if (and alternate (org-entry-get (point) alternate))
(if (string= alternate EDITOR)
(concat (org-entry-get (point) alternate) " ed")
(org-entry-get (point) alternate))
"Unknown"))
"Missing Crossref")))))
(defun my-org-bibtex-insert-link ()
"Check if corresponding file exists and if no link to file exists create it."
(interactive)
(let* ((TYPE (concat org-bibtex-prefix
org-bibtex-type-property-name))
(type (org-entry-get (point) TYPE))
(subdirectory (if (string= type "article")
"Articles/"
"Books/"))
(file-name (concat (my-org-bibtex-find-file-names) ".pdf"))
(full-file-path (concat my-papers-directory subdirectory
file-name))
(link (concat "file:" full-file-path)))
;; Check that no link to file currently exists, and that file exists.
(if type
(save-excursion
(unless (looking-at org-outline-regexp)
(re-search-backward org-outline-regexp))
(if (re-search-forward link nil t)
(when (called-interactively-p)
(message "Link already exists"))
(if (file-exists-p full-file-path)
;; Insert link in drawer with file name as description
(my-org-insert-link-in-drawer nil link file-name)
(when (called-interactively-p)
(error "The file %s does not exist." file-name)))))
(error "Not an org-bibtex entry"))))
(defun my-org-bibtex-find-file-names ()
"Create safe informative file names for supplements to org bibtex
entries."
(let* ((TYPE (concat org-bibtex-prefix org-bibtex-type-property-name))
(TITLE (concat org-bibtex-prefix "TITLE"))
(YEAR (concat org-bibtex-prefix "YEAR"))
(AUTHOR (concat org-bibtex-prefix "AUTHOR"))
(CROSSREF (concat org-bibtex-prefix "CROSSREF"))
(crossref (org-entry-get (point) CROSSREF))
(type (org-entry-get (point) TYPE))
(author (if (org-entry-get (point) AUTHOR)
(org-entry-get (point) AUTHOR)
(when crossref
(my-org-find-crossref-entry
crossref AUTHOR))))
(title (org-entry-get (point) TITLE))
(year (if (org-entry-get (point) YEAR)
(org-entry-get (point) YEAR)
(when crossref
(my-org-find-crossref-entry
crossref YEAR))))
(author-surname (if author
(replace-regexp-in-string ",.*" "" author) ""))
(safe-author
(if author
(concat (replace-regexp-in-string "[ \t]" "_"
author-surname) "_") ""))
;; convert csquotes and biblatex quote commands to straight
;; single quotes and delete emphasis.
(safe-title
(when title
(if (string-match "\\\\\\|{\\|}" title)
(my-org-bibtex-convert-emphasis-and-quotes title t)
title)))
;; straighten other quotes
(sq-title (when title
(replace-regexp-in-string "`" "'" safe-title)))
(file-title
(if title (concat "_"
(replace-regexp-in-string "[ \t]" "_" sq-title))
"")))
(concat safe-author year file-title)))
(defun my-org-refile-supplements (&optional supplement new-name refile-directory)
"Refile various files and place link to them in current entry.
If the current entry is an org-bibtex entry, it gives you the
option to refile it as the main pdf for the current entry or as a
supplement. Primary pdfs are refiled according to the function in
function specified in `my-org-bibtex-find-file-names'. The name
and location can also be set with the optional arguments NEW-NAME
and REFILE-DIRECTORY. NEW-NAME should be a string with the new
title of the supplemnt. Valid options for REFILE-DIRECTORY are
'main 'misc 'supp or a string containing a unix path to an
existing file directory. Other values will be ignored. The value
'main will refile the supplement as the main pdf of current
entry, providing various options if such an entry already exists,
'supp will refile it in the \"Supplements\" subdirectory of
either the \"Articles\" or \"Books\" subdirectory of
`my-papers-directory'. 'misc will refile the paper in
`my-miscellaneous-file-dir'. If the heading is not an org-bibtex
entry 'main and 'supp will also fall back on
`my-miscellaneous-file-directory'."
(interactive)
(let* ((old-name
(or supplement
(read-file-name "Supplement to refile: " my-temp-pdf-dir nil t)))
(TYPE (concat org-bibtex-prefix org-bibtex-type-property-name))
(type (org-entry-get (point) TYPE))
(subdirectory (if (equal type "article")
"Articles/"
"Books/"))
(file-name (when type
(concat (my-org-bibtex-find-file-names) ".pdf")))
(full-file-path (concat my-papers-directory subdirectory
file-name))
(enable-recursive-minibuffers t)
(p-name (lambda () (or new-name (read-from-minibuffer "Suplement name: "))))
prompt-name misc-name misc-link supp-name supp-link
(set-supp-name
(lambda ()
(setq prompt-name (funcall p-name))
(setq supp-name (concat my-papers-directory subdirectory
"Supplements/" prompt-name))
(setq supp-link (concat "file:" supp-name))))
(set-misc-name
(lambda ()
(setq prompt-name (funcall p-name))
(setq misc-name (concat my-miscellaneous-file-dir prompt-name))
(setq misc-link (concat "file:" misc-name)))))
;; Check if refile directory is explicitly named. If so refile
;; file in this directory
(if (and refile-directory
(not (or (equal refile-directory 'main)
(and (stringp refile-directory)
(file-directory-p refile-directory)))))
(list
(when (and (stringp refile-directory) (file-directory-p refile-directory))
(let ((prompt-name (funcall p-name))
(file-loc (concat refile-directory prompt-name)))
(rename-file old-name file-loc)
(my-org-insert-link-in-drawer nil (concat "file:" file-loc prompt-name) prompt-name)))
(when (equal refile-directory 'misc)
(list (funcall set-misc-name)
(rename-file old-name misc-name)
(my-org-insert-link-in-drawer nil misc-link prompt-name)))
(when (equal refile-directory 'supp)
(if type
(list (funcall set-supp-name)
(rename-file old-name supp-name)
(my-org-insert-link-in-drawer supp-link prompt-name))
(funcall set-misc-name)
(rename-file old-name misc-name)
(my-org-insert-link-in-drawer nil misc-link prompt-name))))
;; If not, check if file is an org-bibtex entry
(if type
(if (not (file-exists-p full-file-path))
(if (or (equal refile-directory 'main)
(y-or-n-p "Refile as main pdf for current entry? ")
(list (rename-file old-name full-file-path)
(my-org-bibtex-insert-link)))
(when (y-or-n-p) "Refile as supplement? "
(funcall set-supp-name)
(rename-file old-name supp-name)
(my-org-insert-link-in-drawer nil supp-link prompt-name)))
(if (y-or-n-p
"PDF associated with entry already exists. Replace it? ")
(list
(when (y-or-n-p "Keep current PDF as Supplement? ")
(funcall set-supp-name)
(rename-file full-file-path supp-name)
(my-org-insert-link-in-drawer nil supp-link prompt-name))
(rename-file old-name full-file-path t)
(my-org-bibtex-insert-link))
(when (y-or-n-p "Refile as supplement? ")
(funcall set-supp-name)
(rename-file old-name supp-name)
(my-org-insert-link-in-drawer nil supp-link prompt-name)
;; Make sure there is a link to the original file.
(my-org-bibtex-insert-link))))
;; if not in org-bibtex entry refile file in misc directory
(funcall set-misc-name)
(rename-file old-name misc-name)
(my-org-insert-link-in-drawer nil misc-link prompt-name)))))
(defun my-org-insert-links-drawer ()
"Insert a links drawer near the beginning of the current entry.
Inserts drawer near beginning of entry after scheduling, deadline
and clocking info and after standard logging, property and clock
drawers."
(interactive)
(org-back-to-heading t)
(looking-at org-outline-regexp)
(let ((indent (if org-adapt-indentation
(- (match-end 0) (match-beginning 0))
0))
(beg (point))
(re (concat "^[ \t]*" org-keyword-time-regexp))
end hiddenp)
(outline-next-heading)
(setq end (point))
(goto-char beg)
(while (re-search-forward re end t))
(setq hiddenp (outline-invisible-p))
(end-of-line 1)
(and (equal (char-after) ?\n) (forward-char 1))
(while (looking-at "^[ \t]*\\(:CLOCK:\\|:LOGBOOK:\\|CLOCK:\\|:PROPERTIES:\\|:END:\\)")
(if (member (match-string 1) '("CLOCK:" ":END:"))
;; just skip this line
(beginning-of-line 2)
;; Drawer start, find the end
(re-search-forward "^\\*+ \\|^[ \t]*:END:" nil t)
(beginning-of-line 1)))
(org-skip-over-state-notes)
(skip-chars-backward " \t\n\r")
(if (and (eq (char-before) ?*) (not (eq (char-after) ?\n)))
(forward-char 1))
(goto-char (point-at-eol))
(let ((inhibit-read-only t)) (insert "\n:LINKS:\n:END:"))
(beginning-of-line 0)
(org-indent-to-column indent)
(beginning-of-line 2)
(org-indent-to-column indent)
(beginning-of-line 0)
(if hiddenp
(save-excursion
(org-back-to-heading t)
(hide-entry))
(org-flag-drawer t))))
(defun my-org-insert-link-in-drawer (&optional complete-file link-location default-description)
"Find or create links drawer then insert link with `org-insert-link'.
The arguments COMPlETE-FILE LINK-LOCATION and DEFAULT-DESCRIPTION
are passsed on to `org-insert-link'. See its documentation for
information on how these arguments are handled."
(interactive)
;; make sure that we are currently in an entry.
(if (not (org-before-first-heading-p))
;; look for links drawer.
(save-excursion
(org-back-to-heading t)
(let ((beg (point))
end hiddenp)
(outline-next-heading)
(setq end (point))
(backward-char 1)
(setq hiddenp (outline-invisible-p))
(goto-char beg)
(unless (and (re-search-forward "[ \t]*:LINKS:[ \t]*$" end t)
(re-search-forward "[ \t]*:END:[ \t]*$" end t))
;; if no links drawer found create one.
(my-org-insert-links-drawer))
;; go to links drawer.
(goto-char beg)
(re-search-forward "[ \t]*:LINKS:[ \t]*$")
;; open new line and insert link.
(newline)
(org-insert-link complete-file link-location default-description)
(if hiddenp
(hide-entry))))
;; Give error if before first heading.
(error "Before first heading")))
(defun my-org-bibtex-yank ()
(interactive)
(let ((tempbib-file (concat temporary-file-directory "tempbib.bib")))
(delete-file tempbib-file)
(with-temp-file tempbib-file
(yank)
(my-org-bibtex-preparatory-conversions))
(org-bibtex-import-from-file tempbib-file)))
(defun my-org-bibtex-preparatory-conversions (&optional delimited start end)
(interactive)
(save-excursion
(beginning-of-buffer)
(replace-regexp "^[ \t]*\\([a-z:]*\\)[ \t]*=[ \t]*" "\\1=" delimited start end)
(beginning-of-buffer)
(replace-regexp "\\(title=\\){\\({.*}\\)},$" "\\1\\2" delimited start end)))
(defun add-bibliographic-data ()
"Hook for adding bibliographic data during capture process.
Note, this function places cursor at beginning of buffer to
ensure that it is in main header for post capture hooks. This
allows for subheaders with notes without messing up capture
process. It also makes this function unsuitable for use in main
buffer."
(message "optionally adding bibliographic data")
(when (and (org-entry-get (point) "CUSTOM_ID")
(y-or-n-p "Add bibliographic data? "))
(beginning-of-buffer)
(save-excursion
(if (y-or-n-p "Use Optional Fields?")
(org-bibtex-create-in-current-entry 1)
(org-bibtex-create-in-current-entry)))
))
(defun my-org-search-for-supplements-p ()
(when (y-or-n-p "Search for supplements?")
(my-org-refile-supplements)))
;;** Crossref insertion
(defun my-org-bibtex-crossref ()
(interactive)
(if (y-or-n-p "Add a crossref ?")
(let ((reftex-cite-format '((?\C-m . "%l")))
(enable-recursive-minibuffers t)
crossref)
(if (y-or-n-p "Use Reftex?")
(with-temp-buffer
(insert (concat "\n:" org-bibtex-prefix "CROSSREF: "))
(reftex-citation)
(setq crossref (buffer-string))
(message "%s" crossref))
(setq crossref (concat "\n:" org-bibtex-prefix "CROSSREF: "
(read-from-minibuffer "Insert Crossref. ")))
(message "%s" crossref)))
""))
(defun my-org-bibtex-convert-emphasis-and-quotes (string &optional delete-emphasis)
"Converts quotes and emphasis from various sorts of tex emphasis, into single straight quotes and org-mode bold markup. If DELETE-EMPHASIS is non nil it deletes emphasis rather than converting it."
(with-temp-buffer
(insert string)
(end-of-buffer)
(let* ((emph "\\\\emph{\\|\\\\textit{\\|\\\\textbf{")
(quotes "\\\\enquote{\\|\\\\mkbibquote{")
(tex-markup (concat emph "\\|" quotes))
(smartparens-mode t)
beg end text-begin)
(while (re-search-backward tex-markup (point-min) t)
(setq beg (point))
(if (looking-at emph)
(if delete-emphasis
(list
(re-search-forward tex-markup)
(setq text-begin (point))
(backward-char 1)
(sp-forward-sexp)
(delete-char -1)
(delete-region beg text-begin)
(goto-char beg))
(re-search-forward tex-markup)
(setq text-begin (point))
(backward-char 1)
(sp-forward-sexp)
(delete-char -1)
(insert "\*")
(delete-region beg text-begin)
(goto-char beg)
(insert "\*"))
(re-search-forward tex-markup)
(setq text-begin (point))
(backward-char 1)
(sp-forward-sexp)
(delete-char -1)
(insert "\'")
(delete-region beg text-begin)
(goto-char beg)
(insert "\'"))))
(buffer-string)))
;; (defun my-org-refile-skim-notes (notes-file &optional arg)
;; "Formats and refiles notes exported as text from Skim.app.
;; If called from an an entry inside an org file, it creates an
;; indirect buffer for this entry, and Prompts the user to search
;; for files in `my-default-skim-notes-dir'. Once file is selected
;; it formats the notes and refiles them to the subtree of the
;; current entry, under the subheading \"Unprocessed Notes\" Be
;; aware that this function only works if entry falls under
;; `org-refile-targets'.
;; It will format notes labeled by skim as highlights, strike outs
;; or underlines as quotes. If the note file contains text
;; containing text formatted \"--**pp**--\" it will treat \"pp\" as
;; the number of the first page and correct the page number markers
;; given by skim. If the current entry is an org-bibtex entry, it
;; will also insert bibtex cite commands at the end of each note
;; that is marked with a page number.
;; Note that if page numbers in skim match the page numbers in the
;; document you should still include a note with the text
;; \"--**0**--\" so that the function knows the page numbers are
;; correct. Otherwise it will not insert cite commands.
;; Additionally, note that if the pdf contains a cover page, you
;; should adjust the mark accordingly. So, if an article begins on
;; p. 299 but also has a cover page, you should include a note
;; containing \"--**298**--\" rather than \"--**299**--\".
;; Usage notes. Skim formats individual notes such that they look
;; like top level headers, so it is possible to put subpoints in
;; your notes while working in skim. Simply open the note and enter
;; \"** \" and whatever header you like at the beginning of a new
;; line. If you include \"page, pp\" in the header where pp is the
;; page number assigned to the page by skim, the note will also be
;; processed like all other headings. Also note that skim allows you
;; to edit the text of highlights, underlines, and strike outs.
;; Accordingly it is possible to add comments to a highlight by
;; selecting the highlight and adding subordinate headlines after
;; the highlighted text. This will ensure that only the highlighted
;; text is formatted as a quote."
;; (interactive (list (read-file-name "Notes file to import: "
;; my-default-skim-notes-dir
;; nil 'confirm)))
;; (when (not (or (equal major-mode 'org-mode) (equal major-mode 'org-agenda-mode)))
;; (error "Not in org or agenda buffer."))
;; (push-mark)
;; (if (equal major-mode 'org-agenda-mode)
;; (org-agenda-tree-to-indirect-buffer t)
;; (org-tree-to-indirect-buffer t))
;; (save-buffer)
;; (if (equal org-indirect-buffer-display 'other-window) (other-window 1))
;; (beginning-of-buffer)
;; (let* ((org-file (buffer-file-name))
;; (ind-buffer (buffer-name))
;; (TYPE (concat org-bibtex-prefix org-bibtex-type-property-name))
;; (type (org-entry-get (point) TYPE))
;; (entry-id (org-id-get (point) t))
;; (bib-key (org-entry-get (point) org-bibtex-key-property))
;; (org-refile-use-outline-path nil)
;; (org-refile-use-cache nil)
;; (org-refile-target-verify-function (lambda () (equal (org-id-get)
;; entry-id))))
;; (set-buffer (find-file-noselect notes-file))
;; (org-mode)
;; (beginning-of-buffer)
;; (let* ((pnum-p (re-search-forward "--\\*\\*\\([0-9]+\\)\\*\\*--"
;; (point-max) t))
;; (init-pnum-str (when pnum-p (match-string-no-properties 1)))
;; (init-pnum (if pnum-p (string-to-number init-pnum-str) 0)))
;; (end-of-buffer)
;; (while (not (or (org-before-first-heading-p) (= (point)
;; (point-min))))
;; (unless (looking-at org-outline-regexp-bol)
;; (re-search-backward org-outline-regexp-bol))
;; (let* ((heading (nth 4 (org-heading-components)))
;; (headpageref ", page \\([0-9]+\\)")
;; (quoteref "\\*+ \\(Highlight\\|Underline\\|Strike Out\\),")
;; (beg-quote "#+BEGIN_QUOTE")
;; (end-quote "#+END_QUOTE")
;; end-of-entry)
;; (if (and pnum-p (string-match headpageref heading))
;; (let* ((raw-pnum-str (match-string-no-properties 1 heading))
;; (raw-pnum (string-to-number raw-pnum-str))
;; (pnum (+ raw-pnum init-pnum -1))
;; (pnum-str (number-to-string pnum))
;; (pnum-cite-str (concat my-org-bibtex-standard-cite-command
;; "[" pnum-str "]{"
;; bib-key "}"))
;; (pnum-head-str (concat ",page " pnum-str))
;; (pnum-notes-str (concat "Notes related to " pnum-cite-str))
;; entry-end)
;; (save-excursion
;; (re-search-forward headpageref)
;; (setq end (point))
;; (re-search-backward headpageref)
;; (delete-region (point) end)
;; (insert pnum-head-str))
;; (save-excursion
;; (if (looking-at quoteref)
;; (list
;; (end-of-line)
;; ;; (save-excursion
;; ;; (outline-next-heading)
;; ;; (setq entry-end (point)))
;; ;; (when (re-search-forward "[^ \t\n]" entry-end t)
;; ;; (backward-char 1))
;; ;; (newline)
;; (newline) (insert beg-quote)
;; (outline-next-heading)
;; ;; (re-search-backward "[^ \t\n]" nil t)
;; ;; (forward-char 1)
;; (open-line 2)
;; (when bib-key
;; (insert pnum-cite-str)
;; (newline))
;; (insert end-quote))
;; (when bib-key
;; (outline-next-heading)
;; ;; (re-search-backward "[^ \t\n]")
;; ;; (forward-char 1)
;; (open-line 2)
;; (insert pnum-notes-str)))))
;; (save-excursion
;; (when (looking-at quoteref)
;; (end-of-line) (newline) (insert beg-quote)
;; (if (outline-next-heading)
;; (open-line 2)
;; (newline))
;; (insert end-quote))))
;; (save-excursion (org-demote))
;; (unless (equal (point) (point-min))
;; (backward-char 1))))
;; (beginning-of-buffer)
;; (save-excursion
;; (replace-regexp "[ \n\t]*#+END_QUOTE" "/n")
;; (save-excursion "#+BEGIN_QUOTE"))
;; (insert "* Unprocessed notes\n")
;; (beginning-of-line 0)
;; (let ((mr mark-ring) (gmr global-mark-ring))
;; (push-mark (point-max))
;; (activate-mark)
;; (org-refile 3)
;; (setq mark-ring mr
;; global-mark-ring gmr)) ;;; I seem to have made it safely to here the
;; ;;; problem appears to be with org-refile.
;; (when (y-or-n-p "Save processed notes file? ")
;; (write-file (concat notes-file "-autoconverted.org ")))
;; (set-buffer ind-buffer)
;; (when (y-or-n-p "Refile as supplement? ")
;; (if type
;; (my-org-refile-supplements (concat notes-file "-autoconverted.org")
;; (concat (my-org-bibtex-find-file-names)
;; "-unprocesed-notes.org") 'supp)
;; (my-org-refile-supplements (concat notes-file "-autoconverted.org")
;; nil 'misc)))
;; (when (not (y-or-n-p "Keep original file? ")
;; ;; delete file, or if `delete-by-moving-to-trash' is
;; ;; non-nil move to trash
;; (delete-file notes-file t))))))
(defun my-org-refile-skim-notes (notes-file &optional arg)
"Formats and refiles notes exported as text from Skim.app.
If called from an an entry inside an org file, it creates an
indirect buffer for this entry, and Prompts the user to search
for files in `my-default-skim-notes-dir'. Once file is selected
it formats the notes and refiles them to the subtree of the
current entry, under the subheading \"Unprocessed Notes\" Be
aware that this function only works if entry falls under
`org-refile-targets'.
It will format notes labeled by skim as highlights, strike outs
or underlines as quotes. If the note file contains text
containing text formatted \"--**pp**--\" it will treat \"pp\" as
the number of the first page and correct the page number markers
given by skim. If the current entry is an org-bibtex entry, it
will also insert bibtex cite commands at the end of each note
that is marked with a page number.
Note that if page numbers in skim match the page numbers in the
document you should still include a note with the text
\"--**0**--\" so that the function knows the page numbers are
correct. Otherwise it will not insert cite commands.
Additionally, note that if the pdf contains a cover page, you
should adjust the mark accordingly. So, if an article begins on
p. 299 but also has a cover page, you should include a note
containing \"--**298**--\" rather than \"--**299**--\".
Usage notes. Skim formats individual notes such that they look
like top level headers, so it is possible to put subpoints in
your notes while working in skim. Simply open the note and enter
\"** \" and whatever header you like at the beginning of a new
line. If you include \"page, pp\" in the header where pp is the
page number assigned to the page by skim, the note will also be
given a cite command formatted as a comment. Regardless it will
contain the page number and source as properties. Also note that
skim allows you to edit the text of highlights, underlines, and
strike outs. Accordingly it is possible to add comments to a
highlight by selecting the highlight and adding subordinate
headlines after the highlighted text. This will ensure that only
the highlighted text is formatted as a quote."
(interactive (list (read-file-name "Notes file to import: "
my-default-skim-notes-dir
nil 'confirm)))
(when (not (or (equal major-mode 'org-mode) (equal major-mode 'org-agenda-mode)))
(error "Not in org or agenda buffer."))
(push-mark)
(when (equal major-mode 'org-agenda-mode)
(org-agenda-goto))
(save-buffer)
(let* ((org-file (buffer-file-name))
(TYPE (concat org-bibtex-prefix org-bibtex-type-property-name))
(type (org-entry-get (point) TYPE))
(entry-id (org-id-get (point) t))
(orig-buffer (current-buffer))
(bib-key (org-entry-get (point) org-bibtex-key-property))
(org-refile-use-outline-path nil)
(org-refile-use-cache nil)
(org-refile-target-verify-function (lambda () (equal (org-id-get)
entry-id)))
orig-notes processed-notes)
(set-buffer (find-file-noselect notes-file))
(setq orig-notes (buffer-string))
(with-temp-buffer
(insert orig-notes)
(my-skim-notes-transformations bib-key entry-id)
(my-org-regularize-whitespace)
(beginning-of-buffer)
(my-fullstop-regexp)
(beginning-of-buffer)
(insert "* Unprocessed notes\n")
(beginning-of-line 0)
;; (let ((mr mark-ring) (gmr global-mark-ring))
;; (push-mark (point-max))
;; (activate-mark)
(org-refile))
;; (setq mark-ring mr
;; global-mark-ring gmr))
;;; I seem to have made it safely to here the
;;; problem appears to be with org-refile.
;; (when (y-or-n-p "Save processed notes file? ")
;; (write-file (concat notes-file "-autoconverted.org ")))
;; (set-buffer ind-buffer)
;; (when (y-or-n-p "Refile as supplement? ")
;; (if type
;; (my-org-refile-supplements (concat notes-file "-autoconverted.org")
;; (concat (my-org-bibtex-find-file-names)
;; "-unprocesed-notes.org") 'supp)
;; (my-org-refile-supplements (concat notes-file "-autoconverted.org")
;; nil 'misc)))
(org-id-goto entry-id)
(org-tree-to-indirect-buffer)
(equal org-indirect-buffer-display 'other-window) (other-window 1)
(beginning-of-buffer)
(when (not (y-or-n-p "Keep original file? "))
;; delete file, or if `delete-by-moving-to-trash' is
;; non-nil move to trash
(delete-file notes-file t))))
(defun my-skim-notes-transformations (&optional bib-key id)
(interactive)
(org-mode)
(beginning-of-buffer)
(let* ((pnum-p (re-search-forward "--\\*\\*\\([0-9]+\\)\\*\\*--"
(point-max) t))
(init-pnum-str (when pnum-p (match-string-no-properties 1)))
(init-pnum (if pnum-p (string-to-number init-pnum-str) 0))
(headpageref ", [pP]age \\([0-9]+\\)")
(quoteref "\\*+ \\(Highlight\\|Underline\\|Strike Out\\),")
(beg-quote "#+BEGIN_QUOTE")
(end-quote "#+END_QUOTE")
end-of-entry)
(beginning-of-buffer)
(while (not (= (point) (point-max)))
(if (org-before-first-heading-p)
(outline-next-heading))
(let* ((heading (nth 4 (org-heading-components))))
(if (and pnum-p heading (string-match headpageref heading))
(let* ((raw-pnum-str (match-string-no-properties 1 heading))
(raw-pnum (string-to-number raw-pnum-str))
(pnum (+ raw-pnum init-pnum -1))
(pnum-str (number-to-string pnum))
(pnum-cite-str (concat my-org-bibtex-standard-cite-command
"[" pnum-str "]{"
bib-key "}"))
(pnum-head-str (concat ", page " pnum-str))
(pnum-notes-str (concat "# Notes related to " pnum-cite-str))
entry-end)
(save-excursion
(re-search-forward headpageref)
(setq end (point))
(re-search-backward headpageref)
(delete-region (point) end)
(insert pnum-head-str))
(if (looking-at quoteref)
(list
(save-excursion
(end-of-line)
;; (save-excursion
;; (outline-next-heading)
;; (setq entry-end (point)))
;; (when (re-search-forward "[^ \t\n]" entry-end t)
;; (backward-char 1))
;; (newline)
(newline 2) (insert beg-quote)
(if (outline-next-heading)
;; (re-search-backward "[^ \t\n]" nil t)
;; (forward-char 1)
(open-line 2)
(newline))
(when bib-key
(insert pnum-cite-str)
(newline))
(insert end-quote))
(org-entry-put (point) "NOTES_PAGE" pnum-str)
(org-entry-put (point) "NOTES_PDF_PAGE" raw-pnum-str))
(when bib-key
(org-entry-put (point) "NOTES_PAGE" pnum-str)
(org-entry-put (point) "NOTES_PDF_PAGE" raw-pnum-str)
(save-excursion
(if (outline-next-heading)
;; (re-search-backward "[^ \t\n]")
;; (forward-char 1)
(open-line 2)
(newline))
(insert pnum-notes-str)))))
(when (looking-at quoteref)
(save-excursion
(end-of-line) (newline) (open-line 2) (insert beg-quote)
(if (outline-next-heading)
(open-line 2)
(newline))
(insert end-quote)))
(when (and heading (string-match headpageref heading))
(let ((raw-pnum-str (match-string-no-properties 1 heading)))
(org-entry-put (point) "NOTES_PDF_PAGE" raw-pnum-str)))))
(save-excursion (org-demote))
(if (looking-at quoteref)
(org-entry-put (point) "NOTE_TYPE" "quote")
(org-entry-put (point) "NOTE_TYPE" "notes"))
(if bib-key
(org-entry-put (point) "SOURCE_BIB_KEY" bib-key))
(if id
(org-entry-put (point) "SOURCE_ENTRY_ID" id))
(org-entry-put (point) "IMPORTED_ON" (format-time-string "%Y-%m-%d"))
(outline-next-heading))
(beginning-of-buffer)
(while (not (= (point) (point-max)))
(if (org-before-first-heading-p)
(outline-next-heading))
(let ((cheading (point)))
(save-excursion
(when (org-up-heading-safe)
(let ((ppnum (org-entry-get (point) "NOTES_PAGE"))
(ppdfpnum (org-entry-get (point) "NOTES_PDF_PAGE")))
(when ppnum
(org-entry-put cheading "NOTES_PAGE" ppnum))
(if ppdfpnum
(org-entry-put cheading "NOTES_PDF_PAGE" ppdfpnum)))))
(outline-next-heading)))))
(defun my-org-regularize-whitespace ()
(interactive)
(save-excursion
(beginning-of-buffer)
(replace-regexp
"\n\\{2,\\}\\(#\\+END_QUOTE\\|#\\+LaTeX\\|\\\\.*cite\\)"
"\n\\1")
(beginning-of-buffer)
(replace-regexp "\\(#\\+BEGIN_QUOTE\\)\n\\{2,\\}" "\\1\n")
(beginning-of-buffer)
(replace-regexp "\n\\{3,\\}" "\n\n")
(beginning-of-buffer)
(replace-regexp "[ \t]+$" "")))
;;* Add capture hooks
(add-hook 'org-capture-prepare-finalize-hook 'add-bibliographic-data)
(add-hook 'org-capture-before-finalize-hook 'my-org-search-for-supplements-p)
(add-hook 'org-capture-before-finalize-hook 'my-org-bibtex-rename-entry)
;;* Keyboard shortcuts
(global-set-key "\C-co" 'my-org-refile-skim-notes)
(global-set-key "\C-cy" 'my-org-bibtex-yank)
(global-set-key (kbd "C-c L") 'my-org-insert-link-in-drawer)
(global-set-key (kbd "C-c M-a") 'my-org-bibtex-rename-entry)
(global-set-key (kbd "C-c M-l") 'my-org-bibtex-insert-link)
(global-set-key (kbd "C-c C-w") 'my-org-refile-supplements)
;;* My variables
(setq my-miscellaneous-file-dir
"/Users/leotr/Google-Drive/Miscellaneous-Supplements/"
my-papers-directory "/Users/leotr/Google-Drive/Papers2/"
my-temp-pdf-dir "/Users/leotr/tmp/pdfs/"
my-default-skim-notes-dir "/Users/leotr/tmp/notes/")
^ permalink raw reply [flat|nested] 8+ messages in thread