`orb--pre-expand-template` to use `bibtex-completion-get-value` instead of `bibtex-completion-apa-get-value`

Is there a better way to get orb--pre-expand-template to use bibtex-completion-get-value instead of bibtex-completion-apa-get-value?

My present hack involves redefining the whole function in my config.el under Doom, which is cumbersome.

(defun orb--pre-expand-template (template entry)
  "Helper function for `orb--new-note'.
TEMPLATE is an element of `org-roam-capture-templates' and ENTRY
is a BibTeX entry as returned by `bibtex-completion-get-entry'."
  ;; Handle org-roam-capture part
  (letrec (;; Org-capture templates: handle different types of
           ;; org-capture-templates: string, file and function; this is
           ;; a stripped down version of `org-capture-get-template'
           (org-template
            (pcase (nth 3 template)       ; org-capture template is here
              (`nil 'nil)
              ((and (pred stringp) tmpl) tmpl)
              (`(file ,file)
               (let ((flnm (expand-file-name file org-directory)))
                 (if (file-exists-p flnm) (f-read-text flnm)
                   (format "Template file %S not found" file))))
              (`(function ,fun)
               (if (functionp fun) (funcall fun)
                 (format "Template function %S not found" fun)))
              (_ (user-error "ORB: Invalid capture template"))))
           ;;  org-roam capture properties are here
           (plst (cddddr template))
           ;; regexp for org-capture prompt wildcard
           (rx "\\(%\\^{[[:alnum:]-_]*}\\)")
           (file-keyword (when orb-process-file-keyword
                           (or (and (stringp orb-process-file-keyword)
                                    orb-process-file-keyword)
                               "file")))
           ;; inline function to handle :if-new list expansion
           (expand-roam-template
            (lambda (roam-template-list old new)
              (let (elements)
                (dolist (el roam-template-list)
                  (if (listp el)
                      (setq elements
                            (nreverse
                             (append elements
                                     (list (funcall expand-roam-template
                                                    el old new)))))
                    (push (s-replace old new el) elements)))
                (nreverse elements))))
           (lst nil))
    ;; First run:
    ;; 1) Make a list of (org-wildcard field-value match-position) for the
    ;; second run
    ;; 2) replace org-roam-capture wildcards
    (dolist (keyword orb-preformat-keywords)
      (let* (;; prompt wildcard keyword
             (keyword (cond
                       ;; for some backward compatibility with old
                       ;; `orb-preformat-keywords'
                       ((consp keyword) (car keyword))
                       ((stringp keyword) keyword)
                       (t (user-error "Error in `orb-preformat-keywords': \
Keyword \"%s\" has invalid type (string was expected)" keyword))))
             ;; bibtex field name
             (field-name (or (car (rassoc keyword orb-bibtex-field-aliases))
                             keyword))
             ;; get the bibtex field value
             (field-value
              ;; maybe process file keyword
              (or (if (and file-keyword (string= field-name file-keyword))
                      (prog1
                          (orb-process-file-field
                           (bibtex-completion-get-value "=key=" entry))
                        ;; we're done so don't even compare file-name with
                        ;; file-keyword in the successive cycles
                        (setq file-keyword nil))
                    ;; do the usual processing otherwise
                    ;; condition-case to temporary workaround an upstream bug
                    (condition-case nil
                        (bibtex-completion-get-value field-name entry)
                      (error "")))
                  ""))
             ;; org-capture prompt wildcard
             (org-wildcard (concat "%^{" (or keyword "citekey") "}"))
             ;; org-roam-capture prompt wildcard
             (roam-wildcard (concat "${" (or keyword "citekey") "}"))
             ;; org-roam-capture :if-new property
             (roam-template (plist-get plst :if-new))
             (i 1)                      ; match counter
             pos)
        ;; Search for org-wildcard, set flag m if found
        (when org-template
          (while (string-match rx org-template pos)
            (if (string= (match-string 1 org-template) org-wildcard)
                (progn
                  (setq pos (length org-template))
                  (cl-pushnew (list org-wildcard field-value i) lst ))
              (setq pos (match-end 1)
                    i (1+ i)))))
        ;; Replace placeholders in org-roam-capture-templates :if-new property
        (when roam-template
          (setcdr roam-template
                  (funcall expand-roam-template
                           (cdr roam-template) roam-wildcard field-value)))))
    ;; Second run: replace prompts and prompt matches in org-capture
    ;; template string
    (dolist (l lst)
      (when (and org-template (nth 1 l))
        (let ((pos (concat "%\\" (number-to-string (nth 2 l)))))
          ;; replace prompt match wildcards with prompt wildcards
          ;; replace prompt wildcards with BibTeX field value
          (setq org-template (s-replace pos (car l) org-template)
                org-template (s-replace (car l) (nth 1 l) org-template))))
      (setf (nth 3 template) org-template))
    template))

The Issue:

"${author}" expands to include a trailing comma with bibtex-completion-apa-get-value, which is undesirable.


Sample Code:

(setq entry (bibtex-completion-get-entry "林素英_郭店服喪思想_2003"))

(s-format
   (concat "${author}" (annobib-cited-year-zh) "《${title}》。${location}:${publisher}。")
   'bibtex-completion-apa-get-value entry)

;; use bibtex-completion-get-value to see difference

Resultant File-Header with “${author}” expansion:

:PROPERTIES:
:ID:       b894eb9b-182a-4082-8320-12379602d26c
:ROAM_REFS: cite:林素英_郭店服喪思想_2003
:END:
#+type: book
#+author: 林素英, 
#+title: 從郭店簡探究其倫常觀念:以服喪思想為討論基點

Sample Bibtex:

@book{林素英_郭店服喪思想_2003,
  title = {從郭店簡探究其倫常觀念:以服喪思想為討論基點},
  shorttitle = {從郭店簡探究其倫常觀念},
  author = {{林素英}},
  date = {2003},
  edition = {初版},
  publisher = {{萬卷樓}},
  location = {{台北市}},
  isbn = {978-957-739-425-5},
  langid = {chi},
  keywords = {人道,喪禮,硏究與考訂,簡牘}
}

Hi, please file a feature request on Github. The name can be something like “Allow for a custom bibtex-completion-get-value function”

In case anyone else need to know, the issue has been resolved on Github.

Since this post is the only one in the archive that matches my query for “orb–pre-expand-template”, I think it makes sense to add my question here.

From looking through the code of orb–pre-expand-template, its not obvious to me what I would need to do in order to create a new keyword that I could substitute into my template.

The specific thing I have in mind is to have a value like “Einstein '05” that I could prepend to the title of an org-noter file.

I imagine I could do this by using my own bibtex-completion-get-value function, that knows about this one field, and forwards the rest on to the standard get-value function, but if there’s an easier way to accomplish this, I’m all :ear:s

@gcoladon I’m not sure what you actually mean by “an org-noter file”. Strictly speaking, there is no such thing as. Is it an Org note managed by Org-roam where you keep your Org-noter annotations or the associated PDF file that you mean? An example of what you are trying to achieve would be very helpful.

Thanks for engaging @mshevchuk :slight_smile: I mean what you wrote above. My template for the creation of that file is

("n" "ref+noter" plain "%?" :if-new
      (file+head "roam-stem/${citekey}.org" "#+TITLE: ${author-year} - ${title}
#+ROAM_KEY: cite:${citekey}
#+ROAM_TAGS:

* ${author-year} - ${title}
:PROPERTIES:
:URL: ${url}
:AUTHOR: ${author-or-editor-abbrev}
:NOTER_DOCUMENT: ${file}
:NOTER_PAGE:
:END:
** Abstract
${abstract}
")

I would like to implement an author-year field so that the headline of the org-noter section was something like “Shevchuk '21 - org-roam-bibtex: the best thing ever”.

You hit the spot @gcoladon :rofl:

This can be achieved with a standard Org-capture template tool - expansion of arbitrary %(sexp) expressions:

...
#+TITLE: ${author} '%(substring \"${year}\" 2 4) - ${title}
...

The above will produce the following:

#+TITLE: Last, F. M. 'YY - Title

Mind the escaped double quotes inside the %(substring ...) expression. These are needed since your template is itself a string. Also remember to add year (or date when using biblatex) to orb-preformat-keywords.

To simplify the things a bit I’d recommend moving the template string to a separate file.

("n" "ref+noter" plain 
    (file "/path/to/my/template.file") 
    :if-new (file+head "roam-stem/${citekey}.org" ""))

Then the contents of template.file could look as follows:

#+TITLE: %^{author} '%(substring "%^{year}" 2 4) - %^{title}
#+ROAM_KEY: cite:%^{citekey}
#+ROAM_TAGS:

* %^{author} '%(substring "%^{year}" 2 4) - %^{title}
:PROPERTIES:
:URL: %^{url}
:AUTHOR: %^{author-or-editor-abbrev}
:NOTER_DOCUMENT: %^{file}
:NOTER_PAGE:
:END:
** Abstract
%^{abstract}

I personally find template files to be easier to manage because I don’t need to reload my config every time I make a change to the template and because there is no need to escape string quotes as is required in Elisp strings, which improves readability. One has to use however the standard Org-capture template %^{placeholders} because the Org-roam-style ${placeholders} are currently not supported in template files.

You can explore this approach to further customize the output and, for example, strip author names to your liking with a custom function:

... %(my-process-author-field %^{author}) ...

As a response to the orginal topic starter’s request, ORB has also been supporting a custom “get value” function for a while now. This can be plugged in into orb-bibtex-entry-get-value-function. You can then implement your own “get value” function which would recognize “virtual” fields such as author-year but to my opinion this is an overkill. Although the template string or file would look neater with %^{author-year} than with %^{author} '%(substring "%^{year}" 2 4) but I’m not sure this small gain in brevity would justify the time needed to basically reimplement an existing function.

Perfect, thank you. Now, using %(car (split-string \"${author}\" \",\")), I am able to get the “Shevchuk '12” string I was looking for, and I’ve abandoned the overkill approach. I imagine my code will break for some paper’s author, and at that time I will write more elisp to handle it. :slight_smile:

1 Like

Great!

You can also wrap all these s-exps into a function like this:

(defun my-author-year (author year)
  (let ((author-formatted (car (split-string author ",")))
        (year-formatted (substring year 2 4)))
   (format "%s '%s" author-formatted year-formatted)))

(setq org-roam-capture-templates
  ...
  "#+TITLE: %(my-author-year \"${author}\" \"${year}\") - ${title}"
  ...)

That’s a reasonable approach to my mind. At some point it could probably be refactored into a custom “get value” function.

In fact, I wasn’t happy with the bibtex-completion-get-value’s handling of author names, or rather an absence thereof, from the very beginning of ORB. This function was designed by a different package for a different purpose, namely displaying a list of authors in Helm-bibtex interface, and it was sufficient for that purpose. It’s definitely lacking any fine grained customization options for extraction and manipulation of data as in taking notes. I always wanted a more flexible and full-featured “get value” function for ORB, there is even a reminder comment somewhere in org-roam-bibtex.el about that, but it never made onto the top of the features list. So any contributions in this direction are more than welcome. I’m not talking the ${author-year} keyword - that’s quite personalized - but rather a collection of functions that could do some additional processing on individual fields like extracting “the second author’s initials” or “the last three words of the title”. This has partially been implemented in ORB’s autokey functionality but the subroutines are rather unsophisticated and somewhat clumsy, and cannot be readily used in templates. Also, your comment

made me think about a framework which would allow for creation of such custom user-defined keywords. The general field extraction functions would still be at the backend of such a framework and could be used elsewhere for processing BibTeX data.

I was reading through the org-roam-node.el source today, and came across this snippet, that reminded me of this thread:

       (pcase-let* ((`(,field-name ,field-width) (split-string field ":"))
                    (getter (intern (concat "org-roam-node-" field-name)))
                    (field-value (funcall getter node)))

Is there some way to use this “org-roam-node-” + field-name approach to simplify the solution we talked about upthread?

e.g. could i create org-roam-node-gpc-author and org-roam-node-gpc-year and put that code above into functions like those?