Orb-process-file-field not working in orb-templates

Hi, I am trying to set up orb-templates inspired by zaeph’s init file like so:

  (setq orb-templates
	`(
	  ("n" "ref+noter" plain
	   (function org-roam-capture--get-point)
	   ""
	   :file-name "bib-${citekey}"
	   :head ,(s-join "\n"
			  (list
			   (concat "#+title: "
				   orb-title-format)
			   "#+roam_key: ${ref}"
			   ""
			   "* Notes :noter:"
			   ":PROPERTIES:"
			   ":NOTER_DOCUMENT: %(orb-process-file-field \"${citekey}\")"
			   ":NOTER_PAGE:"
			   ":END:")))))

${citekey} in :file-name and ${ref} in #+roam_key work well, but I get nothing from orb-process-file-field .

My bibtex file is referenced like so (from within org-ref 's use-package declaration):

  (setq reftex-default-bibliography '("~/pdf/bibliography.bib")
	org-ref-default-bibliography '("~/pdf/bibliography.bib")
	org-ref-pdf-directory "~/pdf/"
	;; for helm-bibtex
	bibtex-completion-bibliography "~/pdf/bibliography.bib"))

The bibtex entry I am trying this on for test purposes is:

@Book{erbacher15,
  author =	 {Erbacher, Christian},
  title =	 {Formen des Klärens. Literarisch-philosophische Darstellungsmittel in Wittgensteins Schriften.},
  year =	 2015,
  publisher =	 {Mentis},
  address =	 {Münster},
  isbn =	 9783957430014,
  file = {~/pdf/sci/Erbacher-Formen_des_Klaerens-2015.pdf}
}

What am I doing wrong? Do I need to set anything else?

@upgrd You should also set bibtex-completion-library-path to the path where you store your files, e.g. ~/pdf. Org Roam Bibtex uses bibtex-completion library to do all the BibTeX things. Note also that since recently, you can use

:NOTER_DOCUMENT: ${file}

instead of

:NOTER_DOCUMENT: %(orb-process-file-field \"${citekey}\")
1 Like

@mshevchuk: Thank you very much, I set bibtex-completion-library-path and it works now! ${file} is also good to know, thx!

1 Like

@upgrd, I’m glad it has worked!

I re-open this discussion, since I noted (along with other users) one thing which is either a regression or a change in orb-process-file-field recently. The function has been almost entirely rewritten recently, and since then, my previous capture templates are not working anymore.

I have the following in my init file:

  (setq orb-preformat-keywords
        '("citekey" "title" "url" "author-or-editor" "keywords" "file")
        orb-process-file-field t
        orb-process-file-keyword t
        orb-file-field-extensions "pdf")

 (setq orb-templates
        '(("r" "ref" plain (function org-roam-capture--get-point)
           ""
           :file-name "${citekey}"
           :head "#+TITLE: ${citekey}\n#+SUBTITLE: ${title}\n#+ROAM_KEY: ${ref}

- tags ::
- keywords :: ${keywords}

\n* Notes\n  :PROPERTIES:\n  :Custom_ID: ${citekey}\n  :URL: ${url}\n  :AUTHOR: ${author-or-editor}\n  :NOTER_DOCUMENT: %(orb-process-file-field \"${citekey}\")\n  :NOTER_PAGE: \n  :END:\n\n"

           :unnarrowed t))))

and the part %(orb-process-file-field \"${citekey}\") is now left empty by the capture template. Also, executing manually a command like:

(orb-process-file-field "nikita2019_QuantitativeSexEstimation")

on a proper bibtex entry returns nil.

The variable bibtex-completion-library-path is correctly defined, and nothing has changed in my init file; but this issue appeared roughly with commit 039eed6. Am I missing something?

Thanks!

1 Like

@fsantos Yes, with this commit your config has become broken. orb-file-field-extenstions may not be a string anymore, only a list. However, this change is explicitly stated in the commit message and the variable’s docstring was updated accordingly.

Note also that there is no such variable orb-process-file-field anymore and since quite a long time one can use for convenience the ${file} wildcard instead of the more verbose %(orb-process-file-field \"${citekey}\")

I will update the function slightly, so that it issues a warning instead of failing silently when something’s gone wrong.

1 Like

Thanks, everything is okay now! :cowboy_hat_face:

2 Likes

@mshevchuk , sadly I am also running into the same issue. I cannot get the noter document property to have the file name.

Here is the relevant part of my config.

  :hook (org-roam-mode . org-roam-bibtex-mode)
  :bind (:map org-mode-map
         (("C-c n a" . orb-note-actions))))

(setq
      bibtex-completion-pdf-field "file"
      bibtex-completion-bibliography
      '("~/Dropbox/Org/references/library.bib")
      bibtex-completion-library-path '("~/Dropbox/Zotero/")
      )
(setq orb-preformat-keywords
        '("citekey" "title" "url" "author-or-editor" "keywords" "file")
        orb-process-file-field t
        orb-process-file-keyword t)
(setq orb-templates
      '(("r" "ref" plain (function org-roam-capture--get-point) ""
         :file-name "${citekey}"
         :head "#+TITLE: ${citekey}: ${title}\n#+ROAM_KEY: ${ref}\n" ; <--
         :unnarrowed t)))


(setq orb-templates
      '(("n" "ref+noter" plain (function org-roam-capture--get-point)
         ""
         :file-name "${citekey}"
         :head "#+TITLE: ${citekey} \n#+SUBTITLE: ${title}\n#+ROAM_KEY: ${ref} \n #+ROAM_TAGS:

- tags ::
- keywords :: ${keywords}
\* ${title}
:PROPERTIES:
:Custom_ID: ${citekey}
:URL: ${url}
:AUTHOR: ${author-or-editor}
:NOTER_DOCUMENT: ${file}
:NOTER_PAGE:
:END:")))

@mancub

  1. Check the value of orb-file-field-extensions, it should be a list of extensions as strings without a dot. Only files with extensions from the list will be considered for extending the ${file} keyword. The default value is '("pdf"), which means that only PDF files will be considered.

  2. Make sure the BibTeX entry for which you are creating a note actually has a file associated with it. Basically, whatever is picked up by helm-bibtex or ivy-bibtex will be used by ORB, except that additional filtering based on extension is applied as discussed above. The easiest way to find this out is to run helm- or ivy-bibtex and check whether the file indicator is present - typically a single leftmost character in the list of candidates. Also try to open the file from helm- or ivy-bibtex.

  3. Upgrade to the most recent version of ORB (yesterday’s commit), run the capture and check if *Warnings* buffer exists and if there are any ORB warnings.

@mshevchuk Thank you for the detailed directions, I really appreciate it.

  1. Did this - no change, still no filename in the Noter Document field.

here is the updated config.

      bibtex-completion-pdf-field "file"
      bibtex-completion-bibliography
      '("~/Dropbox/Org/references/library.bib")
      bibtex-completion-library-path '("~/Dropbox/Zotero/")
      )
(setq orb-preformat-keywords
        '("citekey" "title" "url" "author-or-editor" "keywords" "file")
        orb-process-file-field t
        orb-process-file-keyword t
        orb-file-field-extensions '("pdf"))



(setq orb-templates
      '(("n" "ref+noter" plain (function org-roam-capture--get-point)
         ""
         :file-name "${citekey}"
         :head "#+TITLE: ${citekey} \n#+SUBTITLE: ${title}\n#+ROAM_KEY: ${ref} \n #+ROAM_TAGS:
- tags ::
- keywords :: ${keywords}
\* ${title}
:PROPERTIES:
:Custom_ID: ${citekey}
:URL: ${url}
:AUTHOR: ${author-or-editor}
:NOTER_DOCUMENT: ${file}
:NOTER_PAGE:
:END:"))) 
  1. I do see the indicator in Helm that the file exists. However when I try to open it (Open PDF, URL or DOI), it does not open the file, instead it shows me the directory structure in the root folder. I think this might the culprit, something is not getting configured correctly.

  2. EDIT 2: I ended up adding all the modules one by one into the config, and as such, I am now seeing warnings saying that those references are not valid references.

  3. Could the fact that I am using the native comp branch have something to do with these errors? I tested this in emacs 27 as well, and had the same issue.

  4. Edit: Also wanted to add that the orb-process-file-field variable is not being recognized as a variable in Doom for some reason. Screenshot_2021-03-31_10-52-44

okay, did an absolute fresh install of emacs and followed your advice. ORB is now behaving as normal.

1 Like

@mancub

Great!

Could be.

This variable was renamed to orb-process-file-keyword, so use this one instead.

So the issue was likely caused by bibtex-completion (backend for helm-bibtex and ivy-bibtex), which started misbehaving because of stale compiled files after an upgrade. I don’t know whether this was directly related to the native-comp branch. Just for your information, I cannot support different Emacs versions currently, although I have remote plans to do so. The only supported version is whatever the current stable is (27.1 as of today).

1 Like

@mshevchuk to add to this conversation -

I did some testing over the weekend in VMs. The error I had reported is specifically present in the native comp branch. (Tested in Manjaro (AUR), Fedora (COPR), and manually compiled in Ubuntu)

It is not present in stable 27.1 (tested fedora, and Manjaro).

Hope that provides clarity to whoever stumbles upon this thread!

1 Like