Org-roam-bibtex for Org-Roam v2

jethro · May 27, 2021, 5:51am

I agree for the most part, do you have a strategy to provide default templates then? org-roam shouldn’t need to be aware of other community-built packages.

mshevchuk · May 27, 2021, 7:44am

So the main idea is to introduce the :type keyword into org-roam-capture-templates and define the logic behind it. There will need to be a function to filter the list and return a list of templates appropriate for the current context.^[1]

In the simplest case, the keyword :type is not strictly required, i.e. its value may be nil. In this case the default org-roam-capture-template does not change at all:

(setq org-roam-capture-templates
  '(("d" "default" plain "%?" :if-new
     (file+head "%<%Y%m%d%H%M%S>-${slug}.org" "#+title: ${title}\n")
     :unnarrowed t))

The filter function, call it org-roam-capture-get-templates, will return all templates whose :type is nil. So org-roam-capture- becomes:

(defun org-roam-capture- (&key goto keys node info props type)
   ...
 (org-capture-templates
          (mapcar (lambda (template)
                    (org-roam-capture--convert-template template props))
                  (org-roam-capture-get-templates type)))
...)

The value of :type may be a symbol, in which case org-roam-capture-get-templates returns only templates whose :type matches this symbol.

So for dailies org-roam-capture-templates becomes:

(setq org-roam-capture-templates
  '(;; find node template
    ("d" "default" plain "%?" :if-new
     (file+head "%<%Y%m%d%H%M%S>-${slug}.org" "#+title: ${title}\n")
     :unnarrowed t)

    ;; dailies template
    ("d" "default" entry "* %?" :if-new
      (file+head "path/to/daily/%<%Y-%m-%d>.org" #+title: %<%Y-%m-%d>\n")
      :type dailies))

The dailies capture function then becomes:

(defun org-roam-dailies--capture ...
   ...
  (org-roam-capture- :goto (when goto '(4))
                     :node (org-roam-node-create)
                     :type 'dailies
                     :props (list :default-time time)))

Note that selectors may be the same for different types since filtering is done before org-capture is called.

Similar adjustments will be required for org-roam-protocol ref templates.

External packages like Org-roam-bibtex will define their own types, but Org-roam does not need to be aware of them: org-roam-capture-get-templates just filters the list according to the supplied argument. It does not care what this argument actually is.

In fact, Org-roam-bibtex does not require these upstream changes to implement such a filtering as it can be done by Org-roam-bibtex itself. It will, however, benefit from the :type keyword mentioned in the docstring and of course it’s cleaner and more convenient for it to use the exposed API rather than to introduce hacks.

There can be a more elaborate logic behind the :type keyword. For example, the meaning of nil could be changed to match any context and :type could also be a composite list of several contexts where a particular template should be available. But I’m not sure if this is really necessary because the context workflows do not seem to intersect.

Footnotes

¹ This is obviously a take on org-capture-templates-context but rather than having a separate variable whose value matches a template selector from the list, the template itself specifies the context in which it should be made available. The meaning of context is also different here. While in org-capture filtering can be done depending on file type or mode in which org-capture was evoked, here filtering is done depending on who called the template - a simple org-roam find-node function, org-roam protocol, org-roam dailies, org-roam-bibtex, etc . The keyword :context can also be considered instead of :type, but the latter may be preferable so as not to cause ambiguity with org-capture contexts.

mshevchuk · May 27, 2021, 8:18am

Ok, I perhaps misunderstood what you meant by “a strategy to provide default templates”. The simplest way would be to put (defcustom org-roam-capture-templates ...) into org-roam.el and define the default templates there. Alternatively, org-roam-dailies may push its default template onto org-roam-capture-templates upon loading. Conditional pushing may also be implemented depending on whether a dailies template already exists on the list, e.g. it was set by the user with setq.

daniel · September 6, 2021, 7:35pm

I have an init.el from scratch that uses straight.el. Do you have some hints how the org-roam-bibtex-mode could be automatically activated once org-roam is loaded?

nobiot · September 6, 2021, 8:39pm

This, perhaps?

(use-package org-roam-bibtex
  :after org-roam
  :config
  (require 'org-ref)) ; optional: if Org Ref is not loaded anywhere else, load it here

If you don’t use use-package (like me), you might want to try with-after-eval-load

mshevchuk · September 7, 2021, 6:13am

@daniel

(use-package org-roam
   ...
   :config
   (org-roam-bibtex-mode +1)
   ...)

The package itself should be loaded as provided by @nobiot. And as he said, if you don’t use use-package, then (with-eval-after-load 'org-roam (org-roam-bibtex-mode +1)).

daniel · September 7, 2021, 6:41am

Either way results in a Query timeout error. Maybe something like a dead-lock?

@nobiot I followed the instructions but the system does not behave as expected. Maybe the timeout error is somehow related to the issue, that it does not work as proposed in the instructions?

mshevchuk · September 7, 2021, 7:32am

That’s something unusual. Could it be SQL-related? Can you post the exact error, enabling debug through M-x toggle-debug-on-error might also be helpful.

nobiot · September 7, 2021, 8:35am

Unusual; agree with @mshevchuk.
I would even ask if Org-roam without ORB is working (?)

daniel · September 20, 2021, 3:22pm

This is the debugger error. It looks like that it can’t query the org-roam.db database. Probably some deadlock scenario? It only appears when I try to load org-roam-bibtex-mode in the org-roam :config section.

When I remove the org-roam-bibtex-mode statement Emacs loads fine. Once Emacs loaded, I can enable M-x org-roam-bibtex-mode without any issue.

Debugger entered--Lisp error: (emacsql-timeout "Query timed out" 30)
  signal(emacsql-timeout ("Query timed out" 30))
  #f(compiled-function (connection &optional timeout) "Block until CONNECTION is waiting for further input." #<bytecode 0x3861c79>)(#<emacsql-sqlite-connection emacsql-sqlite-connection-3e5175c>)
  apply(#f(compiled-function (connection &optional timeout) "Block until CONNECTION is waiting for further input." #<bytecode 0x3861c79>) #<emacsql-sqlite-connection emacsql-sqlite-connection-3e5175c> nil)
  emacsql-wait(#<emacsql-sqlite-connection emacsql-sqlite-connection-3e5175c>)
  #f(compiled-function (connection sql &rest args) "Send SQL s-expression to CONNECTION and return the results." #<bytecode 0x4157461>)(#<emacsql-sqlite-connection emacsql-sqlite-connection-3e5175c> [:select [file hash] :from files])
  apply(#f(compiled-function (connection sql &rest args) "Send SQL s-expression to CONNECTION and return the results." #<bytecode 0x4157461>) #<emacsql-sqlite-connection emacsql-sqlite-connection-3e5175c> [:select [file hash] :from files])
  emacsql(#<emacsql-sqlite-connection emacsql-sqlite-connection-3e5175c> [:select [file hash] :from files])
  apply(emacsql #<emacsql-sqlite-connection emacsql-sqlite-connection-3e5175c> [:select [file hash] :from files] nil)
  org-roam-db-query([:select [file hash] :from files])
  (let ((current-files (org-roam-db-query [:select [file hash] :from files])) (ht (make-hash-table :test #'equal))) (let ((--dolist-tail-- current-files)) (while --dolist-tail-- (let ((row (car --dolist-tail--))) (puthash (car row) (car (cdr row)) ht) (setq --dolist-tail-- (cdr --dolist-tail--))))) ht)
  org-roam-db--get-current-files()
  (let* ((gc-cons-threshold org-roam-db-gc-threshold) (org-agenda-files nil) (org-roam-files (org-roam-list-files)) (current-files (org-roam-db--get-current-files)) (modified-files nil)) (let ((--dolist-tail-- org-roam-files)) (while --dolist-tail-- (let ((file (car --dolist-tail--))) (let ((contents-hash (org-roam-db--file-hash file))) (if (string= (gethash file current-files) contents-hash) nil (setq modified-files (cons file modified-files)))) (remhash file current-files) (setq --dolist-tail-- (cdr --dolist-tail--))))) (let ((emacsql--connection (org-roam-db)) (emacsql--completed nil) (emacsql--transaction-level (1+ emacsql--transaction-level)) (emacsql--result)) (unwind-protect (while (not emacsql--completed) (condition-case nil (progn (if (= 1 emacsql--transaction-level) (progn ...)) (let (...) (setq emacsql--result result) (if ... ...) (setq emacsql--completed t))) (emacsql-locked (emacsql emacsql--connection [:rollback]) (sleep-for 0.05)))) (if (and (= 1 emacsql--transaction-level) (not emacsql--completed)) (progn (emacsql emacsql--connection [:rollback])))) emacsql--result))
  org-roam-db-sync()
  (cond (enabled (add-hook 'find-file-hook #'org-roam-db-autosync--setup-file-h) (add-hook 'kill-emacs-hook #'org-roam-db--close-all) (advice-add #'rename-file :after #'org-roam-db-autosync--rename-file-a) (advice-add #'delete-file :before #'org-roam-db-autosync--delete-file-a) (org-roam-db-sync)) (t (remove-hook 'find-file-hook #'org-roam-db-autosync--setup-file-h) (remove-hook 'kill-emacs-hook #'org-roam-db--close-all) (advice-remove #'rename-file #'org-roam-db-autosync--rename-file-a) (advice-remove #'delete-file #'org-roam-db-autosync--delete-file-a) (org-roam-db--close-all) (let ((--dolist-tail-- (org-roam-buffer-list))) (while --dolist-tail-- (let ((buf (car --dolist-tail--))) (save-current-buffer (set-buffer buf) (remove-hook 'after-save-hook #'org-roam-db-autosync--try-update-on-save-h t)) (setq --dolist-tail-- (cdr --dolist-tail--)))))))
  (let ((enabled org-roam-db-autosync-mode)) (cond (enabled (add-hook 'find-file-hook #'org-roam-db-autosync--setup-file-h) (add-hook 'kill-emacs-hook #'org-roam-db--close-all) (advice-add #'rename-file :after #'org-roam-db-autosync--rename-file-a) (advice-add #'delete-file :before #'org-roam-db-autosync--delete-file-a) (org-roam-db-sync)) (t (remove-hook 'find-file-hook #'org-roam-db-autosync--setup-file-h) (remove-hook 'kill-emacs-hook #'org-roam-db--close-all) (advice-remove #'rename-file #'org-roam-db-autosync--rename-file-a) (advice-remove #'delete-file #'org-roam-db-autosync--delete-file-a) (org-roam-db--close-all) (let ((--dolist-tail-- (org-roam-buffer-list))) (while --dolist-tail-- (let ((buf ...)) (save-current-buffer (set-buffer buf) (remove-hook ... ... t)) (setq --dolist-tail-- (cdr --dolist-tail--))))))))
  (let ((last-message (current-message))) (progn (set-default 'org-roam-db-autosync-mode (if (eq arg 'toggle) (not (default-value 'org-roam-db-autosync-mode)) (> (prefix-numeric-value arg) 0)))) (let ((enabled org-roam-db-autosync-mode)) (cond (enabled (add-hook 'find-file-hook #'org-roam-db-autosync--setup-file-h) (add-hook 'kill-emacs-hook #'org-roam-db--close-all) (advice-add #'rename-file :after #'org-roam-db-autosync--rename-file-a) (advice-add #'delete-file :before #'org-roam-db-autosync--delete-file-a) (org-roam-db-sync)) (t (remove-hook 'find-file-hook #'org-roam-db-autosync--setup-file-h) (remove-hook 'kill-emacs-hook #'org-roam-db--close-all) (advice-remove #'rename-file #'org-roam-db-autosync--rename-file-a) (advice-remove #'delete-file #'org-roam-db-autosync--delete-file-a) (org-roam-db--close-all) (let ((--dolist-tail-- (org-roam-buffer-list))) (while --dolist-tail-- (let (...) (save-current-buffer ... ...) (setq --dolist-tail-- ...))))))) (run-hooks 'org-roam-db-autosync-mode-hook (if (default-value 'org-roam-db-autosync-mode) 'org-roam-db-autosync-mode-on-hook 'org-roam-db-autosync-mode-off-hook)) (if (called-interactively-p 'any) (progn (customize-mark-as-set 'org-roam-db-autosync-mode) (if (and (current-message) (not (equal last-message (current-message)))) nil (let ((local "")) (message "Org-Roam-Db-Autosync mode %sabled%s" (if (default-value ...) "en" "dis") local))))))
  org-roam-db-autosync-mode(1)
  org-roam-setup()
  run-hooks(after-init-hook delayed-warnings-hook)
  command-line()
  normal-top-level()

mshevchuk · September 21, 2021, 6:51am

So do you have org-roam-setup in the after-init-hook?

daniel · September 21, 2021, 11:11am

I would say so. It looks like this.

(use-package org-roam
...
:hook
((after-init . org-roam-setup))
....
)

mshevchuk · September 21, 2021, 1:40pm

Ok. That’s too early. You can do it in several other ways.

Manual. You don’t need Org-roam and ORB available immediately after startup. Org-roam will be loaded after you call one of the Org-roam commands. NOTE: ORB will not be available until Org-roam is initialized

(use-package org-roam
...
:commands
(org-roam-node-find    ;; calling one of these commands will initialize Org-roam and ORB
 ...)
:config
(org-roam-setup)
(org-roam-bibtex-mode +1)
...)

Org-roam and ORB will be initialized automatically when you load the first Org file.

(use-package org-roam
...
:hook
(org-load . org-roam-setup)
:config
(org-roam-bibtex-mode +1)
...)

Org-roam and ORB will be loaded upon startup. CAUTION: depending on the load order of other packages, it can mess up Org loading. Should better go at the end of config.

(use-package org-roam
...
:config
(org-roam-bibtex-mode +1)
...)

(org-roam-setup)

daniel · September 21, 2021, 2:35pm

I implemented option 2 which works perfect

The org-roam-bibtex mode is now loaded as soon as org-roam is loaded.

mshevchuk · September 22, 2021, 6:10am

Great!

FilTower · January 5, 2022, 6:53pm

Hi @mshevchuk! I’ve just updated ORB to version 0.6.2 (as well as Org Roam to the latest commit) and I have encountered the following issue. When I open my projects.org file and I run helm-bibtex, I get a “No node at point” message and nothing else happens whereas previously I would simply get the list of my Bibtex entries.

The message/issue does not appear if I run helm-bibtex from an Org Roam node, and I can access my references and the rest of my bibliographical notes without problems. So, the issue seems to be related with the fact that my projects.org is not an Org Roam node (no ID attached).

In my config file, first I load helm, then some lines below I require org-ref-helm, org-roam, and eventually org-roam-bibtex. Also, I tell Emacs to load ORB after Org Roam, using (with-eval-after-load 'org-roam (org-roam-bibtex-mode +1)). Basically, when I start Emacs everything gets loaded.

Might that behaviour be connected with one of the changes in ORB v.0.6.2, specifically with "Activating org-roam-bibtex-mode adds orb-get-node-citekey to bibtex-completion-key-at-point-functions" (first entry under ‘Changed’ in the change log of ORB)?

Or do I still have a far from ideal config file that is messing things up?

mshevchuk · January 5, 2022, 10:12pm

Hi @FilTower. That is precisely the reason. Sorry, I pushed it barely tested. Will fix it soon.

mshevchuk · January 5, 2022, 11:04pm

Should be fixed now in 070a7a7.

FilTower · January 6, 2022, 10:09pm

Hi @mshevchuk, thanks for the quick reply and fixing the issue straight away!

reyman · January 7, 2022, 8:45pm

Hi there,

Thanks for all your works on this package, i’m ready to use it but before i do some state of the art … and after 2 hours to try to understand, i’m totally lost.

As org-roam and doom user that use citation inside and outside of roam, i suppose there is three way to follow actually :

continuing with org-ref
starting to use org-cite with new module biblio that need only citar to works : https://www.reddit.com/r/DoomEmacs/comments/rxmr6d/setting_up_org_roam_bibtex/
starting to use org-cite with org-roam-bibtex

But my question is, org-roam-bibtex, that contain lot of functionality, is it compatible with citar with module biblio currently developped ?

Edit : I finally found info here and it seems org-roam-bibtex will be integrated into biblio module of Doom, if i understand well : Add org-ref package for citations and bibliography · hlissner/doom-emacs@098bb1b · GitHub

Topic		Replies	Views
ORB: helm-bibtex -> `open notes` won't create new note Troubleshooting	7	1914	September 11, 2020
ORB always prompting for template capture, even if the node already exists Troubleshooting	1	254	March 4, 2023
An idea for a roam-based bibliographic manager Development	11	1004	September 3, 2021
Minimal setup for helm-bibtex, org-roam v2, org-roam-bibtex Requests	7	2838	November 26, 2022
Best way to programatically create new bib note in org-roam v2? Development	11	757	August 21, 2021

Org-roam-bibtex for Org-Roam v2

Footnotes

Related topics