Org-roam-bibtex for Org-Roam v2

Hi, thank you for your interest!

There is a growing amount of evidence that the current documentation of Org-roam-bibtex does not explain well what the package actually does and what it is useful for. So I’ll reply in a wordy way that can be further reused in the updated docs.

First of all what Org-roam-bibtex is not:

  1. It is not a bibliography manager.
  2. It is not a user interface for a bibliography manager.
  3. It is not a citation processor.
  4. It is not a note taking system.

A little bit of history. Two years ago we had:

  1. A bibliography management library - Bibtex-comletion
  2. Helm and Ivy interfaces to that library - Helm-bibtex and Ivy-bibtex
  3. A citation processor (and more) - Org-ref
  4. A yet another note-taking system based on Emacs/Org-mode - Org-roam - it was just born and became a shining star just within a few months.

The problem:

  1. Helm/Ivy-bibtex is doing a great job at viewing/manipulating BibTeX references. It has fairly basic note-taking capabilities to create bibliographic notes associated with bibliography entries.
  2. Org-ref is doing a great job at citation processing in Org-mode documents. It also has its own fairly basic note-taking capabilities. It uses Bibtex-completion and Helm/Ivy-bibtex to interact with BibTeX files.
  3. Org-roam is doing a great job at note-taking. It recognizes Org-ref citations as a special type of links. It has no clue about BibTeX.
  4. Can we integrate Org-roam, Org-ref and Helm/Ivy-bibtex in such a way that all packages would do together what they can do best?

A solution. This is when Org-roam-bibtex comes in. What Org-roam-bibtex is:

  1. A small extension for Org-roam that enables automatic expansion of special placeholders in org-roam-capture-templates populating them with bibliographic information. It uses Bibtex-completion to interact with BibTeX files and feed that information to Org-roam.
  2. A connector between Org-roam on the one side and Helm/Ivy-bibtex and Org-ref on the other side. It tricks the latter two packages into abandoning their native note-taking capabilities and using Org-roam instead.

A year ago or so we got Citar - this package started as an interface library for Bibtex-completion. Compared to more complex external Helm and Ivy packages, it is based on the native lightweight Emacs completion library - completing-read. It grew up quickly to integrate with Embark, Vertico and co, which share a similar philosophy. It also was the first to provide an extensive support for Org-cite - a native Org library, which cut off a piece of the Org-ref’s pie. In fact, we should thank for that the author of Citar, @bruce, who was very active on the Org-mode mailing list in promoting the development of the native Org-cite syntax. At last! Moreover, Citar also provides integration with Org-roam for note management, somewhat overlapping here with Org-roam-bibtex in functionality.

So now actually we have two toolsets sharing a lot in terms of functionality: Org-ref/Helm-bibtex and Org-cite/Citar. The good news are Org-roam-bibtex supports both! You are free to choose whatever you like and find suitable for your needs!

As the first part of its name implies, Org-roam-bibtex is Org-roam-centric. The second part means it’s aim is to support BibTeX-related workflows remaining at the same time agnostic to the user’s preference of bibliography/citation packages.

3 Likes

As @mshevchuk explains, yes.

Since you’re a doom user, maybe experiment a bit with what the biblio module provides (org-cite only) and conjunction with the +roam2 org flag.

If you don’t like how the open notes functionality works there, and you don’t want to or can’t write your own function for that (which is an option), then give orb a try. The citar README includes instructions on how to replace the default notes function with the orb equivalent.

None of these packages really conflict. The only thing I believe that does conflict is org-cite and at least one of the org-ref syntaxes.

2 Likes

There is no need to do that. ORB defines and sets the function automatically in org-roam-bibtex-mode when Citar is installed.

Within one document, yes. They do not really conflict, just the exported citations will be messed up. And even that will depend on the citation style and how the bibliography is sorted, I guess. For numerical styles definitely.

1 Like

Oh, cool; I need to update my README then!

1 Like

Thanks @bruce and @mshevchuk for your works and these very clear explanations.

I install biblio module (actually don’t using Orb), then configure citar and everything works well for my use case : citations work inside and outside of org-roam folder. Perfect.

Now the overlapp between Citar and Orb is not clear from the user perspective of emacs lambda user like me. I suppose there are strong differences in the depp capability of each lib, but some words about that in Readme could be cool :slight_smile:

I read the note management to better understand the differences.

Thanks again

Part of the challenge is the broader emacs bibliographic ecosystem of packages is rather confusing. We tried to summarize that here (though perhaps at some point we need a diagram or some other way to visualize it!):

I think for just citar vs orb, citar does not depend on org-roam at all; it just supports by default org and org-cite. So it works consistently regardless of whether you use org-roam.

But the default citar note function will, if org-roam is loaded, add some org-roam specific things like ids and the ROAM_REFS property, so that org-roam linking will correctly work.

Citar also does not depend on bibtex-completion anymore.

2 Likes

Citar and ORB are as comparable as flowers and beehives. Clearly, both have something to do with bees and honey, but well. Citar is a player in its own right. It is a full-fledged interface to BibTeX databases allowing you to view and interact with .bib files, insert citations in Org/markdown format. Citar supplies custom processors for Org-cite citations, greatly extending the built-in Org functionality. It is also aware of Org-roam, allowing one to use it as a note repository, but it does not focus on Org-roam. Citar is better comparable to Helm/Ivy-bibtex.

ORB can do nothing of that. It’s an Org-roam extension. It is not imaginable without Org-roam. Its purpose is 1) to make org-roam-capture-templates recognize BibTeX keywords – and no other package can do that, and 2) to expose Org-roam as a note management facility to BibTeX frontends like Citar or Helm-bibtex. Citar is also aware of Org-roam, unlike Helm-bibtex or Org-ref. This I called an overlap: if you don’t care about expansion of BibTeX keywords in org-roam-capture-templates (reason 1. to use ORB), then the only other potential reason to use ORB – to make Citar recognize Org-roam as its note taking backend – is already included with Citar.

Still, ORB provides several minor utility commands that one may find convenient when working with Org-roam buffers: orb-note-actions and orb-insert-link. I believe because of these interactive commands people think ORB provides capabilities similar to Citar or Helm-bibtex. That is not correct: orb-note-actions works only in Org-roam buffers, by design; orb-insert-link is a sibling to org-roam-node-insert enhanced with a capability to insert citations à la citar-insert-citation or the respective helm-bibtex action. Moreover, both these commands use Bibtex-completion as a backend, so ORB is not independent here as well.

2 Likes

Hi everyone,

I am giving orb a test run and wondering about the roam ref format. The manual suggests that this is mostly cosmetic so I was wondering if I could just use the citekey from the bibtex entry as the roam ref? Is there a way to set this so that the Roam Ref is updated at the point of capture? My workflow doesn’t involve either org-ref or org-cite (I use ivy-bibtex for citations into org-mode directly) and I haven’t configured either. Thanks!

Hi @aflatoon ,

yes, you can choose to use either Org-ref v2 cite:citekey, Org-ref v3 cite:&citekey or Org-cite @citekey refs. The only difference between them is that Org-ref-styled formats are technically links recognized as such even when they are in property drawers, which means you can activate them. Org-cite format as Roam Ref is just a piece of text.

You must choose from one of the three formats for Roam Ref because otherwise Org-roam will complain. A valid Roam Ref is an Org-mode link (any, not just citation) or an Org-cite citation. It’s decided by Org-roam, not ORB.

Is there a way to set this so that the Roam Ref is updated at the point of capture?

Roam Ref is set automatically when you run ivy-bibtex and select Edit notes action to start an Org-roam capture process (this is what ORB is responsible for). Be sure to have org-roam-bibtex-mode on. If you simply run org-roam-capture or org-roam-node-find to initiate a capture and select an ORB-enabled template (with BibTeX keyword wildcards) then you’ll be prompted to insert it interactively.

You don’t need to.

1 Like

Thank you very much for the prompt response, that is helpful.

1 Like

Hi there,

I continue to experiment with Citar of @bruce / orb of @mshevchuk and i’m lost in configuration, i’m facing issue to create note from reference.

I’m using the very lastest version of Doom emacs with +roam2 and +biblio module. Also i’m using Vertico.

(unpin! org-roam)

Here a working config for citar without orb :


(after! citar
  (map! :map org-mode-map
        :desc "Insert citation" "C-c b" #'citar-insert-citation)
  (setq citar-bibliography '("~/Notes/Bibliography/main.bib")
        citar-notes-paths '("/home/reyman/Notes/2ndBrain")
        org-cite-insert-processor 'citar
        org-cite-follow-processor 'citar
        org-cite-activate-processor 'citar
        citar-at-point-function 'embark-act
        citar-file-note-org-include '(org-id org-roam-ref)
        citar-symbols
        `((file ,(all-the-icons-faicon "file-o" :face 'all-the-icons-green :v-adjust -0.1) . " ")
          (note ,(all-the-icons-material "speaker_notes" :face 'all-the-icons-blue :v-adjust -0.3) . " ")
          (link ,(all-the-icons-octicon "link" :face 'all-the-icons-orange :v-adjust 0.01) . " "))
        citar-symbol-separator "  ")
  (advice-add #'completing-read-multiple :override #'consult-completing-read-multiple))

Creating note from embark-act works

When i add ORB :

(package! org-roam-bibtex
  :recipe (:host github :repo "org-roam/org-roam-bibtex"))

Modifyng citar by adding this :

(after! citar
  (map! :map org-mode-map
        :desc "Insert citation" "C-c b" #'citar-insert-citation)
...
citar-open-note-function 'orb-citar-edit-note
...
)

With ORB config :


(use-package! org-roam-bibtex
  :after org-roam
  :hook (org-roam-mode . org-roam-bibtex-mode)
  :config
  (setq orb-preformat-keywords
        '("citekey" "title" "url" "author-or-editor" "keywords" "file"))
  (setq orb-roam-ref-format 'org-cite)
  (setq orb-process-file-keyword t
        orb-file-field-extensions '(".pdf", ".djvu")
        orb-insert-follow-link t))

Using this configuration i have these Warning, and adding note of existing ref doesn’t work :

Warning (:warning): Bibtex-completion couldn't find entry with key "texbook". Disable showing Disable logging
Warning (:warning): ORB: texbook :Could not find the BibTeX entry Disable showing Disable logging

Any idea ?

Your ORB config is outdated, it comes from the Org-roam v1 times. Since org-roam-mode is a major mode in Org-roam v2 used in the Org-roam backlinks buffer, loading of ORB will be triggered when you first open the Org-roam buffer. A minimal configuration for ORB compatible with Org-roam v2 is as follows:

(use-package! org-roam-bibtex
  :after org-roam)

If you use Org-ref, you can put (require 'org-ref) in the :config section to ensure it is loaded together with ORB.

To automatically turn on org-roam-bibtex-mode, put (org-roam-bibtex-mode +1) somewhere in your init file, for example (after! org-roam (org-roam-bibtex-mode +1)). You can call this command manually otherwise.

You don’t need to set citar-open-note-function yourself. Enabling org-roam-bibtex-mode will take care of that.

Also, ORB requires bibtex-completion-bibliography to be set. For example, (setq bibtex-completion-bibliography citar-bibliography).

1 Like

Thanks for clarificatiohn @mshevchuk, not easy to get the latest version when you check people dotfile :wink:

I see @bruce work on mainstream integration here https://github.com/hlissner/doom-emacs/pull/6077/commits/fdc548ff74d2187821d22e15585bb4a54b7977cb so i suppose my problem disapear in few time :slight_smile:

If you’re using (as it appears so) Doom, then all you need to do is choose the “vertico” and “biblio” modules, with the “+roam2” flag on “org”, and then set the citar-notes-paths variable.

The default citar note function will work fine with that. My suggestion is you start with this default.

As @mshevchuk said, if you install and configure ORB, that will take over note handling with citar.

1 Like