NOTE: Some of these codes and texts look crooked in this website, this is a visual bug - if you copy the codes and paste in your text editor they will automatically resolve themselves!
Introduction
In my attempt to setup org-roam-bibtex, I quickly found out that there was no “good” guide that could quickly help me set up a bibliography system - with sane defaults - and get out of my way so that I can get on with my life; many guides that exist on the internet, are awfully outdated use deprecated variables that are not in existence since 2021; the system is also confusing to newcomers since it requires multiple packages and the user is expected to go through several manuals before the system could be set up. All in all, it took me 2 days reading through several of these manuals to set up the system. In this guide, I wish to elucidate the distilled essence of the whole, so that a new user may set up a working system in a matter of minutes (not days) and get on with the actual research part without having to read through the manuals (unless they wish to customise or fall in a problem).
We will set up org-roam-bibtex with two backends - org-ref and helm-bibtex.
1.) helm-bibtex is a front end to bibtex-completion this allows us to work with our .bib bibtex file globally, we get a full buffer when we call helm-bibtex from where we can manage our citations easily, we may add new entries, edit entries, add citations to our documents, and many more from the comfort of a dedicated buffer.
Note: ivy-bibtex is an alternate to this which is more minimal, but we will not be setting this up here!
2.) org-ref will allow us to work with citations of the form cite:&citekey and allow us to easily make pdfs, it also gives us a new export backend C-c C-e r where we can also export to htmls with these citations, further it will give us a small window on the bottom of the buffer when doing <Ret> on the citations which will make working with citations on the buffer very easy.
3.) Finally org-roam-bibtex will allow us to integrate our bibliography system with our org-roam system.
4.) We will also set up org-noter which will allow us to work with pdf annotations on an .org file and integrate it with the system.
Basic Setup
The system requires,
i. A .bib file where we will store our entries (./references/master.bib in this guide)
ii. A folder to keep our documents related to an entry - being pdf(s), such as the paper related to the entry and so on (./references/documents/ in this guide)
iii. A folder to keep our notes related to the entry and the document(s) (./references/notes/ in this guide)
In this guide it is assumed that the value of org-roam-directory is ~/roam for convenience
╭── documents
│
roam ───────── references ──────┼── notes ───────── notes-template.org*
│
╰── master.bib
*for org-noter integration, explained below
FIG_01: Diagram explaining the directory and file structure used
Installation
Ensure (or install the following from Melpa: M-x package-install <Ret>):
*pdf-tools makes the experience of working with pdfs inside emacs better.
Configuration
Set the following in your init file:
We do not need to configure org-ref it shares variables with helm-bibtex (bibtex-completion) since V3 2021!
helm-bibtex (bibtex-completion)
;; IMP: Ensure 'latexmk' installed as a system package!
;; see also: http://www.jonathanleroux.org/bibtex-mode.html
(setq bibtex-completion-bibliography '("~/roam/references/master.bib")) ; location of .bib file containing bibliography entries
(setq bibtex-completion-find-additional-pdfs t) ; support for multiple pdfs for one %citekey
(setq bibtex-completion-pdf-field "File") ; in bib entry, file = {/path/to/file.pdf} could be set to locate the accompanying file
;; for multiple files use, file = {:/path/to/file0.pdf:PDF;:/path/to/file1.pdf:PDF}
(setq bibtex-completion-library-path '("~/roam/references/documents/")) ; in this dir, %citekey-name(s).pdf would automatically attach pdf(s) to %citekey
;; if only !exist "file" field in bib entry
(setq bibtex-completion-notes-path "~/roam/references/notes/") ; dir to keep notes for the pdfs
;; BEGIN: Change insert citation (<f3>) behaviour of helm-bibtex for org-mode
(defun custom/bibtex-completion-format-citation-org (keys)
"Custom cite definition for org-mode"
(s-join ", "
(--map (format "cite:&%s" it) keys)))
(setq bibtex-completion-format-citation-functions
'((org-mode . custom/bibtex-completion-format-citation-org)
(latex-mode . bibtex-completion-format-citation-cite)
(markdown-mode . bibtex-completion-format-citation-pandoc-citeproc)
(python-mode . bibtex-completion-format-citation-sphinxcontrib-bibtex)
(rst-mode . bibtex-completion-format-citation-sphinxcontrib-bibtex)
(default . bibtex-completion-format-citation-default))
)
;; END: Change insert citation (<f3>) behaviour of helm-bibtex for org-mode
(setq bibtex-autokey-year-length 4 ; customisations for 'bibtex-generate-autokey'
bibtex-autokey-name-year-separator "-" ; press C-c C-c (bibtex-clean-entry) on a bib entry w/o %citekey
bibtex-autokey-year-title-separator "-" ; to automatically insert a %citekey based on meta data
bibtex-autokey-titleword-separator "-" ; use M-x crossref-add-bibtex-entry <ret>: to add an entry from
bibtex-autokey-titlewords 2 ; https://www.crossref.org/
bibtex-autokey-titlewords-stretch 1
bibtex-autokey-titleword-length 5)
org-roam-bibtex
If we use org-roam for .org files exclusively, we can just set this variable and forget about it. We do not need to enable org-roam-bibtex-mode seperately when working with .org files of org-roam, the mode basically toggles this variable!
;(setq bibtex-completion-edit-notes-function 'bibtex-completion-edit-notes-default) ; default to org-ref for notes
(setq bibtex-completion-edit-notes-function 'orb-bibtex-completion-edit-note) ; use org-roam-capture-templates for notes
org-noter integration
- Configuration for pdf-tools:
IMP: After installing pdf-tools, do M-x pdf-tools-install <Ret> and put this line in your init file:
(pdf-loader-install) ; use PDFView in place of Doc View
- org-noter integration with org-roam and org-roam-bibtex
(setq org-noter-notes-search-path '("/home/USER/roam/references/notes/")) ; V IMPORTANT: SET FULL PATH!
(setq orb-preformat-keywords '("citekey" "title" "url" "author-or-editor" "keywords" "file") ; customisation for notes, org-noter integration
orb-process-file-keyword t
orb-attached-file-extensions '("pdf"))
Create an org-roam-capture-template - add only the template to your already existing variable, do not copy paste this as-is!!
(setq org-roam-capture-templates
'(
("b" "bibliography notes" plain ; Org-noter integration
(file "~/roam/references/notes/notes-template.org")
:target (file+head "references/notes/${citekey}.org"
"#+title: ${title}\n")
:empty-lines 1)
)
)
For example, my full entry looks something like this, they also contain some extra export ‘options’ i deem sane!
(setq org-roam-capture-templates ; Org-roam capture templates
'(
("d" "default" plain
"%?"
:target (file+head "%<%Y%m%d%H%M%S>-${slug}.org"
"#+title: ${title}\n#+options: author:nil, date:nil, toc:nil, num:5, H:5, html-postamble:nil\n")
:empty-lines 1
:unnarrowed t)
("b" "bibliography notes" plain ; Org-noter integration
(file "~/roam/references/notes/notes-template.org")
:target (file+head "references/notes/${citekey}.org"
"#+title: ${title}\n#+options: author:nil, date:nil, toc:nil, num:5, H:5, html-postamble:nil\n")
:empty-lines 1)
)
)
Other configurations
Create a file ~/roam/references/notes/notes-template.org and add the following:
- tags ::
- keywords :: %^{keywords}
* %^{title}
:PROPERTIES:
:Custom_ID: %^{citekey}
:URL: %^{url}
:AUTHOR: %^{author-or-editor}
:NOTER_DOCUMENT: %^{file}
:NOTER_PAGE:
:END:
Create another file ~/roam/references/master.bib and prepend the following:
Manual: https://www.jonathanleroux.org/bibtex-mode.html
---------------------------------------------------------
+---------------Keybindings-----------------+---------Particulars--------------+
| C-c C-e C-a bibtex-Article | Article:Journ,Mag,Newsp,Periodi. |
| C-c C-e C-b bibtex-InBook | Section inside Book |
| or (C-c C-e I) | |
| C-c C-e b bibtex-Book | Book |
| C-c C-e B bibtex-Booklet | Bound but lacking Publshr/Inst. |
| C-c C-e C-c bibtex-InCollection | Article in a Collection |
| or (C-c C-e i) | |
| C-c C-e C-p bibtex-InProceedings | A Conference Paper |
| or (C-c C-e C-i) | |
| C-c C-e p bibtex-Proceedings | The whole Conference proceedings |
| C-c C-e m bibtex-MastersThesis | Thesis for Grad lvl |
| C-c C-e P bibtex-PhdThesis | Thesis for Phd level |
| C-c C-e C-m bibtex-Manual | Technical Manual |
| C-c C-e C-t bibtex-TechReport | Inst.Report,WhitePaper,WorkPaper |
| C-c C-e C-u bibtex-Unpublished | Accepted/Submitted/Not Publ. |
| | (Include 'note' to specify which)|
| C-c C-e M bibtex-Misc | Web pages,Slides,Personal notes |
| C-c C-e M-p bibtex-Preamble | Formatter code for .bbl file |
| C-c C-e C-s bibtex-String | Define Variables inside entry |
+-------------------------------------------+----------------------------------+
Finally ! We add just one keybinding! and everything will fall in place
(global-set-key (kbd "C-c f r") 'helm-bibtex) ; keybinding
I use the "f" key as a org-roam modifier, and "r" seems natural for references, use whatever works for you. The important point is, we shouldn’t need to have more than one keybinding, one is more than enough!
Misc optional configurations
Writing technical documents requires us to write in paragraphs, whereas org mode by default is intended to be used as an outliner, to get around this problem, setting up org-export to preserve line breaks is useful, there are two ways to achieve this, we can add \n:t to #+options: as a document specific setting, or we can set
(setq org-export-preserve-breaks t)
in our init file for this to work globally in all .org files.
Usage
When we are writing a document, we can press C-c f r to bring up a dedicated buffer where we can see our bibtex entries, pressing <Tab> on an entry will show all the options available. Pressing <f3> for example will insert that citation that was selected and close the buffer, pressing <Ret> on an entry will take us to the url <f1> if exists otherwise, and so on.
Pressing C-o on the buffer will change selection to the “Fallback Options”, the first one of which is to include a new entry to our bibtex file.
When an cite:&citekey exists on a document, pressing <Ret> on it will show a mini buffer where we can do most of what we need: open the associated pdf(s), “find” notes, go to the url, change the citation, remove it, and many more, this minibuffer is of `org-ref’.
Notes created of the associated documents are also compatible with org-noter.
Example of a .bib entry for an article
@article{cohen1963,
author = "P. J. Cohen",
title = "The independence of the continuum hypothesis",
journal = "Proceedings of the National Academy of Sciences",
year = 1963,
volume = "50",
number = "6",
pages = "1143--1148",
url = {https://localhost:1},
file = {path/to/pdf.pdf},
}
Note: if we include the file entry, bibtex-complete will not look for a file in ./references/documents/ , although the document says that it should look for a file if the file provided does not exist, but for some reason if the file doesn’t exist, emacs opems / in Dired. So use either-or,
Documents kept in ./references/documents/ must be of the type : citekey-name.pdf. in our example, they may be:
- cohen1963.pdf
- cohen1963-original.pdf, cohen1963-whatever.pdf and so on, all of these documents will be automatically connected to citekey:cohen1963.
Finally, very important: in every .org file where we include citations, we MUST include two lines at the end for the export to pdf C-c C-e l p to work correctly, namely:
#+latex: \bibliographystyle{unsrt}
#+latex: \bibliography{references/master}
For other bibliography styles click here
When we are exporting to html from C-c C-e r h, only inclusion of
bibliography:/home/USER/roam/references/master.bib
at the end of the document is needed. Notice that for latex export we can get away with using relative file path and without the extension .bib, but for html, we must use the full file path alongside with the extension .bib!
Further Notes
-
Installing latexmk will suffice for exporting, emacs is configured to auto detect its installation and adjust its export-process automatically, you don’t need to fiddle with org-latex-pdf-process. If you do not have it installed as a system package, you’ll inevitably fall into problems and need to read the manuals and scour support forums for answers.
-
org-noterintegration is non-commutative: meaning the intended workflow is to first create the note fromorg-roamand then open the pdf and doM-x org-noterinside the pdf. To create a note, either press<Ret>on an inserted citation and pressn; or, callhelm-bibtex (C-c f r), select the entry and press<f8> (edit notes). If the note is created from org-noter on the other hand, whenorg-roamwill first try to visit the file it will fail to create the#+titleand will append :PROPERTIES: to the first heading created by org-noter instead of the whole page. This is probably not what you want. -
M-x orb-insert-linkwill insert a link for the notes page of a citation when called, similar toorg-roam-node-insert, could be bound to a key for some workflows; I have it bound to
C-c f C-r i.
Also,M-x orb-note-actionswhen on an notes page will give some options such as “Open Pdf” “Add Pdf” “yank citekey” “Go to bibtex entry” and so on, pressing<Tab>will show all the options available, can be also considered to be bounded ex:C-c f C-r n -
This guide was written for vanilla-emacs, on a GNU/Linux system, I have not used Windows for many years, and I have never used Mac nor any other Emacs distribution, so I cannot know how to make it work for these other distributions, or if this guide is 1-1 compatible with them.
Conclusion
I think that is all, I have tried my best to make this error free, if any error has inadvertently crept in please let me know. I hope you have a productive time doing research. Thank you.
momentarily I’m on a windows machine will let know if it works
