Using ORB and org-noter on PDFs of book chapters

How To
1

I am about to take an algorithms class in grad school, and I want to take notes into an org-roam system that will enable me to make connections between the algorithmic things I learn in class, and other things I do and learn in the future.

One of the books we use is Jeff Erickson’s creative commons textbook, found here: http://algorithms.wtf/

My idea for how to do this was to split the PDF up into chapters, since as chapters they are more like research papers on a particular topic. I have a feeling that taking notes on chapters and connecting concepts at the chapter level makes a lot more sense for me than at the book level, but maybe I’m wrong there. Maybe sub-chapters is better, maybe book-level is better, I dunno, this is just a hunch.

So I split up the book into one PDF per chapter, and then I took the unorthodox step of creating a bibliography entry for each chapter. And then I loaded up the skeleton of each chapters’ contents into the corresponding org-noter compliant format.

Now, I can use either org-roam-find-file or helm-bibtex to navigate to my org-mode notes, and from the notes I can open up the PDF file the corresponds to it, and of course to all the other cool org-roamy things one does around here.

I zipped up the PDFs and org files and bib file and put them on Dropbox, in case anyone is interested in seeing what I did and perhaps adapting it for their own use case.

But my main reason for writing this post today was, to ask what people who are much more experienced with all this stuff would have done differently, or any ideas for improvement upon this system people have.

I apologize for any broken links between the files; I don’t know how to excise a subset of an org-roam-bibtex library for sharing with others. Is there actually a way to do that?

OK I just realized I do have two items on my wish list already:

  1. I had to include what I consider a superfluous extra heading level in each .org file so that NOTER_DOCUMENT can be a property on a heading that’s a parent to all the chapter’s contents. I tried putting that property into a file-level property, but that didn’t work. Shouldn’t that work? Or would I need to submit a PR to org-noter to recognize org-file-level properties?

  2. I would like to be able to go straight from helm-bibtex to a window that opens the PDF, the org-mode noter file, and the org-roam window, with the PDF in the widest window, my notes in a more narrow window, and the roam window narrowest of all. Does anyone have a config for doing that?

1 Like
2

I was thinking that most readers would probably prefer to see a snippet of tjese files right here in this thread rather than having to download my tar file. So, here’s a typical org-noter notes file:

#+TITLE: Erickson's Algorithms, Ch 3: Dynamic Programming
#+ROAM_KEY: cite:210416_erickson_03_dynam_prog
#+ROAM_TAGS:

* Dynamic Programming
:PROPERTIES:
:ID: 210418_144100
:URL:
:AUTHOR: Erickson, J.
:NOTER_DOCUMENT: ~/pdfs/210416_erickson_03_dynam_prog.pdf
:NOTER_PAGE: 13
:END:
- tags :: [[id:210418_135936][Erickson's Algorithms]]
- keywords :: [[id:210418_135812][Dynamic Programming]]

** Mātrāvṛtta
:PROPERTIES:
:NOTER_PAGE: (1 . 0.624622)
:END:
*** Backtracking Can Be Slow
:PROPERTIES:
:NOTER_PAGE: (2 . 0.556022)
:END:

And here’s the bibtex entry corresponding to this chapter:

@chapter{210416_erickson_03_dynam_prog,
  author          = {Erickson, Jeff},
  title           = {Algorithms, Ch 3: Dynamic Programming},
  year            = 2019,
  publisher       = {S.N},
  address         = {S.L},
  isbn            = 9781792644832,
}

In case anyone was wondering whats up with cite keys that look like that, I chose this style of citation key to match the format of the file names I use for my files in org-roam. I like to have the yymmdd date as the prefix of the file to facilitate both searching by year, or year-month, or yymmdd via helm bibtex or org-roam-find-file, and I like the idea of having the same format for citekey as for pdf file name as for org file name.

3

Hi @gcoladon,

My few comments here.

That is actually a pretty orthodox BibTeX way. The only thing is that there is no such BibTeX (nor biblatex) entry type @chapter. This is not a big deal if you are working in Emacs (helm-bibtex, ORB, etc) but other software will be confused, e.g. Zotero. BibTeX, and more specifically biblatex (from now on this dialect is implied here), have several entry types to describe a book and a part thereof. The most common types for books are @book and @collection.

A book in BibTeX sense is a piece of work that has one or more authors who authored all the content. A book may have an editor or editors, but not necessarily so. BibTeX defines a special bibliographic entry type @inbook to be used with parts of a book - “A part of a book which forms a self-contained unit with its own title” (see the biblatex manual), that is most commonly chapters, but such units as “part” or “paragraph”, it follows, would conform with the BibTeX standard as long as they have their own title.

A collection in BibTeX means a piece of work that as a whole does not have authors. Rather individual chapters are authored by one or more authors, whereas a collection of such article-chapters is edited by one or more editors, responsible for the general editorial work. So a collection is more like a journal issue, although it is typically not periodical and has unlike the former a book layout. For referring to an individual chapter of a collection one is supposed to use the @incollection entry type.

There are also @proceedings and @reference entry types for, respectively, conference proceedings and reference works such as encyclopaedias and other compendia, which physically often look like books. The respective @inproceedings and @inreference entries should be used to refer to parts of such works.

The book you are reading is a @book in BibTeX sense, since it has a single author, therefore a BibTeX way would be to use the @inbook entry type to refer to its chapters.

Also, BibTeX has a “crossref” mechanism, which allows for child entries such as @inbook to inherit the bibliographic information of its parent entry such as @book.

So in your case, the BibTeX entries may look like:

@book{210416_erickson_dynam_prog,
  author          = {Erickson, Jeff},
  title           = {Algorithms},
  year            = 2019,
  publisher       = {S.N},
  address         = {S.L},
  isbn            = 9781792644832,
}

@inbook{210416_erickson_03_dynam_prog,
  title           = {Dynamic Programming},
  chapter         = {3},
  crossref        = {210416_erickson_dynam_prog},
}

@inbook{210416_erickson_04_dynam_prog,
  title           = {...},
  chapter         = {4},
  crossref        = {210416_erickson_dynam_prog},
}
...

So typically for the above @inbook entry BibTeX would infer the author, book title, year, publisher, etc. fields from the parent entry referenced via the crossref field. Bibtex-completion and ORB are no exceptions. If crossref doesn’t work for you in ORB, please let me know. There were some complaints in the past but I could not confirm them.

1 Like
4

Conceptually it should. At least the argumentation is solid, so you could attempt on requesting this feature from the author of org-noter.

Not that I know of.

5

Here’s the issue I filed, for reference: Allow :NOTER_DOCUMENT: to be in file-level :PROPERTIES: drawer · Issue #143 · weirdNox/org-noter · GitHub

1 Like
6

@mshevchuk, thank you for the education in BibTeX! I knew about neither @inbook nor crossref. I converted over to using them, and everything worked beautifully.

1 Like
7

@gcoladon Great!

8

I would like to be able to go straight from helm-bibtex to a window that opens the PDF, the org-mode noter file, and the org-roam window, with the PDF in the widest window, my notes in a more narrow window, and the roam window narrowest of all. Does anyone have a config for doing that?

I have something kinda similar for ivy-bibtex with a slightly different workspace config (just opens PDF and org-mode file). Warning: it’s an enormous hack (but that’s the only way I found to do this). I use Doom’s workspaces (which are built on persp-mode IIRC).

(defun zjn--bibtex-open-notes-and-pdf (keys)
    (let* ((key (first keys))
           (org-roam-find-file-function (lambda (file) (switch-to-buffer (find-file-noselect file) nil 'force-same-window)))
           (pdf (bibtex-completion-find-pdf key)))
      (+workspace-switch key t)
      (delete-other-windows)
      (when pdf
        (let ((pdf-buffer (find-file (first pdf))))
          (switch-to-buffer pdf-buffer nil 'force-same-window)
          (split-window-right))
        (windmove-right))
      (let ((org-capture-link-is-already-stored t))
                                        ; prevent trying to grab a link to the PDF
        (orb-edit-notes key))))
  (ivy-bibtex-ivify-action zjn--bibtex-open-notes-and-pdf zjn--ivy-bibtex-notes)
9

Thank you @znewman01 for sharing! I may try this. For now, I am not too put out by having to open the notes files from helm-bibtex, C-c C-n to first heading, and then C-c r n to open org-noter for. Come to think of it, I think I will create a small function that will prepend the going-to-the-first-heading action to the org-noter call, that’s probably enough of a hack to ge this off my list of annoyances, for now :slight_smile:

10

OK seeing @znewman01’s code above, and not wanting to go to sleep just yet, I came up with this approach which appears to me to solve most of the problem:

  (defun gpc/orb-find-file-open-noter (file)
    (find-file file)
    (outline-next-visible-heading 1)
    (org-noter)
    (other-window 1)
    (org-roam))

  (defun gpc/orb-edit-notes (orig-fn &rest args)
    (let ((org-roam-find-file-function #'gpc/orb-find-file-open-noter))
      (apply orig-fn args)))

  (advice-add 'orb-edit-notes :around #'gpc/orb-edit-notes)

Feedback welcome. It opens the PDF & Noter file from helm-bibtex, and also opens an org-roam window to the right. But it seems like you must move the point in the org-mode notes file in order for the roam window to load up. Almost there!

11

For some reason I can’t get org-noter to open the org documents associated with the PDF. The way I do: I open the PDF first, and then I do M-x org-noter. It will prompt me to create a new note.

13

How did you open the PDF?

14

I dragged and dropped it to the Emacs window, and it gets normally opened in a buffer.

15

@Adela

I have a feeling that your Org-noter is doing what it is designed to do.

See this part of its README on GitHub.

In a document
Run M-x org-noter when viewing a document (eg. PDF, epub…).
This will try to find the respective notes file automatically. It will search in all parent folders and some specific folders set by you. See org-noter-default-notes-file-names and org-noter-notes-search-path for more information [my emphasis]

If you “associate” an org file with a PDF document with using the :note-document: property within the org file, you would need to start org-noter in it, not in the PDF document.

16

@Adela, @nobiot , I might be misunderstanding what @nobiot is saying, but I think that the behavior reported by @Adela is not the expected behavior (at least, when everything is properly configured —in particular, the paths where noter has to store and search for notes).

In other words, the usual workflow can be:

  1. Open pdf for the first time
  2. M-x org-noter: you will be given a choice of where to store the notes (which can include multiple different directories, if you have set several in org-noter-notes-search-path).
  3. Take notes, etc. Save org buffer (and, possibly, the pdf if you highlighted, etc).

Now, whenever you open the pdf again (i.e., steps 4 to infinity), if you M-x org-noter in the pdf buffer the org file with the notes will be opened again (i.e., you will not be prompted for a new file). In other words, you can (should, indeed) use the same org files for all the notes of that pdf, and can open and add notes as many times as you want.

@Adela says “dragged and dropped”: I have no idea what that does. I open pdfs by directly C-x C-f (i.e., find-file), or clicking on them from Zotero (I have configured xdg-open to use emacsclient to open pdfs —I’m on Linux), or from helm-bibtex (which I uses find-file too) or from dired: the procedure above works reliably regardless.

(Indeed, this long explanation is what I understand is meant by the quoted part of the README in @nobiot’s answer).

17

Thanks. I see that :note-document: was not there in the original ORG documents uploaded by gcoladon.

18

Also, is it possible to set org-noter-notes-search-path to be in addition the current folder and subfolders where the PDF is located? Then it would work, I assume.

19

Thanks. Dragging and dropping onto Emacs has the same behaviour as opnening via C-x C-f :slight_smile: Now, it behaves exactly as you say. I realized I must specify the property :NOTER_DOCUMENT: to point to the PDF in the org-noter ORG file. Then I can run org-noter from the noter document and all is good. One question though. As I scroll the PDF, I see that the ORG-NOTER document jumps accordingly to the specific heading. However, is it possible to do the same the other way around? Moving within the org-noter buffer and having the PDF document jump to the specific page? Thanks.

20

EDIT: I see this is redundant with @rdiaz02’s more sophisticated advice :slight_smile: Thanks.

Don’t know. I rarely use Org-noter and read PDF in Emacs.
I guess you can easily try it yourself and see if it works for your case.

21

Wasn’t it there?