Interoperability between Org Roam and regular Org

Hi, I’ve built a little graph in Org Roam:

I’d like to “linearize” this and work on it inside of one (big) regular Org Mode file, including the ability to export to downstream formats like PDF. I’d also like to save changes back into the Org Roam notes.

The obstacle that comes to mind is links like [[file:20200810132653-top.org][Top]]. Inside of the “downstream” Org file, this should presumably look like [[*Top]] (assuming that Org Roam notes get mapped to top-level sections). All well and good, but then where does the file:20200810132653-top.org information go, so that it can be re-inserted later?

I’m sure several solutions are possible. Curious to know if there’s a “standard” way to do this yet!

Hi, that of course should go into a property under the respective headline. I’m curious, however, whether you’ve already succeeded in linearising your Org roam files in the first place?

that of course should go into a property under the respective headline.

Yeah, that makes sense, though the script to restore things back to Org Roam would have to know to look there, and would need to reinflate things appropriately.

Anyway, here’s what I came up with so far, dealing with the Org Roam to Org direction. This seems to work fine though it may not be the ideal solution yet.

roam2org.sh

#! /bin/bash

emacs --batch -l ~/bin/downsample-org-roam.el --eval "(combine-files)" "$@"

and,

downsample-org-roam.el

(defun downsample ()
  ;; node title becomes a top-level section
  (if (looking-at "^#\\+TITLE:")
      (replace-match "*"))
  (forward-line 1)
  ;; roam tags become org properties
  (if (looking-at "^#\\+roam_tags:\\(.*\\)")
      (replace-match ":PROPERTIES:
:tag:\\1
:END:"))
  ;; All subsections go down in depth
  (while (re-search-forward "^\\*" nil t)
    (replace-match "**"))
  (goto-char (point-min))
  ;; Links to files are replaced with internal links to sections
  (while (re-search-forward "\\[\\[file:\\([^]]*\\)\\]\\[\\([^]]*\\)\\]\\]" nil t)
    (replace-match "[[*\\2][\\2]]"))
  (concat "# " (buffer-name) "\n" (buffer-substring-no-properties (point-min) (point-max))))

;; `command-line-args' corresponds roughly to $@ in the shell
(defun combine-files ()
(princ
(apply #'concat
(mapcar (lambda (file) (progn (find-file file)
                           (downsample)))
        (nthcdr 5 command-line-args)))))

Then just run:

roam2org.sh *.org


(Edited after I improved the script somewhat!)

1 Like

Does it actually alter your original Org roam files? Usually one would do something like

(let ((contents (buffer-substring-no-properties (point-min) (point-max))))
    (with-temp-buffer
       (insert contents)
       ;; ... do the processing
  ))

For writing changes back into an Org roam file, I would recommend you to look at the built-in Org parser, the org-element library.

Maybe also the forward parsing could be done by means of org-element and other standard tools, because your approach, for example, will work only when org-odd-levels-only is set to nil, that is each nested headline gets only one additional star. Many people, however, use org-odd-levels-only, in which case each nested headline gets two stars.

* Parent
*** Child 1
***** Child 2

It can be painful to handle this and possibly other situations without re-inventing parts of the built-in Org functionality. For promoting subtrees, see the org-promote-subtree function.

Yeah, I rewrote it with with-temp-buffer for direct use from within Emacs. Since the version I wrote above ran in a script and didn’t ever “save” the contents, it could be a bit more hacky.

I also ended up using the org-element library for subsequent downstream processing of the outputs (because I wanted to nest elements that were not tagged as “high level”), viz., here’s what I needed:

(org-map-entries (lambda ()
                   ;; don’t demote the top level items and their sub-items
                   (if (string= (org-entry-get nil "tag") "HL")
                       (progn (org-end-of-subtree)
                              (setq org-map-continue-from (point)))
                     ;; demote everything else
                     (org-do-demote)))
                 nil 'file)

It certainly makes sense to use a similar process to set up the nesting in the initial pass as well. Probably I can clean this up and get it into a state where it can go into a “contrib” commit. (I’m not sure what the protocol is for that, though it looks like you discussed a potential contrib workflow with Jethro at some point!)

Well, thanks to @zaeph and @jethro, that contrib code had quickly grown into a separate package, org-roam-bibtex, of which I was initially sceptical, and which eventually incorporated more functionality than was intended in the beginning. This is a plausible way to bring your ideas and their realisations into Org Roam’s infrastructure.

I actually envisioned something like this for org-roam-bibtex as a means of producing annotated bibliography for printing out. Moreover, such an annotated bibliography was one of the earliest features requested by others, as were the requests to provide a means for migrating from one file-style notes to Org Roam. So, both one way and round trip general Org Roam–Org export functionality with some perks like filtering, sorting and so on, will find its users, and implementing it is a worthy project.