V2 Migration: issues with roam-refs, org-open-at-point and the graph

[laotang]

After publicly contemplating to stay on v1 for as long as possible, I gave v2 another spin yesterday. The migration was successfully in that my notes have been transformed to the new format and the basics of the system seem to work. Still, I experience some issues that are probably linked to my particular setup and would be thankful for any ideas on how to fix them.

1. org-id and org-open-at-point

I run a “one frame-two window” setup (see also below), which served me well in the last year. In this, I want all links in one window to open in other-window. In v1 I had to set this in my .emasc:

(setq org-link-frame-setup '(
  (file . find-file-other-window))

I have no idea how to get this to work with org-id links in v2, as all links open in the same window. This earlier discussion suggests that a bug in org-id might be responsible.

2. roam-refs and zotero-links

I use org-roam to take notes on other documents (newspaper articles, scientific publications and laws). These are stored in Zotero. Using the add-on Zutilo in Zotero I get links such as this one “zotero://select/groups/2425626/items/7PXILIN4”, which I can open in Emacs by adding this to my .emacs:

(defun zotero-open-ext (path-to-media)
  (shell-command (concat "open zotero:" path-to-media)))
(org-add-link-type "zotero" 'zotero-open-ext)

I added these links to #+roam_ref in v1 and it worked great. In v2, org-roam struggles and I get following warning:

"Invalid ref, skipping…" for almost all Zotero links stored in :PROPERTIES:.

3. org-roam-graph

I understand that org-roam-graph in v2 is still under development - but right now I get very different results between v2 and v1. Compare these two versions of the graph for a preliminary note on “political culture” in v1 and v2 (C-u 1 M-x org-roam-graph). (For me, the v1 graph is the correct one).

v1:

v2:

[nobiot]

1. org-id and org-open-at-point
   An easy way is to override an Org-roam function.

2. roam-refs and zotero-links
   This can be achieved via customizing of Org mode.
   EDIT: Quickly reviewed what I wrote.
   Use org-link-set-parameters instead of org-add-link-type (its documentation says it’s obsolete)

3. org-roam-graph
   I agree that it looks weird compared with V1. Can be a customizing or a technical defect. Sorry; no idea (I don’t use org-roam-graph)

org-id and org-open-at-point

Override org-roam-open-id-at-point as below.

It requires only one line of change (change one argument from nil to non-nil).
Source with the relevant line marked

I believe there should be a more elegant way (less impact; this is a global change). Maybe it should be looked at by the project but I cannot think of a good way for a pull request…

  (defun org-roam-open-id-at-point ()
  "Try to navigate \"id:\" link to find and visit node with an assigned ID.
Assumes that the cursor was put where the link is."
  (let* ((context (org-element-context))
         (type (org-element-property :type context))
         (id (org-element-property :path context)))
    (when (string= type "id")
      (let ((node (org-roam-populate (org-roam-node-create :id id))))
        (cond
         ((org-roam-node-file node)
          (org-mark-ring-push)
          (org-roam-node-visit node 'other-window 'force) ; change the second argument from nil to non-nil
          t)
         (t nil))))))

roam-refs and zotero-links

EDIT: Use org-link-set-parameters instead of org-add-link-type, which is obsolete (see its documentation). The new function takes care of (org-link-make-regexps).

I have done a quick test and this change correctly lets Org-roam insert a zotero ref to the DB.

I cannot test the zotero-open-ext function so it may not follow the link correctly but this change works for roam_refs.

org-roam-graph

Sorry I cannot help you here; I don’t use it at all.

One obvious suggestion would be to have a look at org-roam-ui project.

If you would like to get org-roam-graph “fixed”, a good starting point would be to see the differences between the nodes that are correctly displayed and those that are not, and see if there is any pattern.

[laotang]

Thanks again for a great answer and all of your effort. Very much appreciated!

@nobiot

Override org-roam-open-id-at-point as below.

This solves the org-id and org-open-at-point issue.

@nobiot

EDIT: Use org-link-set-parameters instead of org-add-link-type , which is obsolete (see its documentation). The new function takes care of (org-link-make-regexps) .

This is also a great advice. I still see the occasional Zotero-link in org-roam-graph (when it works), but I think this is by design. In v2 org-roam-graph seems to extract all links and presents them to the user (see below).

@nobiot

org-roam-graph

Sorry I cannot help you here; I don’t use it at all.

Thanks anyway. One of the previous changes may have made org-roam-graph in v2 even more unreliable - it now only works every other time I try it (maybe one in four times). Perhaps the best advice for org-roam-graph users of v1 is to just wait a bit longer and/or to check org-roam-ui.

[laotang]

Struck at home with a relatively mild case of covid, I had some spare time to give v2 another spin.

As much as I want to move on, I quickly had returned to v1 (for now). This was mainly related to two major issues for me, which I discuss in more detail below. On a positive note: org-roam-ui appears to have great promise. However, for now org-roam-graph in v1 is enough for my needs (strangely org-roam-graph has not changed since my last encounter with v2).

A bit of context: I have 2200+ notes and do not experience a slowdown of the system yet.

issue 1: not all file-links had been migrated to id-links

I am not sure what stopped the migration process and noticed this only by accident. A substantial part of the file-links had not been transformed into id-links. Of course this had an effect on the org-roam buffer, which only showed a fraction of the existing backlinks.

v1 had 8 backlinks:

v2 sees only 3 backlinks:

This is a scary bug. Unknown data loss is the worst kind of them…

issue 2: the new org-roam buffer shows way too much context

Is there a way to limit the display of the backlinks to their respective surrounding context as in v1? There seems to be much more noise in the v2 buffer.

v1 (look at Yan 2021 in the org-roam buffer):

v2 (the relevant line is not even shown):

[nobiot]

Can’t say anything for issue 1.

For issue 2, try to customize org-roam-preview-function. Refer to this comment from @jethro in GitHub. The comment is related to the PR that introduced the customizing variable.

[laotang]

Thanks, really appreciate this! On issue 2: shouldn’t this be the default? Irrespective of all potential new benefits of using v2 - the default experience and usability being a step backward from v1 seems to be a common theme in v2 (for me). Especially in regard to the org-roam buffer, which in default does not work with a mouse (or has regular links), is positioned in a questionable way, and shows too much context. Emacs being Emacs means there are solutions to all of this, but these are curious choices. Sorry for being critical here, especially as someone who is not capable to contribute to the project himself.

[nobiot]

For the mouse click, see this part of the wiki.

In V2, the backlink buffer (Org-roam buffer) is its own major mode based on magit-section, not org-mode – I suspect this change nfluences the user experience by shifting it from familiar to unfamiliar.

[laotang]

@nobiot

I suspect this change nfluences the user experience by shifting it from familiar to unfamiliar.

I will admit, there is something to this - but I would also argue that in some ways the old version was more geared to the needs of (average?) users. Whenever I give v2 a spin, I will end up with way more lines in my config file, “fixing” certain aspects of v2 for the future (when I try again). Though I am not sure how users feel that never experienced v1.

Sorry for the grumbling and thanks again for the help!