The Org-roam v2 Great Migration

Development
1

I’ve been personally using and developing Org-roam v2 for a few months now (along with some others), and I think it’s finally stable and usable enough for public testing. I’ve released a tag on GitHub that marks this milestone:

This thread is for handling any problems/queries wrt to upgrading to v2.

Why should you migrate to v2?

  • only v2 is in active development
  • the v2 codebase is greatly simplified and should provide a much smoother experience
  • Experience the upgraded Org-roam buffer

Sample configuration:

(use-package org-roam
  :custom
  (org-roam-directory "/path/to/org-files/")
  :bind (("C-c n l" . org-roam-buffer-toggle)
         ("C-c n f" . org-roam-node-find))
  :config
  (org-roam-setup))

Read more about Org-roam v2 here

How do I migrate to v2?

You should be able to (for the most part) convert your v1 files to the v2 format as follows:

roam: links, and roam_tags are not automatically converted.

Should you hold off on the migration?

Stick to Org-roam v1 if:

  • You dislike using ID links
  • You rely on external packages like org-roam-server and org-roam-bibtex
  • You rely on org-roam graphing
11 Likes
2

That’s awesome, that migration script is very useful I still have some nores hanging around in v1 format.

Thanks @jethro! I’m going to try this now

3

Maybe the migration script should be included in the org-roam source for v2, and exposed as a command?

4

There seems to be unmatched parenthesis error in this migration script.

5

You might want to explicitly say that 2 is currently on the branch, and when you plan to merge to master?

6

Great work!! Conversion went relatively smoothly, yet there are some questions. Don’t know where’s the place to ask, so here they are:

  • Org Roam files now can have filetags attached. They seem no to be inherited, however. Are there any plans to allow tag inheritance à la org mode?

  • What is the conceptual difference between filetags (or tags at headings) and ROAM_TAGS? In v1, I used the ladder as a type category (“Thought”, “Reference”, “Quotation”, “Overview”, etc.). Is it recommended to use filetags for that now? They show up in the node completion, ROAM_TAGS don’t.

  • I added an ID to every heading and have more than 3.000 nodes now. It would make sense, for me, to display the nodes in an hierarchical manner (“Title/Heading/Subheading”) when "find"ing a node. Is anything like that planned?

  • How is “org-setup” supposed to be used? I do not understand why it is not a global major mode, since that’s all it does. I put it in the :config part of my use-package declaration, but it feels wrong.

7

I’ve been eyeing on v2 for a while and tried it out today. I reverted to v1 shortly after.
Here’s my thoughts and experience. I think most of them are non-issues, and will soon be fixed going forward, but I’m echoing my voice anyway.

Before I start, I want to mention that I use Doom Emacs. I have all my notes already version controlled, so I can easily revert changes to my notes, but it is not the same for many others.
As such, introducing automated changes to notes should be made with caution.

Installing

For some reason, I could not fetch v2 by adding :branch v2 into my package! recipe, even when I copied the recipe from the wiki guide. I ended up going to my local straight.el clone and switch the branch to the alpha release.

I’m guessing this is not Org-roam problems. Moving on.

Migrating notes

  • Firstly I found the downloaded migrate script have some syntax error, as I pointed out in a reply above. A trivial fix is enough to make the script functional

Step 1:

  • As the script runs, it opened the buffer for every single notes. So I have to wait for inline Latex previews to render.
  • The script made error and halt for my old notes, because the #+latex_header: \addbibresource{...} parts in those notes was wrong and I had to fix this. I don’t know why the script cares about bib sources. Maybe it tried to re-render math previews in those files, but the LaTeX previews did not re-render when I manually visited those files – the preview images were already cached.

Step 2:

  • Lots of duplicate ID errors popped up and I don’t know why, but they did not halt the script, so I let it happened
  • After this step, I tried restarting Emacs, but it asked me to confirm changes to all 664 of my notes. A quick check via magit shown that all notes have been created with IDs, so I kill emacs without saving all visited files.
  • I restarted Emacs. The duplicate errors popped up again. And I ran step 3.

I also change all #+roam_tags to #+filetags, using rg and ivy-occur, and I restarted again.

Using v2

  • org-roam-find-node successfully found my nodes. However, none of them showed any backlinks. A project-wise rg search returned almost zero file: links in my notes folder; that should mean that links are successfully converted. So why no backlinks are displayed?
  • Calling the template for org-dailies returned an error. I guess the dailies functionality has not been made into v2 yet?

After playing around with v2 a little, I decided to revert back to v1. Backlinks and capturing didn’t work for me, so I think I should wait a little more until v2 rounds its edges.

8

Hmm, I think it’s fine left outside as a separate standalone script, I’ve added execution instructions to the script for those less adept with Emacs Lisp.

Yeah I’ve left out how to actually switch branches… Will add to the original post. I don’t know when I plan to merge to master yet, I guess when more people are onboard and there are fewer bug reports.

1 Like
9

Tag inheritance should work now: I have nodes that inherit the file node’s #+filetags. If they don’t for you, do share:

  • your Org-roam config
  • your Org files
  • the commit version on v2

There is no more ROAM_TAGS, there’s only one tag source now, and that is how Org computes it.

I think you should be more selective towards what you add IDs to, but I am considering storing the outline path in the DB and allowing for its use in the completions.

Putting it in :config is correct, just think of it as a one-time executable thing that sets up Org-roam. org-roam-teardown removes what org-roam-setup sets up.

1 Like
10

Perhaps it would be better to call it org-roam-startup. When I see the word setup I think about configuring it to work, and expect that to be a one-time thing, or when something changes.

1 Like
11

Let me suggest to simply add a link to the wiki – it mentions V2 branch with a direct link. Even a bit of how-to for Doom users (albeit a report of it not working; I guess someone can look at it).

I have kept the wiki fairly up-to-date, including the recent change over to a user option for sections.

If any part of the content is obsolete, simply update it. Or if there are too many inconsistencies to update, then I’d suggest that we declare that it has served its intended bridging purposes and archive it (remove it) to avoid further confusion.

3 Likes
12

Still, it should be wrapped in a global minor mode, because that’s exactly what they are for. I’ll do PR if I find some time; but here’s a starter:

  (define-minor-mode org-roam-active-mode
"Set up hooks for org roam."
:global t
(if org-roam-active-mode
(org-roam-setup)
  (org-roam-teardown))) 
(org-roam-active-mode +1)
13

Thanks for the detailed responses!

As to the ID stuff, I think the point is rather that IDs should not be necessary to FIND a node, that’s all. So adding an outline path to the DB is great; even greater would be to allow to access all headings, independently if they have an ID or not. This would reduce the need to create many IDs, because they would then be only used for links. But to find links, it is very useful to access all headings, not just those already linked.

I remember reading somewhere on discourse that one reason against such a system is that there are “general headings” which should not show up as a node. My suggestion is to display only those un-ID’ed headings which do not have a further subheading - this would automatically filter out all those top-level headings which are used for sorting only. The idea is that only those headings with the deepest nesting actually contain information, while the other (“upper” ones) are for internal organization. That’s at least how I organize my reference pages.

Re: “inheritance of filetags”: I use commit “38b5375” of the v2 branch, but I have no time to debug it right now. I’ll look into it and file a more detailed issue on github if the issue persists with the latest commit.

15

Re “inheritance of filetags”: have you tried using C-c C-c on the line containing the #+filetags?

16

thanks for asking! The behavior is inconsistent. I just had an actual note where tag inheritance did not work. Reading your comment, I created a minimal page to make it reproducible – and alas, here tag inheritance did work. :expressionless: So time to dig into it some time later…

EDIT: I found the problem, it was a missing colon in one file. File tag inheritance is working fine! That’s actually a cool feature, I like it.

1 Like
17

I just got migrated, and so far I’m liking it quite a bit. For one thing, emacs startup time is way faster, so that’s great.

A few things I’ve run into with migrating:

  • To get straight.el to build from the v2 branch, I had to rm -rf my existing org-roam repo folder. To do this without throwing an error, you need to remove org-roam from your init, restart emacs, close emacs, then delete the repo, then add the new, updated recipe to your init file.
  • I received a few hundred errors for non-unique IDs when running the script provided by @jethro linked above. I haven’t checked to see if that has caused any issues or done anything to try to mitigate yet
  • I’m running into a small, strange apparent bug. If I attempt to call org-roam-capture before I have called org-roam-node-find after a restart of emacs, I get the error (void-variable org-roam-find-file-hook) (see below for full debugger output). Re-evaluating org-roam-setup does not seem to cause the error again, nor does org-roam-db-sync.

Here’s the full debugger output:

Debugger entered--Lisp error: (void-variable org-roam-find-file-hook)
  (member #'org-roam-db--update-on-save-h org-roam-find-file-hook)
  (if (member #'org-roam-db--update-on-save-h org-roam-find-file-hook) org-roam-find-file-hook (setq org-roam-find-file-hook (cons #'org-roam-db--update-on-save-h org-roam-find-file-hook)))
  eval-buffer(#<buffer  *load*-400208> nil "/Users/Jeff/.emacs.d/straight/build/org-roam/org-roam-db.el" nil t)  ; Reading at buffer position 18477
  load-with-code-conversion("/Users/Jeff/.emacs.d/straight/build/org-roam/org-roam-db.el" "/Users/Jeff/.emacs.d/straight/build/org-roam/org-roam-db.el" nil t)
  require(org-roam-db)
  eval-buffer(#<buffer  *load*> nil "/Users/Jeff/.emacs.d/straight/build/org-roam/org-roam-capture.el" nil t)  ; Reading at buffer position 1439
  load-with-code-conversion("/Users/Jeff/.emacs.d/straight/build/org-roam/org-roam-capture.el" "/Users/Jeff/.emacs.d/straight/build/org-roam/org-roam-capture.el" nil t)
  autoload-do-load((autoload "org-roam-capture" "Launches an `org-capture' process for a new or existing note.\nThis uses the templates defined at `org-roam-capture-templates'.\nArguments GOTO and KEYS see `org-capture'.\n\n(fn &optional GOTO KEYS)" t nil) org-roam-capture)
  command-execute(org-roam-capture)
18

Let me just +1 using the outline path for completions. Selective with ID’s or not, the nodes within files have a context, just as files within folders has. Using that hierarchy makes total sense, even if each node is its own entity as well.

19

For the spacemacs users, I used this little patch to layers/+emacs/org/packages.el

-    (org-roam :toggle org-enable-roam-support)
+    (org-roam :toggle org-enable-roam-support
+              :location (recipe :fetcher github :repo "org-roam/org-roam" :branch "v2")
+              )
3 Likes
20

Just failed to migrate from v1 to v2 by using the mentioned script in this thread.

  • files are successfully added ID and aliases
  • but db is not successfully build, because some emacs sql expression error
  • file links were not converted to IDs
  • org-roam-node-find can only find my 4 notes (among of hundreds of them)

I would give a try next time when the migration script is more stable and I have more time.

21

Next time you try, if you can capture the sql error – ideally as text but even as screenshot otherwise – and post it here, it would almost certainly help us/people figure out what went wrong so we can make it more stable more quickly :slight_smile:

1 Like