# Org-roam major redesign **URL:** \ **Category:** Development\ **Created:** [January 23, 2021, 1:17pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198 "2021-01-23T13:17:57Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![jethro](https://avatars.discourse-cdn.com/v4/letter/j/b5a626/32.png) [@jethro](https://org-roam.discourse.group/u/jethro)\ **Post date:** [January 23, 2021, 1:17pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/1 "2021-01-23T13:17:57Z")
I’m making an extremely large breaking change to Org-roam moving forward. Org-roam was developed in a time where files were the lowest denomination for a note. Fast forward to today, we now support headlines. The current implementation we have is a mess: there are special paths for handling file and id links, and it is getting difficult for me to make changes without accidentally breaking something else. There is a strong need to rebuild Org-roam’s basic mechanisms. I think this is the only reasonable path forward, allowing me to trim a big deal of technical debt and increase development speed. As the primary maintainer of the project, I’m finding it hard to wrap my head around all these different features that I don’t use, which are supported in awkward ways. I think the proposed change is much easier to reason about both as a developer, and as a user, and will result in a higher quality product overall. What is likely going to happen is that I will release a final tag on Org-roam v1, and start working on Org-roam in a separate dev branch. That Org-roam will be tagged v2. Those who wish to continue using Org-roam v1 will have to pin the repo to v1. ## The proposal I term the lowest denomination we have in Org-roam a _node_. A _node_ is defined as follows: > A node is any headline or top level file with an ID. Nodes link to other nodes using ID links. Nodes also have an implicit hierarchical structure, from the levels in the Org file. Nodes can link to many other nodes. Org-ref links etc. will continue to be supported. Here we enforce usage of Org IDs. This is simplifying for many reasons: all links within Org-roam are ID links, and we no longer have to deal with handling different kinds of path links and path breakages on file changes such as renames and deletions. We also no longer have to handle files and headlines differently within the schema, which has been a recurring pain point. ## The migration strategy It should be simple to write an elisp script that adds IDs to everything, and converts existing file links. ---
**Author:** ![nobiot](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/nobiot/32/159_2.png) [@nobiot](https://org-roam.discourse.group/u/nobiot)\ **Post date:** [January 23, 2021, 3:34pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/2 "2021-01-23T15:34:11Z")
Looking very much forward to v2 and beyond 🚀 ---
**Author:** ![arozbiz](https://avatars.discourse-cdn.com/v4/letter/a/898d66/32.png) [@arozbiz](https://org-roam.discourse.group/u/arozbiz)\ **Post date:** [January 23, 2021, 3:44pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/3 "2021-01-23T15:44:29Z")
This is brilliant, thank you. One question: you write “A node is any headline or top level file with an ID.” Is there a reason why the headline is the lowest (most granular) node level? E.g., why can’t plain list items or quote blocks etc. be nodes? Is it because Org-id only supports UUIDs at the headline level? I love Org-Roam for all the usual reasons (built on Emacs, local storage, plain text), but one think I miss about Roam Research is its hyper-granular block-level architecture, especially when it comes to transclusion. (I’ve also been playing around with @nobiot’s amazing Org-transclusion work, and I have to assume it would make his life easier if one could add UUIDs at a more granular level than just headlines). Anyway, amazing work all! ---
**Author:** ![nobiot](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/nobiot/32/159_2.png) [@nobiot](https://org-roam.discourse.group/u/nobiot)\ **Post date:** [January 23, 2021, 4:38pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/4 "2021-01-23T16:38:34Z")
Let me quickly jump in to comment on Org-transclusion, and leave the UUID question for Jethro. Firstly, thank you for your kind words, and commenting with my work in mind. My intuition is that Org-roam supporting UUID at the block-level and other more granular levels than headlines is unlikely to get me/Org-transclusion anything extra. [Edit: fixed the problem the exchange with Jethro made clear to me] Blocks, tables, and lists that are named (`#+name` keyword) can be transcluded using Org Mode’s standard links (`[[file:path/to/file.org::name]]`). ![image](https://global.discourse-cdn.com/free1/uploads/orgroam/original/1X/10d6c3bb4f9e3f6def2b582fa4b17b54dfd8fbb3.png) I am indebted to Org-roam for the idea of authoring Org-transclusion; it is intended to work alongside Org-roam. But it is a standalone package by design, and, at the moment, I do not intend to make it dependent on Org-roam. I will strive to make the writing process for users of both packages easy (because I’m one of them). I think that this sort of “interconnectivity” comes from sticking to their common foundation; that is, Org Mode without much reliance on specialised features. ---
**Author:** ![jethro](https://avatars.discourse-cdn.com/v4/letter/j/b5a626/32.png) [@jethro](https://org-roam.discourse.group/u/jethro)\ **Post date:** [January 23, 2021, 4:43pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/5 "2021-01-23T16:43:00Z")
> [@arozbiz](#): > > One question: you write “A node is any headline or top level file with an ID.” Is there a reason why the headline is the lowest (most granular) node level? E.g., why can’t plain list items or quote blocks etc. be nodes? Is it because Org-id only supports UUIDs at the headline level? This is an interesting thought. But how then would you reference a plain list item? It’s possible to make blocks a node too, but they will have to have an ID attached to them. ---
**Author:** ![nobiot](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/nobiot/32/159_2.png) [@nobiot](https://org-roam.discourse.group/u/nobiot)\ **Post date:** [January 23, 2021, 4:44pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/6 "2021-01-23T16:44:49Z")
One idea is to use `#+name: UUID` like I show above. Org Mode can then use the link `[[file:path/to/file.org::UUID]]` – that’s what I do for the block quote, table, and list examples above. > **[External Links (The Org Manual)](https://orgmode.org/manual/External-Links.html#External-Links)** > > External Links (The Org Manual) > ‘file:projects.org::some words’ (text search)[27](https://orgmode.org/manual/External-Links.html#FOOT27) ---
**Author:** ![jethro](https://avatars.discourse-cdn.com/v4/letter/j/b5a626/32.png) [@jethro](https://org-roam.discourse.group/u/jethro)\ **Post date:** [January 23, 2021, 4:49pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/7 "2021-01-23T16:49:04Z")
> [@nobiot](#): > > One idea is to use `#+name: UUID` like I show above. I don’t think this works for plain list items, but blocks, definitely. ---
**Author:** ![nobiot](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/nobiot/32/159_2.png) [@nobiot](https://org-roam.discourse.group/u/nobiot)\ **Post date:** [January 23, 2021, 4:52pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/8 "2021-01-23T16:52:20Z")
Oh, yes, you’re right. I see now the hole in this approach; they don’t correctly end (end in the next headline/block, etc.). Will need to fix this… Thanks! [Edit: I think I have fixed the problem in Org-transclusion for the moment; I think this is a separate topic from the original intent of Jethro’s post] ---
**Author:** ![Gustav](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/gustav/32/511_2.png) [@Gustav](https://org-roam.discourse.group/u/Gustav)\ **Post date:** [January 23, 2021, 5:04pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/9 "2021-01-23T17:04:30Z")
This seems to me like a good path forward. Looking forward to see what will come out! Regarding translating file links and id-links, I played around with this when evaluating gkroam a while back. Translation functions back and forth between file and ID-links can be found here: - [Helper functions to translate ID-links back and forth to gkroam-links · GitHub](https://gist.github.com/Whil-/b7e4a5dd58c245fe1a3e0164107e3297) (Note that the functions work between normal file links and ID-links as well, even though the gist mentions “gkroam”) ---
**Author:** ![d12frosted](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/d12frosted/32/278_2.png) [@d12frosted](https://org-roam.discourse.group/u/d12frosted)\ **Post date:** [January 23, 2021, 5:14pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/10 "2021-01-23T17:14:05Z")
> [@jethro](#): > > Here we enforce usage of Org IDs. I really like this idea. This is something I’ve enforced in all my notes and now with assumption that every note has an `id`, I can write some generic functions to work with any notes (files, headings). Some of them are extracted as part of [vulpea](https://github.com/d12frosted/vulpea) library. Having it structured helps me so much. But making `id` mandatory in the core, I think it would also enable many cool features. So looking forward! Please let me know if there is something I can help you with. ---
**Author:** ![d12frosted](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/d12frosted/32/278_2.png) [@d12frosted](https://org-roam.discourse.group/u/d12frosted)\ **Post date:** [January 23, 2021, 5:16pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/11 "2021-01-23T17:16:35Z")
> [@arozbiz](#): > > One question: you write “A node is any headline or top level file with an ID.” Is there a reason why the headline is the lowest (most granular) node level? E.g., why can’t plain list items or quote blocks etc. be nodes? Is it because Org-id only supports UUIDs at the headline level? One of the reasons is, as you said, `org-id` supports IDs only on heading level and file level (starting with one of the latest release of org mode). But another reason is performance. I tried using `org-roam` with relatively big file (1k headings) and experience was awful. I think org-roam really shines when using lots of small org files as opposed to another popular approach in org mode - few huge files. That being said, I agree with @jethro that it’s an interesting thought. ---
**Author:** ![nobiot](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/nobiot/32/159_2.png) [@nobiot](https://org-roam.discourse.group/u/nobiot)\ **Post date:** [January 23, 2021, 5:26pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/12 "2021-01-23T17:26:04Z")
> [@jethro](#): > > I don’t think this works for plain list items, but blocks, definitely. This is going off tangent from the original intent (sorry!). But it looks like in theory you could use `#+name UUID` idea for plain-list elements. See an excerpt below. `org-element` seems to correctly takes the `#+name` for the `plain-list` element with the correct end position (the element is `plain-list` with the correct `:end` position, and `:name`). The reason why my code copies more than it should is an error in my code. [Edit: I think I patched the problem in Org-transclusion for now] > (plain-list (:type unordered :begin 2198 :end 2242 :contents-begin 2211 :contents-end 2241 :structure ((2211 0 "- " nil nil nil 2225) (2225 0 "- " nil nil nil 2241)) :post-blank 1 :post-affiliated 2211 :name “list” :parent nil) ---
**Author:** ![arozbiz](https://avatars.discourse-cdn.com/v4/letter/a/898d66/32.png) [@arozbiz](https://org-roam.discourse.group/u/arozbiz)\ **Post date:** [January 23, 2021, 5:56pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/13 "2021-01-23T17:56:22Z")
I agree entirely with the many-small-files approach. But even within a relatively small file it can be useful to link to something lower than the headline level. But if Org-id doesn’t support that I guess it’s a limitation. I asked this question about Org-id on the Org mailing list but didn’t get an answer. ---
**Author:** ![jethro](https://avatars.discourse-cdn.com/v4/letter/j/b5a626/32.png) [@jethro](https://org-roam.discourse.group/u/jethro)\ **Post date:** [January 23, 2021, 7:09pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/14 "2021-01-23T19:09:25Z")
One thing I would like to additionally point out is that the simplifications also makes daemonizing the DB building much easier, which would mean remove any performance problems. ---
**Author:** ![scotto](https://avatars.discourse-cdn.com/v4/letter/s/b38774/32.png) [@scotto](https://org-roam.discourse.group/u/scotto)\ **Post date:** [January 23, 2021, 8:22pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/15 "2021-01-23T20:22:39Z")
You could use dedicated targets, which can be put anywhere – it’s what I do in plain org-files. ---
**Author:** ![danderzei](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/danderzei/32/59_2.png) [@danderzei](https://org-roam.discourse.group/u/danderzei)\ **Post date:** [January 24, 2021, 5:46am UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/16 "2021-01-24T05:46:30Z")
He would v2 break my current setup if I don use links to headers? ---
**Author:** ![jethro](https://avatars.discourse-cdn.com/v4/letter/j/b5a626/32.png) [@jethro](https://org-roam.discourse.group/u/jethro)\ **Post date:** [January 24, 2021, 5:51am UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/17 "2021-01-24T05:51:38Z")
No, but your files will need IDs too, and your links need to be to the file IDs. ---
**Author:** ![danderzei](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/danderzei/32/59_2.png) [@danderzei](https://org-roam.discourse.group/u/danderzei)\ **Post date:** [January 24, 2021, 6:34am UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/18 "2021-01-24T06:34:57Z")
So that is a yes. How does one convert? ---
**Author:** ![d12frosted](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/d12frosted/32/278_2.png) [@d12frosted](https://org-roam.discourse.group/u/d12frosted)\ **Post date:** [January 24, 2021, 8:43am UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/19 "2021-01-24T08:43:31Z")
I am not `org-roam` maintainer, so not sure if the solution for migration will be baked into `org-roam` (in my opinion, it should). But in case it will not, just ask me, I will share a script that will migrate your notes. ---
**Author:** ![shlevy](https://yyz2.discourse-cdn.com/free1/user_avatar/org-roam.discourse.group/shlevy/32/555_2.png) [@shlevy](https://org-roam.discourse.group/u/shlevy)\ **Post date:** [January 24, 2021, 1:01pm UTC](https://org-roam.discourse.group/t/org-roam-major-redesign/1198/20 "2021-01-24T13:01:53Z")
This sounds great! This won’t impact [[roam:]] links, right? [Next page](https://org-roam.discourse.group/t/org-roam-major-redesign/1198.md?page=2)