Performance Testing

Development
1

I’m loving org-roam, and, like many here, care about the longevity of my note system. With my habits (e.g. making nodes for most humans I interact with), I could easily end up with 10,000+ nodes in a few years. If that’s unlikely to be workable with org-roam, it would be good to know!

After searching through the code / tests / GH issues / Slack, I found very little info on the scalability of org-roam’s current implementation. I thought it would be good to aggregate info here, and make GH issues (e.g. scaling tests) as-needed.

Some things I’ve noticed so far:

  1. at ~500 nodes, most core operations are snappy for me
  2. running org-roam-insert on a new entity takes ~3-5 seconds until the link is inserted.
  3. I imported ~400 more nodes from Roam yesterday, and had no noticeable slowdown.
  4. Tangentially related: org-agenda can’t handle large numbers files, so I’ve been using capture templates with backlinks to workaround this. This is unfortunate, since interspersing TODOs directly in the nodes is a great Roam feature IMO.

Is anybody operating at near the 10,000 node scale?

2

I’m unable to test this myself, but one of the founders of zettelkasten.de has made available an archive of 10,000 markdown files for this type of stress-testing:
https://github.com/Zettelkasten-Method/10000-markdown-files

3

@dangirsh, @cobblepot, I was going to do a quick test to prepare a reponse to this exchange; it turned out to be a rather lenghty write-up for that purpose. So I created a separate post; you might find some data and my conclusion useful: What does it feel like to work with 10,000 notes in Org-roam: Benchmarking Org-roam's search methods. Thank you, @cobblepot, for the link. It was very useful.

4

Thanks for the details write-up + video @nobiot!

This gives me peace-of-mind for now.

1 Like
5

Just hit another snag: Org’s refile mechanism seems too inefficient to directly use on the entire set of org-roam files. Has anybody had success with this?

6

Most of Org-mode breaks down with both a large number of files, as well as with files that are large (e.g. org-agenda). That said, moving things around is a bit of an anti-pattern in the Zettelkasten method. Luhmann typically writes a note, assigns it an ID and files it away without editing them later on.

7

That’s fair @jethro . Perhaps this wasn’t your intent, but I’ve been using org-roam as a graph layer on top of Org for various use-cases, including journaling, pdf notes, code prototyping, and personal CRM.

The Zettelkasten method was only one of many use-cases for RoamResearch. Maybe if the package was named org-zettel, I wouldn’t have tried to do everything with it :wink:

8

Hello @dangirsh.

In my case, I use the usual wiki approach: tag the task with a link to the state (like “Pendiente”). So, in the node “Pendiente” there are backlinks to every pending task. This is not as efficient as org-agenda since there are no remote editing capabilities or filtering, but since I have not so many tasks opened, it’s enough for me.

I hope this solution could be useful for others… :slight_smile:

Best regards…