Org-roam major redesign

I think the easy way to do this is to search :ID: in Git. Since the
smallest unit is a headline, this will be a durable solution for
tracking the creating time of the notes. To gather the statistics of
everyday work, I would prefer to run a periodic script to track the
number of :ID: with respect to the latest change.

For a long time, I also use another metadata: modified time. However,
I do not have any practical usage over the past year. Further thinking
about this meta data, I think it is possible to track from the
git-blame function.

A quick and dirty workaround script for this trival task:

git diff ~/PATH/TO/ORG-ROAM-NOTES/ > ~/.emacs.d/tmp.log 2>&1
grep ':ID:       ' ~/.emacs.d/tmp.log | wc -l

will give how many nodes created.

For v2 something like what you suggest is needed. Creation time will need to be stored as metadata within a headline drawers. Then during parsing, headline drawers are checked for timestamps. A specific keyword (like created) can be used to specify creation time. I believe org ql has a similar feature.

I implemented parsing #+created for the current version. I am willing to implement for V2.

1 Like

Single line update from V1 to V2 that will update my daily tracker:

 #!/bin/bash
 
-TODAY=`date +"%Y%m%d"`
-ls -l ~/Dropbox/myNote/zk/ | grep "$TODAY" | grep -v "~" |  awk '{print $9}' | wc -l > ~/.emacs.d/nyan-mode/log.today_effort 2>&1
+cd ~/Dropbox/myNote/zk/ && git diff ~/Dropbox/myNote/zk/ > ~/.emacs.d/nyan-mode/tmp.log 2>&1

When I hear the news about V2, I tend to write fewer new files, but write more headlines. Now, back to the previous state where I can “fool” myself.

I love the idea. If I understand it correctly, this means every subtree with an ID will be cached in org-roam db and treated as its own atomic note.

Does this mean that each node/subtree-with-ID will appear as its own node in graph (via org-roam-server)? That would be fantastic!

I’ve been using org and org-id for 1.5 years and have highly inter-linked subtrees across 4-5 very long org files. It sounds like org-roam v2 will be able to take this structure and generate graphs through org-roam-server.

Will this unify roam-tags with regular org-tags?

How will this affect org-id? I believe org-id currently catalogs the names of org files and the IDs they contain. It is kept in a file (.something file in .emacs.d, I believe) to serve as a look-up table (I assume).

Thanks for building org-roam v2. I look forward to trying it when it is available (is it already?)

I believe org-expiry has some auto-stamping method for inserting creation date in the properties drawer of a newly created note. M-x org-expiry-insinuate turns on the automatic insertion of creation date. I’ve often wondered that this could be refactored to also insert modification date.

That is correct.

Yes, but org-roam-server needs to be updated for the new db schema.

Issues with processing large org files still remain (for now), but it will support the structure.

Yes, Tags will be specified via Org tags (and FILETAGS for file-level nodes)

Unaffected. Org-roam duplicates that information, and can choose not to use it.

I’ve personally been using the V2 branch for a few months now and making changes where necessary. Works fine for the minimal subset of features I rely on.

That’s encouraging! I will probably wait on V2 because my workflow relies on org-roam-server and graphing. Based on your response above, it sounds like that aspect is in development phase. Thanks for your hard work on this package.

Hello,

I’m looking for an adequate tool for note-taking (Zettelkasten method) to be used on my Linux (Debian) machines…tried several of them - Neuron looks interesting, but a bit too exotic - Haskell/nix-install etc. Installed Zettlr & Joplin - they look nice, not perfect, but I prefer not to depend on Electron apps and prefer using real editor (Joplin does support it though). There are Foam & Dendron - the latter is very interesting, but simply do not want to use VSCode editor and OniVim2 is not ready having no support for tree/graph views.

Org-roam, probably, does everything what I need, but I’m not quite Emacs/org-mode user and recently stumbled upon nb which is very cool and markup-agnostic, so I wonder about your rationale to keep (part of) your notes in *.md?

For task management I do use Taskwarrior, Thunderbird for email - iow. not org-mode, so not sure if using org-roam just for note-taking would provide optimal setup and/or whether there is some ‘cons’ in using org-roam with the Markdown?

It’s all in retrospect for me, so I could come up with any reason to justify why I’m using markdown files now and try to say something smart :wink:

I tried Neuron and Zettlr; it’s fun to try out different tools. Md-roam “sort of” works nicely with them. This might change when I adapt Md-roam to V2 with ID-based links. I will see how this will pan out soon.

It’s probably more useful to think “Why have I not stopped using markdown with org-roam?”. Two honest reasons:

  1. I use iA Writer (markdown editor) on iPad and iPhone to take notes (especially websites, as I tend to read web articles more on iPad/iPhone than on PC)
  2. I learned how to code elisp through developing Md-roam by studying how Org-roam is programmed – and it turned out to be a lot of fun. I could not code elisp before that. I do not code for living, so it was a discovery for me

So… It’s all person specific to me. The reason why I haven’t quit markdown with Org-roam is that it poses the least friction with my life away from PC, it’s fun, and perhaps the joy of discovery still lingers on (like a chick hatching out of an egg identities its mother).

I think this is looking at “pros and cons” from a different angle. I don’t weigh them with different tools. It just so happens that I found a tool set that is frictionless with my life and fun; I just stopped searching for “better”.

I still “shop for a better tool” because it’s fun. But I would also want to spend more time using the tool to make something.

EDIT: Since you asked about note-taking use of Emacs… Not sure what will be optimal for you; just sharing what I do.

I use Emacs mostly for note-taking for both work and private. I also use it to write long prose – like my Org-roam setup guide for Windows and write a draft of a book. That’s it. Emacs for me is for note-taking + converting the notes into longer material – “text editing” without programming (except for Elisp).

I mostly use iPad, Apple Pencil, and a drawing app for day-to-day task management (in place for pen and paper). MS Outlook inbox is my GTD inbox for my corproate work. I use Org Mode only a little bit to organise my personal projects (mostly Emacs packages like Md-roam; these projects are done in PC) as an experiment to see if it works.

I am not sure if I would like to put everything in my PC, probably because I used to travel a lot pre-COVID.

1 Like

@nobiot, much of your organizing approach sounds a lot like mine, right down to using a pen-based app. Do you try to integrate that in any way with org-roam or org-mode? I use OneNote a lot, but it’s always kind of “off to the side” – I’d like to mix it in somehow with Emacs tools.

This has become long after writing it – it’s an interesting topic (“transition” from OneNote). We might like to post separately, as it is going off topic from V2 :wink:

I still use OneNote in parallel. It has been (still is) integral to my work; the integration from Outlook for meeting notes, iPad app for handwritten notes, search including the handwritten notes, etc. etc.; these are just so good if your daily work life is embedded in MS Office Suite.

I decided to try out Emacs and Org-roam for work throughout this year. This was triggered by the fact now that I have too many OneNote items (I think it’s more than 5,000) that the API started to not work when you create meeting notes from within Outlook (one of the major benefits gone like this).

I am still experimenting and started to identify some “key gaps” (not very firm “requirements”, really — just some points of frictions). Some are not functional; it’s relation between my adaptation and tool’s concept.

  1. Search text
    Solved with Ripgrep (within Emacs)

  2. Image pasting (for screen shots)
    Solved with my custom function for Windows

  3. Notes on “recurring” events, such as meetings on a specific theme / topic / issue
    Creating a shell of a meeting notes from Outlook calendar entry (with participants, date, etc.) is not achievable – not worth trying in my view. I think this is where using the classic bullet points style of Org Mode might be more useful than the “atomic” nature of Zettelkasten notes; that is, I can create a note on a theme or meeting, and use each heading as an event with a time stamp (e.g. Meeting on 2021-03-20), etc. etc.

  4. Handwritten notes, and search
    I am thinking of something like this. Not sure if this is going to work, but it should be fun to try:
    Org/ vs. org/org-roam/ directories OR am I supposed to link to files outside of org-roam? - #7 by nobiot

2 Likes

Karlicoss experiments are very interesting in this case…

1 Like

Thanks! I’ve concluded to settle on Emacs & org-roam. :slight_smile:

This does not apply here.

I’m glad to hear it - that’s something I still have to tackle. :wink:

After some consideration I’ve concluded to focus solely on Emacs as editor and org-mode markup…Emacs is for quite some time around and hopefully will continue so. Otoh, having experience with several markups (markdown, rst, Asciidoc) I believe that org-mode is rich-enough as well as readable to serve as single-source markup for my own needs.

Lastly, although I like Taskwarrior for task management, it does not handle recurring tasks properly, so it’s another reason to stick to org-mode. :smiley:

1 Like

Hi, @jethro. I wanted to test the new version in order to get an insight of what needs to be adjusted in Org Roam BibTeX to keep it working.

So far I spent quite a bit of time trying to make org-roam-buffer show the backlinks. With my setup of Doom with no extra config in a separate VM, it appeared that the following:

(require 'org-roam)
(org-roam-setup)

was not enough.

To make the Org Roam buffer actually work I had to:

(require 'org-roam-backlinks)
(require 'org-roam-unlinked-references) ; to be able to follow links
(add-to-list 'org-roam-mode-sections #'org-roam-backlinks-insert-section)

I assume the same should be done for ref links to show up.

Is it the intended way to load the org-roam-buffer functionality, which is still in transition toward org-roam-modules or similar, or did these pieces just did not make it yet into the branch?

Thanks for trying v2 out so early, appreciate any feedback.

Yeah, the setup is pretty involved now, I haven’t decided on what would be good defaults, so I’ve left this in my config for now:

2 Likes

I feel that these three sections are good default. Feedback (tested on Ubuntu Linux via virtual machine on Windows):

  1. It would be good that org-roam-mode-sections will be a customise variable with a default — e.g. the three sections. I am sure that’s your intention :slight_smile:
  2. I would appreciate documentation for the sections and the new org-roam-buffer in general
    Again, just confirming what I believe you are already thinking of — I had to go into your dotconfig on GH to understand what are the acceptable values
  3. Ripgrep for unlinked reference section was a bit tricky
    This is not directly about Org-roam V2 but about Ripgrep and OS package manager, I think; nevertheless, I felt it would be worth mentioning somewhere (especially when Ubuntu is popular):
  • Ubuntu (20.x.x) apt does not provide Ripgrep with pcre2 option — the scoop package manager on Windows does (!). I resorted to compiling Ripgrep myself (easy enough, but…)
  • PATH setting was a bit tricky. shell-command in Emacs did not recognise rg in custom path (in my case, symlinked from ~/.cargo/bin). I had to do it from /usr/local/bin. I believe this is an Emacs quirk. For some reason, Emacs’s shell session recognised ~/.cargo/bin, so did Ubuntu’s Terminal app; not shell-command-to-string — I just assumed it is some sort of PATH setting in Emacs, but I didn’t dig any further.
2 Likes

I haven’t tested it thoroughly but the general impression is that the engine is quite solid. I find both the partial transition to the object-oriented model and the use of magit-section to be very good decisions. The code is much cleaner and it is much easier to read after having familiarized oneself with the basics of structures and methods. I’d argue, to maintain and extend/scale too. I’ve run into one minor wrong-type-argument error, will fix it in my private fork and submit a PR some time later, maybe after catching a few others.

2 Likes

@mshevchuk, is this the same issue?

1 Like