Org-roam major redesign

[mshevchuk]

@nobiot, yes this one

[gcoladon]

In the setup that @jethro linked to, I don’t see any mention of which branch to check out. In hid `packages.el’ file, he has a line like this,

(package! org-roam :recipe (:host github :repo "org-roam/org-roam"))

But I can’t tell how that is loading the v2 branch, unless he’s got that branch checked out locally or something?

I use Doom as well, and have a line like this in packages.el:

(package! org-roam :recipe (:branch "v2"))

But that doesn’t seem to get me onto v2. What are Doom users doing to get the latest org-roam v2 code loaded?

[jethro]

@gcoladon

But I can’t tell how that is loading the v2 branch, unless he’s got that branch checked out locally or something?

I do have the branch checked out locally, permanently. Maybe this will work?

(package! org-roam :recipe (:host github :repo "org-roam/org-roam" :branch "v2"))

[gcoladon]

Dang, that still doesn’t seem to do it. BTW I am using the existence of org-roam-find-node or org-roam-find-file as the test of which version I have installed. I still have v1 functions.

Where does one check out a github directory to in order for Doom to use that local code rather than the code at github? I am happy to give that a try. But I couldn’t figure it out by Googling for the answer.

[jethro]

@nobiot

It would be good that org-roam-mode-sections will be a customise variable with a default — e.g. the three sections. I am sure that’s your intention

Yes, something like this

@nobiot

I would appreciate documentation for the sections and the new org-roam-buffer in general
Again, just confirming what I believe you are already thinking of — I had to go into your dotconfig on GH to understand what are the acceptable values

Definitely need to work on the docs, I’m going to pull some ideas from the stuff you have done.

@nobiot

Ripgrep for unlinked reference section was a bit tricky
This is not directly about Org-roam V2 but about Ripgrep and OS package manager, I think; nevertheless, I felt it would be worth mentioning somewhere (especially when Ubuntu is popular):

Thanks, I’ll have a section on this in either the manual or a dedicate post somewhere.

@nobiot

For some reason, Emacs’s shell session recognised ~/.cargo/bin , so did Ubuntu’s Terminal app; not shell-command-to-string — I just assumed it is some sort of PATH setting in Emacs, but I didn’t dig any further.

Emacs shell uses the $PATH variable in your default shell environment (bash, for example), while Emacs uses the variable exec-path, they can be different.

[jethro]

For Doom, I checkout the v2 branch locally at ~/.emacs.d/.local/straight/repos/org-roam

You’ll then have to redo byte-compilation: M-x straight-rebuild-package org-roam, and restart Emacs, then you should be good.

[nobiot]

@jethro

Emacs shell uses the $PATH variable in your default shell environment (bash, for example), while Emacs uses the variable exec-path , they can be different.

Oh yes. Forgot to mention, it was after exec-path.
So it went like this (I think it was org-roam-unlinked-reference).

1. check exect-path for rg (via ~/.cargo/bin) –
2. check rg is pcre2 enabled –
3. shell out with shell-command-to-string fails with bash error rg is not recognised

I could not work out how to make step 3 work for ~/.cargo/bin) – no need to pursue; it’s not Org-roam thing. It was a bit of a bump in the way…

Widows seems to have its own issue with shelling out to Rigpgrep and org-roam-unreferenced-link and I removed it from the sections config for now – not priority for me, but I might see what the issues might be.

[gcoladon]

@Mark Thanks for sharing that – I also discovered doom sync -u somewhere, and did that along the way, but wasn’t sure if that was the change that got me onto v2. I also went into the .local/straight folder where org-roam was checked out, and when I did git status there, it said I was on v2 but was 17 commits behind, so I did a git pull there. And then several types of restarts and reloads… and the straight-rebuild-package thing too… and now I’m on v2

[nobiot]

see if you can do M-x org-roam-setup. Also org-roam-disgnostics should display the version as 2.0.0.

[nobiot]

Not sure why. Have you loaded org-roam.el?

[nobiot]

@Mark,

My suggestion has been the same since the Actual State thread. Please take a small step at a time.

If you have no reason to stay with Doom, please at least consider using vanilla Emacs with minimal config. I have written a step-by-step guide. It’s for Windows, but the majority of config should be applicable to macOS and Linux. After I have written it, I switched back from Doom to vanilla; no looking back.

Some people have different opinions and experience, which is healthy as a community, but I will still say that Doom is not beginner friendly, even compared with vanilla. This is so especially when you don’t have a programming background or Vim background (I don’t know if you have either).

For the same reason, I would suggest to refrain from using V2 if you are still learning Emacs. V2 is still in early alpha and the community has just begun trying it out — that is, we are not collectively ready to support beginners of Emacs and Org-roam on V2. For example, Doom does not support V2 yet (even Jethro, Doom user, uses personal config.el to enable V2).

[Mark]

@nobiot

Thanks for this. I’ll start from scratch…

[bruce]

@gcoladon

Dang, that still doesn’t seem to do it.

I think branches aren’t working with doom and/or straight. I tried using it a few weeks back with another package, and I could only get the default branch to work.

`:branch` argument not working as expected in package recipe

[mshevchuk]

@bruce

I think branches aren’t working with doom and/or straight. I tried using it a few weeks back with another package, and I could only get the default branch to work.

This worked fine for me to switch to the “v2” branch:

(package! org-roam 
  :recipe (:host github :repo "org-roam/org-roam" :branch "v2"))

If the branch is not picked up automatically by Doom when switching from another branch, one can always purge the repo from $DOOM/.local/straight/repos.

[bruce]

@mshevchuk

… one can always purge the repo from $DOOM/.local/straight/repo .

Thanks for the tip!

By “purge” do you mean rm -rf ...?

EDIT - it actually installed fine this time; no need to purge.

I think maybe we need a wiki page for people happy to test, like me.

I commented out the roam feature, and added jethro’s config to mine, and adjusted the path, then synced the db; also ran the setup command.

It can’t find any nodes though, nor will the buffer work (says no node at point).

Do I need to do something to update the files?

[nobiot]

Are you using org-id to assign an ID to either file-level property drawer, or headings? No node probably means the IDs are not correctly recognized for files and/or headings.

Ensure to have a title to a file via #+title keyword (I have been sending issues in Jethro’s way related to my missing titles). I don’t use headings much, but they should not suffer from missing “title”.

Once you assign an ID to a file/heading, and save, org-roam-db-sync may or may not run automatically. I think it should run automatically via after-save-hook, but it hasn’t been consistent for me (Windows may suffer from some emacsql related quirks).

Once node IDs are stored in the db, you should be able to see list of the nodes via org-roam-node-find (not -files any longer).

To open the backlinks buffer (no longer a side-buffer by default), run org-roam-buffer (on an ID’ed headline or ID’ed file). To get it to work, some config is required for the “sections” to appear — as you have copied Jethro’s config, I’d assume you have seen it.

This should do it for now. I have tested these features in my test Ubuntu VM; now trying the same settings with my real notes on Windows, reporting issues along the way. I have managed to identify some problems with missing titles to display the backlinks buffer just today.

[bruce]

Thanks much for the thorough reply!

@nobiot

Are you using org-id to assign an ID to either file-level property drawer, or headings?

Not AFAIK. I was using whatever standard v1 org-roam approach to creating the files.

EDIT - yeah, adding org-ids does solve that. But I have to do it for the file too.

Ensure to have a title to a file via #+title keyword (I have been sending issues in Jethro’s way related to my missing titles). I don’t use headings much, but they should not suffer from missing “title”.

All my files have those titles, so at least those should be picked up; right?

[nobiot]

@jethro , some additional feedback before I forget (let me know if you prefer a larger chunks, rather than quick many waves).

I notices two things around completion in minibuffer via org-roam-node-find.

1. Tags are now based on Org Mode tags (#filetags and heading :tag:). This is interesting, but for me using sub-directory as a tag has been a good way. (I don’t use Org Mode tags). Not necessarily asking for change, but perhaps you could consider a way for custmization like V1? At the moment, caching tags in DB is buried in a function so need to advise the entire function for this feature (if I understand Elisp correctly).

2. Tags are now added to the minibuffer text string as completion-annotation. I have noticed that you cannot use it to filter in minibuffer. Perhaps your intention is to enhance the sql query to be able to filter in a different way. My first gut response was “inconvenience” — since I use sub folders as tags to differentiate work vs personal, and use minibuffer to filter when I want to focus on one or the other. Again, not necessarily asking for change; this is pure first impression I got from the UX change.

[nobiot]

Yes, the #+titles get picked by db-sync (which is also called in org-roam-setup). I am now away from PC and cannot give you more, but files is a table in the cache, so you can inspect if all your files have been added to the cache.

Existence of fiels in db is not sufficient for “nodes” to appear in the node-find (sorry in my previous post, the name of the function might have been incorrect). You need “nodes” (separate table) — only “ID”ed files and headings get recognized and put into the nodes table — as far as I have seen so far.

@jethro will correct me if I am gravely mistaken

[bruce]

@jethro - you’re using selectrum at al; right?

Would be really cool for node-find to exploit affixation and consult-buffer-like grouping.

I’m not sure how the latter works (probably via the metadata category of the candidate, I’d guess), but it should be trivial, and would make it easier to segment files from sub-file nodes when browsing.

I thought I saw on the marginalia tracker that you added an annotation function and category, but I’m only see the node titles.