Org-roam major redesign

[bruce]

If I move the old db out of the way, org-roam won’t create a new db, no matter what I do (setup, sync-db, etc.).

[jethro]

@nobiot

Once you assign an ID to a file/heading, and save, org-roam-db-sync may or may not run automatically. I think it should run automatically via after-save-hook , but it hasn’t been consistent for me (Windows may suffer from some emacsql related quirks).

If you can reliably reproduce this, do file an issue.

@nobiot

Ensure to have a title to a file via #+title keyword (I have been sending issues in Jethro’s way related to my missing titles). I don’t use headings much, but they should not suffer from missing “title”.

Yes, file nodes shouldn’t have issues with missing titles, will fix this soon.

@bruce

All my files have those titles, so at least those should be picked up; right?

Only items with IDs (including files) will be picked up.

[jethro]

@nobiot

• Tags are now based on Org Mode tags ( #filetags and heading :tag: ). This is interesting, but for me using sub-directory as a tag has been a good way. (I don’t use Org Mode tags). Not necessarily asking for change, but perhaps you could consider a way for custmization like V1? At the moment, caching tags in DB is buried in a function so need to advise the entire function for this feature (if I understand Elisp correctly).

You can redefine org-roam-db-insert-file-node and org-roam-db-insert-tags to change how tags are collected if you really want to, but I advise against it. It’s unlikely that I’ll add back subdirectory tagging because I intend to use Org’s tag-related functions (e.g. forming sparse trees).

Also I’m trying to minimize areas of customizability especially around areas of what goes into the db, because the display and retrieval functions need to make solid assumptions of what’s inside.

@nobiot

Tags are now added to the minibuffer text string as completion-annotation. I have noticed that you cannot use it to filter in minibuffer. Perhaps your intention is to enhance the sql query to be able to filter in a different way. My first gut response was “inconvenience” — since I use sub folders as tags to differentiate work vs personal, and use minibuffer to filter when I want to focus on one or the other. Again, not necessarily asking for change; this is pure first impression I got from the UX change.

@bruce

@jethro - you’re using selectrum at al; right?

Would be really cool for node-find to exploit affixation and consult-buffer -like grouping.

The commands written are for the basic completing-read API, we’ll probably want powered-up versions of (org-roam-node-read) and (org-roam-ref-read) for helm, ivy, and consult, but those wont’ be shipped with Org-roam (feel free to have a go).

@nobiot

Existence of fiels in db is not sufficient for “nodes” to appear in the node-find (sorry in my previous post, the name of the function might have been incorrect). You need “nodes” (separate table) — only “ID”ed files and headings get recognized and put into the nodes table — as far as I have seen so far.

That is correct

[nobiot]

@bruce

If I move the old db out of the way, org-roam won’t create a new db, no matter what I do (setup, sync-db, etc.).

@bruce , where are you looking to confirm this? The default DB location now is under your emacs config directory. Check org-roam-db-location variable — it’s from a recent commit of V1. In my case, it’s “~/.config/emacs/org-roam.db“.

[bruce]

Yeah, I figured that out. I had assumed I could specify the db location, but it’s fine as is.

Now working as expected, once I added the org-ids.

Is it possible to pin the org-roam buffer to the sidebar, as with v1?

[bruce]

@jethro

The commands written are for the basic completing-read API, we’ll probably want powered-up versions of (org-roam-node-read) and (org-roam-ref-read) for helm , ivy , and consult , but those wont’ be shipped with Org-roam (feel free to have a go).

Gotcha.

edit: nevermind, I see the code now.

[nobiot]

@bruce

Is it possible to pin the org-roam buffer to the sidebar, as with v1?

I am halfway-there. I am doing this below. It does not refresh the content when I open a different note – that’s the next step.

(setq display-buffer-alist
      '(;; Left side window
        ("org-roam:.*"
         (display-buffer-in-side-window)
         (window-width . 0.25)
         (side . left)
         (slot . 0))))

;; this one  is for org-roam-buffer-toggle
(setq display-buffer-alist
      '(;; Left side window
        (".org-roam.*"
         (display-buffer-in-side-window)
         (window-width . 0.25)
         (side . left)
         (slot . 0))))

EDIT: Corrected the snippet above to work with org-roam-buffer-toggle – the buffer name is different between org-roam-buffer and org-roam-buffer-toggle. I don’t know regex enough to make one that works both cases (I think you can). My brain is fried now. I would love someone do it…

[jethro]

@bruce

Is it possible to pin the org-roam buffer to the sidebar, as with v1?

V2 by jethrokuan · Pull Request #1401 · org-roam/org-roam · GitHub adds this, with org-roam-buffer-toggle. This is the persistent buffer. org-roam-buffer still lets you create a “frozen” buffer for the node.

You’ll still need @nobiot 's snippet on customizing display-buffer-alist to configure how it looks.

[bruce]

… adds this, with org-roam-buffer-toggle

Great; I figured that was coming.

Not sure when or how you want bug reports, @jethro, but are you aware of this?

Error in post-command-hook (magit-section-update-highlight): (wrong-type-argument (or eieio-object class) nil obj)

I get it when turning on org-roam-mode, though not consistently enough for me to have a sense of details beyond that ATM.

[jethro]

@bruce

I get it when turning on org-roam-mode

When are you doing this? This is not a thing anymore (org-roam-mode is now the mode for the buffer)

Anyway I’ll need a full backtrace to figure out what’s going on.

[bruce]

@jethro

When are you doing this? This is not a thing anymore (org-roam-mode is now the mode for the buffer)

I think I figured out how to reproduce, and why a user would do this:

1. use find-file to open an org-roam note
2. recognize you are now in org mode, run org-roam-mode

That’s it.

Can you reproduce that?

[nobiot]

@bruce, I believe the function of org-roam-mode has changed. It’s only a mode for the backlink window; you don’t use it in the main Org buffer. You just use org-roam-buffer or (new one) org-roam-buffer-toggle from the Org buffer.

@jethro
By the way, org-roam-buffer-toggle works nicely .
I added another snippet above for this new toggle one (buffer name is different, so the snippet needs to be adjusted).

I like the way moving around in the same buffer, and the backlink buffer changes if you move across one node to another.

[bruce]

OK, thanks @nobiot.

But surely I’m not the only user that will try this? It shouldn’t generate an error message, and break things.

[nobiot]

I’d agree.

This was my suggestion a few weeks (months? I forgot) ago.

Your suggestion for org-roam-mode is to programmatically prevent an error message – that’s for @jethro .

I might start collecting some of these “migration” issues. This was your suggestion about wiki – essentially the same idea – that includes the difference in concepts, such as use of tags, file vs node, etc.

@nobiot

I could anticipate a lot of “migration” queries from V1 users, and even with new users starting directly with V2, as all the YouTube, blogs, and other instructions that community has put together so far will need to be revised. There will be void of community-based knowledge base for new users to fall back on, at least in the initial phase.

All this means there will likely be a lot of backlog questions to answer, leaving little breathing space for you and the project to keep going with V2.

Maybe I am thinking too much, and you can just swim through the new water with no problem, knocking off questions as they come. But I will be happy to do some early documentation of some sort, if it helps, of course (I cannot do this until at least end of March in the current plan of my other pressing engagements).

As an idea… We could start a wiki that collects V1 → V2 migration issues and solutions. It could be in this forum, or the wiki page in GitHub repo. It does not have to be a long-term thing, so just something that can be quickly put together would be good enough. Just a thought.

[bruce]

Definitely agree @nobiot.

To demonstrate what I mean by “breaking”, here’s what accompanies the error message.

[bruce]

@jethro

… adds this, with org-roam-buffer-toggle . This is the persistent buffer.

Just installed and configured this.

Looks good; thanks!

I assume I should be able to click on a link in the buffer and have the file open?

Still TODO?

Edit: also, I notice the buffer seems to conflict with Doom evil mode, in the sense I can’t use the kbs anymore there.

@nobiot

Corrected the snippet above to work with org-roam-buffer-toggle – the buffer name is different between org-roam-buffer and org-roam-buffer-toggle .

Maybe my brain is fried too, but looks to me it’s the same variable.

[nobiot]

@bruce

I assume I should be able to click on a link in the buffer and have the file open?

RET.
It should be modifiable with org-roam-mode-map (or some name like this).

[bruce]

@nobiot

It should be modifiable with org-roam-mode-map (or some name like this).

Yeah, you got the right name. Thanks again.

“Modifiable” so clickable with mouse, etc.?

And so that the buffer stays, but the “thing” opens in another buffer, as with v1?

Or those are more likely on the TODO list?

[mshevchuk]

@bruce

But surely I’m not the only user that will try this? It shouldn’t generate an error message, and break things.

@bruce There is absolutely nothing wrong with new org-roam-mode “breaking things”. Just try running fundamental-mode or magit-mode or actually any other major-mode not derived from org-mode while in an Org buffer. The things will look equally broken. They are not, however. Emacs tries its best to display your Org data in the new major mode. Accidentally, if you run org-mode in an Elisp buffer, for example, in your Doom config.el you’ll break break that buffer too. To “fix” the things just run the major-mode command appropriate for that data, e.g. org-mode for Org data.

Now let’s clarify why one shouldn’t even attempt to fix the accompanying error. The error is triggered when you run a magit-section-... command in the broken buffer. I could consistently reproduce it by hitting M-1, M-2, etc when trying to switch to another perspective – Doom binds the keys to +workspace/switch-to-N. But putting the buffer in org-roam-mode rebinds these keys to magit-section-show-level-N-all (org-roam-mode is derived from magit-section-mode) and since the buffer is broken – or rather org-roam-mode is broken by having been fed with wrong data – errors are triggered by commands which are not supposed to be run in this environment. You were probably hitting different keys and some of them would trigger magit-section commands and throw the error. Attempting to fix this error would be just a waste of time because it is absolutely normal under the circumstances – when using a thing in a wrong way expect that it will misbehave.

So the only problem here is (or rather will be) when v2 is rolled out and users start transitioning to it. People are used to org-roam-mode being a minor mode that can be enabled anywhere and many will get frustrated unless this major breaking change is very clearly communicated in some migration guide.

[mshevchuk]

@bruce

Edit: also, I notice the buffer seems to conflict with Doom evil mode, in the sense I can’t use the kbs anymore there.

Conflict is incorrect word here. It’s just Doom evil mode doesn’t have support for new org-roam-mode.