V2 transition: org-roam-node-find questions + empty org-roam-buffer

Today, after a package-update, I somehow ended-up with v2 and gave it a spin. The experience, to be honest, was a bit rough. Here are some unsorted questions:

  1. Is there a way to limit org-roam-node-find to files? I have more than 1400 notes, with many more headings - of which some are referred to in other notes. This creates about 10000 nodes, some with extremely meaningful titles such as “Introduction” etc.
  2. Is there a way to show tags or filetags in org-roam-node-find?
  3. Is there a way to restore the last-modified order in org-roam-node-find? If yes, why is that not the default?
  4. The line height in org-roam-node-find seems off (the width, too), see below:

  1. The migration script complained about a backslash \ in one note and ended. Not sure if it finished.
  2. And finally, the org-roam buffer is empty - despite org-roam-node-find working properly.

Feeling like an old men yelling at the clouds, sorry for that. It is just that v1 works so well now and I am a bit sad to see it go.

1 Like

I switched to using ivy (from the default emacs completion) and the node find shows in most recent order. FYI my setup is

(use-package ivy
  :straight t
  :init
  (setq ivy-use-virtual-buffers t)
  (ivy-mode 1)
  :config
  ;; (setq ivy-initial-inputs-alist nil)
  (setq ivy-use-selectable-prompt t)
  (setq ivy-count-format "(%d/%d) ")
  (define-key ivy-minibuffer-map (kbd "C-S-<return>") 'ivy-immediate-done)
  )

(use-package orderless
  :straight t
  :after ivy
  :config
    (setq ivy-re-builders-alist '((t . orderless-ivy-re-builder))))

;; From https://github.com/daviwil/emacs-from-scratch/blob/master/show-notes/Emacs-Tips-Prescient.org
(use-package ivy-prescient
  :straight t
  ;; :after counsel
  :config
  (ivy-prescient-mode 1)
  ;; Remember candidate frequencies across sessions
  (prescient-persist-mode 1))

I am facing exactly the same problem. I had files within which there were headlines and thus now nodes. I have 1500 notes.

@laotang @shrysr
I understand that it’s not ideal.
I don’t have answers, but here are some ideas.

Too many nodes

I would set IDs for only headlines that I want to be nodes. If a headline does not have an ID, Org-roam does not cache it as a node.

Now you already have IDs for 10,000 headlines, it’s not practical to manually change them.

Three ideas here. All require some work and some additional scripts (they should not be difficult to craft). Not ideal; not sure if there are a better way. Any preference?

  1. Have a script to remove IDs from headlies
    Now… You have some headline nodes that are linked by other nodes. This can be detected by database query. If the number of such nodes is small, I would deal with them manually after removing all the IDs.

  2. Similar idea. You can add property :ROAM_EXCLUDE: t to headlines. This would be an option when you need all the headlines to have an ID for some reason – I would prefer Idea 1 above. This option would also need to take care of the headlines that are linked in the same way as Idea 1.

  3. Create a custom version of org-roam-node-find (and other “find” functions). They can take a custom filtering function.

Tags in minibuffer for completion

Tags should appear in minibufffer. Variable org-roam-node-display-template controls the fields that appear in minibuffer, and the default contains tags: ${title:*} ${tags:10}.

It is possible that Helm or your particular way of configuring it may influence the behaviour you observe.

Sort order

Org-roam’s default sort order is based on file’s modified time. It is set in variable org-roam-node-default-sort (the default, file-mtime; that is, files modified time).

I suspect that you ran the migration script, which modified all your files and thus overrode their original modified time.


I should be able to help you with some scripts regarding too many nodes (not necessary a script that would perfectly work for you but some snippets, etc.). I think Idea 1 is the cleanest.

1 Like

Thanks for the detailed reply and the assistance! I will give v2 another try next weekend, when I have more time at hand.

Too many nodes

I will try Idea 1. For a while I was already using org-id in the past, so there might be a bit of “baggage” from those days.

No tags

Helm not working properly might also explain the line height and width issue (see screenshot). Is anybody else running Helm and v2?

Sort order

That is correct. I ran the script and it modified all files. Is there a way to improve this script to maintain the “original modified time”?

Empty org-roam buffer

Sorry to ask, but do you have any idea why the org-roam-buffer was empty?

Thanks for idea and the details from your setup. If I cannot fix Helm this may be the route to go. @nobiot might be correct, though: the transition script probably changed all “last modified” dates, hence the chaos.

If it is completely blank without even the title of the node, It’s probably because migration didn’t complete properly.

I’d do two things:

  1. toggle-debug-on-error and see if you get error message when you call org-roam-buffer-toggle
  2. Make a backup copy of the db file and temporarily set a new org-roam-directory to check if the Org-roam itself is working

I think Org-roam as a package is working (because it works on my end) so the second one may not be necessary.

I think it’s related to your data. If my assumption is correct, you would need to find out what part of the data is causing this problem. Check 1 above hopefully gives you some clue. If not, I’d park this issue for now and focus on the migration script to finish without an issue, and then come back to it.

1 Like

thank you for the response and ideas. They make sense. I would be happy to help craft a solution to the best of my limited lisp skills.

I have actually always wanted the headlines to be shown as nodes but would want to identify that they are … child nodes of a file. I.e say I have a file Emacs tips and tricks file or Python notes file under which there are multiple headings or a draft blog post which has already been crafted using ideas and notes from back links. And there are also several atomic notes referring the file itself. So the file is a referenced node but so are the headlines within but not always. So in some cases - I could convert the files into atomic notes but in others Like longer articles they cannot be separated.

I suppose one way might be to tag these longer notes somehow and remove org Id for the headlines within those files for the headlines that are not linked as you have outlined in idea 1. Or use idea 2 a bit. I would need to try review my notes to see how many such files I actually have with the problem. Let me get back to you.

Hmm - I am happy seeing internal headlines as nodes - but can I also see or differentiate a parent file? Iirc This used to be possible with org-brain while inserting a link brain: since nodes were explicitly defined as child or parent. If the mini buffer could show the file title with node… I suppose it also helps with context. It might solve the problem in a way.

2 Likes

I agree with the advice. I initially had trouble with the migration and only a partial list of nodes. Toggle debug on error helped me find that the issue was a problematic note or node which was actually empty, but with an ‘empty’ headline. Deleting this note enabled migration to complete.

I use helm - and I see tags in my mini buffer but - the view is similar to the screenshot above, ie the lines are broken or split. When I use counsel - the lines are not broken, but I do not remember seeing tags. I will check.

1 Like

@nobiot @shrysr

Will try toggle-debug-on-error, thanks for the hint. The issue with tags & Helm appears to be also mentioned in this Github exchange.

You might like to look at what HieuPhay has done for their minibuffer.
There is also a collection of tips that you might like to use on the “Hitchhiker’s Guide” wiki; I believe HieuPhay took some ideas from them.

From the left: number of backlinks, note type (a special type of tag), node title (carefully highlighted by node hierarchy), and tags (not shown in this capture)
Source on Slack
His config.el.

1 Like

I use helm - and I see tags in my mini buffer but - the view is similar to the screenshot above, ie the lines are broken or split. When I use counsel - the lines are not broken, but I do not remember seeing tags. I will check.

I believe this issue is because some versions of helm/emacs (I haven’t had a chance to narrow down the problem yet) ignore the value of the 'invisible property added to the tags that are appended to completion candidate strings here. As I understand it, the tags are added as invisible parts of the string so that your search can match against the tag, but it shouldn’t be displayed in the helm search buffer (since it is invisible). Interestingly, this wasn’t a problem for me on emacs 27.1, but when I updated to 27.2 it started happening. I haven’t figured out what is causing the problem, so I made this change to org-roam to remove the extra “invisible” tags on the new lines as a temporary workaround until I figure out if there is a problem with my config.

1 Like

For no tags with Helm, try setting variable org-roam-node-display-template without the length specifiers like this:

(setq  org-roam-node-display-template` "${title} ${tags}")
;; default is "${title:*} ${tags:10}"
;; remove :* and :10

If you are on Emacs 27.2, you might get tags duplicated (@mattc, you might like to send a PR with your change, perhaps?)

If you get this problem, remove the tags part of the template like this:

(setq  org-roam-node-display-template` "${title} ") ;; Add space after the closing bracket "}")

If you are on Emacs 27.2, you might get tags duplicated (@mattc, you might like to send a PR with your change, perhaps?)

I was holding off because I wasn’t sure whether my fix was the best way to address this, but you’re right that it would be good to at least get this conversation going on GitHub, so I opened an issue and a PR with my potential fix.

1 Like