How to toggle the appearance of domain A rather than B in org-roam-node-find completion interface?

Imagine I have two COMPLETELY DIFFERENT domain areas, or may be three or more, but to keep my question simple, which I am working on in PARALLEL in org-roam following the Zettelkasten method. When I do org-roam-node-find C-c n f to look for a specific node I created the completion interface would be polluted by nodes from domain A and domain B, which is unhealthy to the sanity of my shrinking brain. How to achieve this separation, given that I am interested in both A and B but on a condition of being separated at the completion interface?
It is not that when I am done with A I will be working with B, no, I need to work on them both at the same time but each for a totally different purpose. To see all nodes belonging to both of them showing up is what driving me crazy. I hope someone would relate to that obnoxious feeling.

Ideas or thoughts that came to my mind to achieve this but I dunno if they can see the light or worth anything:

  1. Creating different directories and capture templates cannot be a solution, as they share the same database, hence all nodes are mixed. Unless somehow there can be a suppression of a subset of nodes that belong to directory X of one of the domains.

  2. Tags, again given that somehow there is a way to suppress a subset of nodes that have certain tag(s) pertaining to a certain domain.

  3. IDs, I know they are randomly generated, but randomness can be programmed too. Give randoms to A different than randoms to B. It sounds like a crazy idea already.

1 Like

I would suggest option 1. The following can be easily configured for org-roam-node-find.

I can give you more concrete guidance with code examples tomorrow (or on the weekend) in my time zone if you are keen. In the meantime, you can look up related posts here and wiki entry in the GitHub repository — I’m out and about only with my phone at the moment.

Glad to see your kind response. Just can’t wait.

I came across (User contributed tricks) in Github as you suggested, and it seems to me the nearest thing to my question is (filtering by subdirectory) code snippet:

(cl-defmethod org-roam-node-directories ((node org-roam-node))
  (if-let ((dirs (file-name-directory (file-relative-name (org-roam-node-file node) org-roam-directory))))
      (format "(%s)" (string-join (f-split dirs) "/"))
    ""))

(setq org-roam-node-display-template "${directories:10} ${title:*} ${tags:10}")

I use use-package to manage emacs packages. I put the code in the :config, but got confused as how to bring the filtering thing into practice. The only benefit after implementing the above snippet is that I can now see which nodes are belonging to what directories they come from, but still no separation, forgive my declining IQ, I mean all of the nodes from A and B would appear mixed together.

Not sure if I understand your question fully, so I want to ask if this would work for you. When running org-roam-node-find, org-roam figures in which domain you are on automatically and displays only nodes which are on that domain. If you switch domain and re run it it will be reevaluated and show the nodes of the other domain.

I am pretty sure I can write the code that does this (and hopefully it is performant) but I am not certain if it will help you.

For a brief explanation of the code (I don’t mind writing it myself, but you should know the idea), it is trivial to find a node’s backlinks (org-roam already does this for the org-roam buffer) and then to collect the amount of backlinks each of those has. If we ask for the maximum number of backlinks, the node returned should be the index node of that domain. Then query for the backlinks of that node and run org-roam-node-find filtered to show just those. The implementation shouldn’t take me more than a few hours so if you are interested I can probably have it ready by tonight.

There are two things I would like to point out.

  1. The subdirectories you display with the snippet can be used to filter the nodes in the minibuffer.

  2. In addition, you can create a custom commands that pre-filters the list by subdirectory

1. Filtering in the minibuffer

See the screen shots below. I have created subdirectories dirA and dirB and placed some Org files in them (Figure 1). As you can see in Figure 2, you can type “(dirA)” to see only the files in dirA. For this “vertical” completion in minibuffer, I’m using the built-in fido-vertical-mode in this example. Vertico and Ivy are two very popular options, and I believe easier to use than the built ones like fido-vertical-mode. My personal choice is Vertico (available in ELPA).


Figure 1.


Figure 2.

2. Custom commands to pre-filter the list in minibuffer

This method may be a little more advanced, but I believe it is still easy enough. You can create a custom commands, each corresponding to one of your domains (subdirectory).

See Figure 3 and code below. The code only shows an example for dirA but I trust that you will be able to extend it to other subdirectories.


Figure 3

(defun my/org-roam-node-find-dirA ()
  "Show list of `org-roam-node-find' only under dirA."
  (interactive)
  (org-roam-node-find nil nil
                      (lambda (node)
                        (file-in-directory-p
                         (org-roam-node-file node)
                         (expand-file-name "dirA" org-roam-directory)))))

Exactly. So when I work on one domain (A or B), I want to see nodes related to the domain I am working on, while suppressing (or filtering out) all other nodes that are related to the other domain. So typically I have subdirectories under my org-roam-directory like /domainA-literature-notes and /domainA-concept-notes but I have also /domainB-literature-notes and, of course, /domainB-concept-notes. So one way is to filter caching of nodes based on the user input what directories under the main roam-dir would like to cache right now, the user can pre-define a schema or more to follow in the init setup file. A scheme can be defined or customized with a set of subdirs a user want to include. For example, Scheme 1 (/domainA-literature-notes and /domainA-concept-notes) and Scheme 2 (/domainB-literature-notes and /domainB-concept-notes), we can go on and on with other Schemes, I will leave it to your wild imagination. This approach will lend flexibility to the user experience, where users can come up with flexible schemes (elisp list of schemes with a list of associated subdirs) having interesting even overlapping combinations may be of subdirs suitable to their needs, or I may add, changing needs in life.

Another use case which I felt the need for is the following, when we work on Zettelkasten system we all should be aware of the common pitfall of using org-roam as an archival library system rather than a think-tank, which is fine, not against that, but make no mistake that was not in the mind of ZK system originally, the system was meant to be used for developing notes rather than dumping them. But how about taking the best of the two worlds in emacs? Here comes your code of filtering domains (or shall I say subdirs, there may be another superiour idea to use for filtering other than creating subdirs, but bear with me now). I use the notes dumping myself in org-roam because org-mode is attractive and emacs ecosystem is irresistible so why not dumping reference notes using it rather than doing it in a reference management software like Zotero, which by all means cannot attain to be another emacs, but at the same time we were told to put all our subdirs under the one and only org-roam-directory, and that put us in an awkward situation if we are to abide by the Zettelkasten mantra, not mixing dumping with developing notes. So filtering here can be life-saving, to filter out the subdir of the references of each domain and as if we are asking org-roam to blind the eye on our dumping tendency for a while so that we can enjoy an orthodox ZK experience. Just to remind the reader we are talking about two or more totally different domains here that in no other way we want to see their nodes mixed.

I assume the following workflow for developing ideas and writing:

A) fleeting notes - taken care of by org-roam-dailies and org-agenda, etc
B) reference notes - this is the dumping part of notes. I like to do it in org-roam rather than in Zotero.
C) literature notes - no dumping but developed notes sourcing B.
D) concept notes (aka permanent or der Zettel - the Note) sourcing C.
E) project (articles, blogs, seminar, etc) sourcing D.

Great! One more thing, is it possible to let the function accept multiple strings, i.e. multiple dirs? And what if we want to HIDE specific dirs rather than showing them? Thanks.

If you use a nested structure with subdirectories as mentioned here, then @nobiot’s solution is exactly what you need. Its a very easy filter function that shows you things in one directory. What I was proposing was for a more Zettelkasten-y system where all files are in the same directory. As this is how my system works, there is a decent amount of taxonomy using some tags to classify important notes, one for dailies and one for reference notes.

In such a system, the only real way to classify notes is through backlinks (which is the intended way of the system and why I started using it like that). So for that you need a more intricate sorting function like the one I mentioned which tries (not without failure however) to find the index of said topic and then the backlinks that stem from it. However, the less interconnected the notes are (being in different domains as you said for example), the less the chance of failure is.

Since you do seem interested in a more Zettelkasten-y approach, I could try to write the code for something like this.

I was about to adopt this approach (one-for-all) directory in the beginning, but realized that separation of states would be problematic down the road. So I opted for the subdirs approach which is also seen in many setups, and I think, for a very good reason like the one we discuss here. Because you are ultimately going to see everything gathered in one database anyway, so why not give yourself a leeway for the future using a setup like the one @nobiot helped with to do custom separation, which I think is healthy and less taxing in silico. Imagine you have 2000 nodes in domain A and 1000 nodes in domain B. Using one-for-all directory would mean caching 3000 nodes and then narrowing down the matches by well-chosen tags, hopefully, and for this you need a quite decent tagging system to take care of the job, but if you divide your nodes into subdirs, you can then with little code as above, achieve the same separation headless without bothering a fuzzy search.

Yeah, it is definitely a good approach, especially for well separated topics. I am using org-roam mainly for taking notes for university (there are also some things about programming and Emacs there which are irrelevant but in their own index) so there are a lot of interconnected topics, meaning that if I were to use this approach, I would reach one of the problems the Zettelkasten is supposed to solve, which is this file needs to be in more than one directory at once. But it is true that directory separation makes filtering ten times easier.

How familiar are you with reading programming languages in general and Emacs Lisp?

This below works on my end.

A couple of notes on how it should work and limitations:

  • You will need to manually maintain the list of subdirectories directly in the code; see where it is done close to the top `(list “dirA” “dirB” “dirC”)
  • These subdirectories need to be relative to org-roam-directory – “dirA” is directly under org-roam-directory. If you want to specify a farther descendant, I think you can do “sub/sub/sub-directory” and so on, but I only tested direct subdirectory (children)
  • In order to select multiple directories, once you select a directory e.g. by TAB, then you type “,” – that should be Emacs default. The exact key to press may be different depending on your completion (Vertico, Ivy, etc.). It may be already customized differently on your end
  • You can use this command to exclude nodes in directories. To do this, you can pass universal-argument. This is done with C-u M-x <command-name> by default. Again your Emacs may be customized differently

I think it’s as far as I can get you. It will be too much for me to make it any more generic – but you can ask questions about how it should work.

(defun my/org-roam-node-find-in-selected-dirs (&optional exclude)
  "Show list of `org-roam-node-find' only in selected directories.
You can pass EXCLUDE to exclude files in the selected directories
with using `universal-argument' (\\[universal-argument]).

This command assumes that the subdirectories in the list are
relative to `org-roam-directory`.

Directly change the list in the program."  
  (interactive "P")
  (let* ((dirs-selected (completing-read-multiple "Select dir(s): "
                                         (list "dirA" "dirB" "dirC")))
         (abs-dir-names (mapcar (lambda (dir)
                                  (expand-file-name dir org-roam-directory))
                                dirs-selected)))
    (org-roam-node-find
     nil nil
     (lambda (node)
       (let ((file (org-roam-node-file node))
             (result nil))
         (dolist (dir abs-dir-names result)
           ;; Ensure to do all the dirs in
           ;; abs-dir-names, but once result is t,
           ;; don't override it.
           (unless result
             ;; `file-in-directory-p' seems work to when the file is
             ;; directly under the directory. It returns nil when the
             ;; file is a farther descendant. `string-prefix-p' can be
             ;; used instead if both dir and file are absolute file
             ;; names, and see if file is a farther descendant of dir
             (setq result (string-prefix-p dir file))))
         ;; When exclude, the result needs to be reversed: t->nil,
         ;; nil->t Note `not' is not suitable because nil returns nil.
         (when exclude
           (if result (setq result nil) (setq result t)))
         result)))))

@nobiot this worked amazingly well on my end, I use helm setup.
I bound this function to C-c n F. Pressing TAB invokes CRM-complete conveniently, C-SPC for multiple selections. I think this great piece of code should find its way into org-roam package one day.
The universal argument was a very smart idea to take care of the excluded subdir(s).

So for each session one has to repeat the same keystrokes to enter the excluded/included dirs when searching for the relevant nodes. There is a convenient way to reduce that by pressing M-p/n this will bring up the history forward or backward, emacs shines in these details.

Thanks a lot.

org-roam-2

1 Like

Good to know that it works for you. Thanks for coming back for confirming. Not sure of incorporation into Org-roam’s code, but I’d at least try to update the wiki :slight_smile:

Wiki added: User contributed Tricks · org-roam/org-roam Wiki · GitHub

1 Like

Thank you. Added this as a bullet point in the wiki, too.

I have a feeling that this M-p/n can be also further made easy by passing a history variable to HIST of completing-read-multiple, but I don’t think I will have enough time and energy to test this idea at the moment. If you or anyone is interested, it will be great to see how far this little function can go :slight_smile:

Oh, that would be great to reduce the keystrokes even further. Unfortunately, I am not familiar with how to pass a history to CRM. May be someone would chime in to help.

To the point of filtering in org-roam, I think it needs to be addressed differently depending on the use case and workflow of the user. The function thankfully provided by @nobiot comes in handy for (on-the-fly filtering) that requires alternating node finding scheme in one session without being strongly committed to a specific scheme in mind, the user would like to have all subdirs at disposal and no intention to filter some indefinitely.

But for another use case, where the user wants to stick to a fixed PRESET subset of subdirs for a number of days on domain A without bothering any of B and for another number of days on domain B without bothering any of A , I guess, a different setup would be necessary.

One suggestion as pointed above, passing a subset of subdirs to a variable (scheme), and it has as many subdirs as the user want to process for each domain (domain A: subdirA, subdirB, subdirC; domain B subdirD, subdirE). The variable must be customizable so that user will set the variable to one of the schemes with its associated subdirs once without bothering to input different schemes and their associated subdirs each time each session to find the relative nodes, this will save many keystrokes in my opinion.

On that day when working on domain B is required, the user can again change the value of that variable to set the scheme value to B and its associated subdirs. I think this mechanism is effortless and more convenient to the org-roam-busy user in this case.

Easy to achieve. See the slightly adjusted version of my/org-roam-nde-find-in-selected-dirs and a sample usage with a preset subdirs.

You can also


(defun +org-roam-find-in-only-subset ()
  (interactive)
  (my/org-roam-node-find-in-selected-dirs (list "daily" "projects" "personal")))

(defun +org-roam-find-except-daily ()
  (interactive)
  (my/org-roam-node-find-in-selected-dirs (list "daily") 'EXCLUDE)) 
;; you can pass t instead of 'EXCLUDE

;; Below is an adjusted version of the main command to achieve the preset functions like those above
(defun my/org-roam-node-find-in-selected-dirs (&optional dirs exclude)
  "Show list of `org-roam-node-find' only in selected directories.

You will get a completion in minibuffer to choose one or more
directories from a pre-defined candidate list.

Optionally You can pass EXCLUDE to exclude files in the selected directories
with using `universal-argument' (\\[universal-argument]).

Optionally in Elisp, you can pass DIRS, a list of
directories. In this case, you do not get the completion in
minibuffer.

This command assumes that the subdirectories in the list are
relative to `org-roam-directory`.

Directly change the list in the program."
  (interactive)
  (let* ((exclude (or exclude current-prefix-arg))
         (dirs-selected
          (if dirs dirs
            (completing-read-multiple "Select dir(s): "
                                      (list "daily" "personal" "projects"))))
         (abs-dir-names (mapcar (lambda (dir)
                                  (expand-file-name dir org-roam-directory))
                                dirs-selected)))
    (org-roam-node-find
     nil nil
     (lambda (node)
       (let ((file (org-roam-node-file node))
             (result nil))
         (dolist (dir abs-dir-names result)
           ;; Ensure to do all the dirs in
           ;; abs-dir-names, but once result is t,
           ;; don't override it.
           (unless result
             ;; `file-in-directory-p' seems work to when the file is
             ;; directly under the directory. It returns nil when the
             ;; file is a farther descendant. `string-prefix-p' can be
             ;; used instead if both dir and file are absolute file
             ;; names, and see if file is a farther descendant of dir
             (setq result (string-prefix-p dir file))))
         ;; When exclude, the result needs to be reversed: t->nil,
         ;; nil->t Note `not' is not suitable because nil returns nil.
         (when exclude
           (if result (setq result nil) (setq result t)))
         result)))))

You can easily turn the list of subdirs used in the functions into a (customizing) variable (or variables) that you can tweak more easily than directly changing the function(s). I will leave it to you and the community for now; my lunch time is nearly finished :slight_smile:

Wow! This is it. Sir you have just advanced org-roam single-handedly to a new level of user experience. No more unnecessary keystrrrrrrrrrrrokes. I bound these two functions to two keybindings and that’s it. All went well. If org-roam workflow was like roaming aimlessly before, now with much discernment and as fast as one keystroke. Thanks a million.

1 Like