Subdirectories in .gitignore are not picked up by roam-db-sync

Troubleshooting
1

Hello!
I’ve experienced and somewhat resolved a problem with my workflow from “fd” backed implementation of “'org-roam–list-files-fd”

Update:
this workaround doesn’t seem to work too well - the org files from “Work” directory are not returned by “org-roam-node-find”
Not sure when they are filtered out, I guess simplest workaround is to use “find”

I want to leave notes on possible workaround for those who would encounter it in the future, and also ask for advice on how to handle that better:

My setup is:

  1. all notes and journals in ~/org
  2. work notes are in ~/org/Work
  3. I have .gitignore and syncthing ignore for “Work”
    so I can sync my personal notes between work and personal notebook, and keep work notes separately
  4. Use templates for notes to create notes in apropriate subdirectories and add tags

Using Doom emacs for the longest time all worked well, and after some upgrade I found out that work notes are not in the db, and “C-u org-roam-db-sync” doesn’t pick them up.

I found out that Doom emacs reconfigures “org-roam-list-files-commands” variable to use “fd” and “fdfind” as priority

And “fd” (or rg too I think) by default exclude files in .gitignore from results!
My workaround is to advice “org-roam–list-files-fd” to modify string to executable with key “-I” which makes “fd” to include all files

(defun my/add-fd-ignored-files-flag (&rest rest)
  (pcase (cl-first rest)
    (`(,exec ,path)
   (list (string-join (list exec "-I") " ") path)))
  )

(advice-add 'org-roam--list-files-fd :filter-args #'my/add-fd-ignored-files-flag)

And now “C-u org-roam-db-sync” picks them all up.

Is there a better way?
Maybe “org-roam–list-files-fd” and “org-roam–list-files-rg” to be somehow configurable?

I guess by default “consult-ripgrep” just doesn’t include ignored files and to include them you pass in explicit argument for “rg” command.
With that Doom implementation of “+vertico/project-search” passes “include .gitignore files” attribute on invocation with universal argument.

2

Why not stick to find?
See org-roam-list-files-commands – it’s a customizing variable to control which “search” executable to use. I don’t know how much of this standard way is applicable to Doom.

I think consult-ripgrep is easy to configure. Refer to this discussion.

3

Update 2:
For some reason switching to “find” also works only until emacs restart
After restart “org-roam-node-find” and “org-roam-node-insert” don’t offer notes from “Work” directory for selection

Will need to dig deeper or give up keeping notes in a repo with .gitignore for some subdirs

Maybe reason is in the way completion candidates are gathered, or maybe notes are lost from the db

4

The “org-roam-node-read–completions” functions calls “org-roam-node-list”
which does SQL query

I’ve manually called “org-roam-db-query” with the “SELECT …” statement, and it seems that after force reinit, I do have “Work” notes in the db, but after emacs restart they are not returned by the query

That is a strange thing, I don’t expect to get to the bottom of this easily, would probably just remove Work from gitignore
and unless anyone offers any pointers on where too look, would leave things at that

5

My only guess is that you use customize, which Doom throws away. If this is the case, I think you would be fine if you used the :custom (or :config) section of use-package! macro – these are Doom specific.