Is it possible to include all files in the Org-roam graph, not just Org files? It would be awesome to see them in org-roam-ui too. I have read some stuff about trying to make Org-roam a filesystem, but not anything in the complete same way I am imagining. Because all these attempts I see treat Org files as somehow special class files over all the other files, but I would like to link any file to any other file.
Of course with non-Org formats you cannot have links in inline text, etc (well, unless the format has those functionalites like Markdown). But you could still get the main benefits of linking, tagging, and titling. I am thinking that one simple way to do it is by having an associated Org file with each non-Org file, that stores its properties and title. So the Org file would be a separate metadata file, linked by ID to the actual file itself.
I doubt this has been done already, and I imagine there would be a lot of small things to get right, but I can’t imagine it would be too hard because it is building on existing functionality not anything new. I’d like to do it myself, although I have not much Emacs Lisp experience. Does anyone have any pointers in the right direction with this idea?
I am not sure if Development is the right category for this or not, but it is my best guess.
Use org-roam-db.el as a reference only. Do not use any of the functions directly, because the underlying org-element will output warning if it is used in non-Org buffers.
This would mean you’d need to create your custom functions to update database (the same database that you specify org-roam-db-location. This then should let org-roam-ui to take the data entries as they are with no changes
See variable org-roam-db--table-schemata for the tables and their schema.
You’d likely need to update 3 tables files, nodes, and links.
For non-Org files, you will need to decide what constitute an ID for them. Consider file name prefix, for instance (like Denote does), for binary flles, including PDF documents, images, etc. For plain text files, such as Markdown, etc. you could read the file content, but it may be easier to have a consistent way.
Thank you a lot for taking the time to answer all that, those are helpful pointers.
When you said you have done it already, do you mean just for Markdown files or also other file types? From looking at md-roam it seems just Markdown, but that still creates a principle to do it for other file types so I’ll look at how you’ve done it.
For ID, I was thinking of using UUID as file name for both Org and non-Org files.
Md-roam is clearly designed with markdown in mind but should work with any plain-text extensions as long as:
the file has the YAML meta data like in the README
the file start with --- (three hyphens) at the top line
the YAML fense (block?) ends with --- (or change the variable to another common convention — I forgot if it was “…” or something else
This is because md-roam does not rely on markdown-mode and use regexp to find ID within the YAML meta data block. It does not matter if the ID is UUID, date, or the names of your family members. That is the same for Org-roam and Org-ID — I am writing a guide about it including the capture…
You can set md-roam-extension to “txt” for instance. I think it is now a list of extentions to support those other than “md” (the name has stayed singular “extension” because that’s how I started out with). I am out so can’t verify my statement at the minute.
Md-roam does not work for binary files because it needs to read the text content to find their ID. It is still a hack because it sometimes still causes org-element to complain that the buffer isn’t org… but it has worked reliably for me for years. I have started to think that it may have been better if I wrote my own functions instead of advising org-roam-db functions…
This should be easy to do if you use org-roam-db as a reference. You’ll notice that it makes the update operation easy by deleting and creating the same record by the ID as the primary key.
A challenge will likely be how to automatically trigger the update function for your “non-text/proprietary/etc” files. As you won’t be visiting them within Emacs, you cannot use the find-file-hook and before-save-hook, which org-roam-db-autosync-mode uses.
One easy way is not to automate this, and manually trigger the update operation every time you add a new binary file to org-roam-directory (I’d imagine you just use the OS’s file manager for this).
Alternatively, you can look to use the built-in filenotify library to “watch files for change on disk” (C-h P, describe-package) – I do this for my own purpose, and it seems to work OK-ish so far (on Linux and Windows).
