Few weeks ago I started to build an interactive graph visualizer and integrated note browser (see a demo here). At first it was just a personal project and curiosity. However, the community showed appreciation and interest in the tool, and graphing capabilities of org-roam became a central discussion topic with many ingenious proposals. Right now I think it is a good idea to consider building it at full scale, and that’s why I need your comments on a few things.
There are a few possible ways to organize the codebase.
At its core, what I am doing is just taking a JSON file (relationships between nodes and edges) and some HTML files (org-publish of the entire org-roam directory) and displaying them in a webpage using a http-server.
Now, I think this much should be a package separated from org-roam (primarily because this is not written in Elisp).
However, I am not sure what to do about the conversion of the SQL database to a JSON file part. This can be done in any language (and to do it incrementally, I think we should do it from Emacs, but I am not sure about scalability and performance). If we are doing it in Emacs using Elisp, should this be part of
org-roam codebase? If not using Elisp, what other option do we have?
Also, as I am displaying backlinks below each file in the note browser,
org-publish needs to be force called on file X each time I reference file X in any other file. I am not sure if this will cause performance issues since AFAIK
org-publish is synchronous in nature. Would love to know if someone has a better idea.
Modularity and Other Graphing Features
Clustering and centrality measures seem to be the most sought after features. I think it is possible to achieve these using something like igraph where we add extra information in the JSON file. However, I am not sure if this can be done incrementally, as it would most possibly require computation using the entire database (which is compute-expensive). What are our options then?
Right now, I am using a shell script that generates the JSON database and starts an http-server. This is complicated to setup cause you have to manually modify some parts of your Emacs dotfile as well. What would be the most user-friendly option for packaging this entire thing?
I welcome all of your comments, questions, and suggestions.