A way to cache the contents of the roam buffer?

How To Requests
1

I have some big files containing many nodes that have many, many backlinks (main ideas in my PhD research that I link from source quotations and such). Navigating this file is a pain – every time I move from one heading to another emacs hangs while the org-roam buffer regenerates. It’s particularly bad as I use evil-mode and like to navigate with the keyboard.

Does org-roam have a facility to cache the generated contents of the roam-buffer so it doesn’t have to regenerate all the time?

Alternatively, if I could set a delay before updating the buffer, say not until point is at a given node for, say, three seconds? Other suggestions could work too, I just want to keep the roam buffer open and following me as I’m working without all the lag.

2

How about overriding the function org-roam-buffer--redisplay-h like this?
This way, Emacs will wait for you to be idle for 3 seconds and then attempt to re-display org-roam-buffer (please test – it seems to work on my end).

   (and (get-buffer-window org-roam-buffer)
-       (org-roam-buffer-persistent-redisplay)))
+       (run-with-idle-timer 3 nil
+                            #'org-roam-buffer-persistent-redisplay)))
3

Well, I can try writing a cache, but I want to ask some basic preliminary questions before venturing on it. What is causing the holdup? Please supply empirical data regarding function calls, knowing this would allow us to target the appropriate function call for giving cache coverage.

Use the profiler.el library to do tests, the relevant functions are

(profiler-start)
[do task 3-4 times]
(profiler-report)
(profiler-stop)

Regarding delaying refresh

(defvar org-roam-buffer-redisplay-timer nil
  "Timer object for delayed redisplay of `org-roam-buffer'.")

(defvar org-roam-buffer-last-node nil
  "Stores the last visited Org-roam node for delayed redisplay.")

(defun org-roam-buffer--scheduled-redisplay-h ()
  "Delay redisplay of `org-roam-buffer' by 5 seconds, only if the node changes.
Runs as a post-command-hook."
  (when-let ((node (org-roam-node-at-point)))
    (unless org-roam-buffer-last-node
      (setq org-roam-buffer-last-node node))
    (when org-roam-buffer-redisplay-timer
      (cancel-timer org-roam-buffer-redisplay-timer))
    (and (get-buffer-window org-roam-buffer)
	 (setq org-roam-buffer-redisplay-timer
	       (run-with-idle-timer 5 nil #'(lambda (node)
					      (unless (eq node org-roam-buffer-last-node)
						(setq org-roam-buffer-last-node node)
						(org-roam-buffer-persistent-redisplay)))
				    node)))))

(advice-add 'org-roam-buffer--redisplay-h :override #'org-roam-buffer--scheduled-redisplay-h)

Cannot test extensively because of my personal configurations, tested lightly. Will refresh only when idle for 5 seconds

4

Thanks for the suggestions!
@nobiot thanks! I tried @akashp’s timer code instead of modifying source files. :slight_smile:

@akashp here is the profiler report from moving into and out of a heading with ~30 backlinks. I’m not sure how to track down the anonymous lambda.

                                                                333,186,153  95% - org-roam-buffer--redisplay-h
                                                                333,186,153  95%  - org-roam-buffer-persistent-redisplay
                                                                332,873,908  95%   - org-roam-buffer-render-contents
                                                                332,482,810  94%    - org-roam-backlinks-section
                                                                326,649,381  93%     - org-roam-node-insert-section
                                                                288,534,976  82%      - org-roam-preview-get-contents
                                                                275,521,735  78%       - org-indent-refresh-maybe
                                                                275,263,383  78%        - org-indent-add-properties
                                                                251,362,967  71%         - org-at-item-p
                                                                245,778,327  70%          - org-element-at-point
                                                                231,278,663  66%           - org-element--parse-to
                                                                146,862,651  41%            + #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_38>
                                                                  3,933,488   1%            + org-element--cache-put
                                                                    123,264   0%            + org-element-org-data-parser
                                                                    145,728   0%             derived-mode-p
                                                                    475,496   0%           org-indent-set-line-properties
                                                                      1,056   0%         + org-current-level
                                                                      1,056   0%        + org-get-limited-outline-regexp
                                                                  9,846,730   2%       + org-mode
                                                                  1,361,384   0%       + set-auto-coding
                                                                    363,288   0%       + custom/org-roam-preview-default-function
                                                                    272,984   0%       + org-element--cache-after-change
                                                                    205,737   0%       + org-roam-strip-comments
                                                                     97,336   0%       + org-indent-notify-modified-headline
                                                                     34,288   0%       + org-fold-core--fix-folded-region
                                                                     13,648   0%       + #<compiled -0x1c692097ad1ad977>
                                                                 22,766,044   6%      - org-roam-fontify-like-in-org-mode
                                                                 13,548,469   3%       + org-mode
                                                                  8,940,166   2%       + font-lock-ensure
                                                                    226,624   0%       + #<compiled -0x1c692045bbe6dd77>
                                                                    155,160   0%      + mapconcat
                                                                     23,888   0%      + magit-insert-section--create
                                                                     16,656   0%      + magit-insert-section--finish
                                                                     16,576   0%        jit-lock-after-change
                                                                      9,344   0%      + magit-insert-heading
                                                                  5,666,428   1%     + org-roam-backlinks-get
                                                                     10,169   0%     + magit-insert-section--finish
                                                                      4,144   0%       jit-lock-after-change
                                                                    353,242   0%    + org-roam-mode
                                                                     28,912   0%    + magit-insert-section--finish
                                                                      4,144   0%      erase-buffer
                                                                    312,245   0%   + org-roam-node-at-point
                                                                 14,612,032   4% + command-execute
                                                                  1,599,015   0% + timer-event-handler
                                                                    874,638   0% + redisplay_internal (C function)
                                                                      8,288   0%   winner-save-old-configurations
                                                                      8,192   0% + ...
                                                                      6,144   0% + evil-escape-pre-command-hook
                                                                      1,008   0% + gcmh-register-idle-gc
                                                                        688   0% + flycheck-display-error-at-point-soon
                                                                        104   0% + eldoc-schedule-timer
                                                                         84   0% + eldoc-pre-command-refresh-echo-area

Your delay timer override makes a big difference. I set the idle timer to 1 second (5 was quite long) and I can now navigate around with evil keys as I would expect. I’ll keep it like that for a while to test a bit more rigorously, but this definitely seems like a satisfactory solution most of the time. I have a few nodes with >100 backlinks (which is probably mainly sloppiness on my part), and those take quite a long time, 20+ seconds, to render the backlinks. A cache would be useful in these cases, but I’m on the occasions I need to actually work with those nodes I can probably just open a dedicated roam buffer to keep it from getting regenerated all the time.

1 Like
5

I know what is causing the holdup, you do not need to do anything. No cache is required. I will post a more detailed answer in a while.

6 
(defun patch/org-roam-buffer-render-contents (fn &rest args)
  (let ((org-inhibit-startup t))
    (apply fn args)))

(advice-add 'org-roam-buffer-render-contents :around #'patch/org-roam-buffer-render-contents)

Remove the delay function I gave earlier and do this. It will clear the holdup. The problem is you have set org-indent-mode to run in all org-buffers, not using a hook, but by using the option (setq org-startup-indented t) this makes it run in temporary buffers too.

This was a talked about issue Org-roam buffer rendering is slow when large files are used and Org mode indentation is enabled · Issue #2399 · org-roam/org-roam · GitHub

7

Oh, wow, that makes a huge difference, thanks! It’s still a bit laggy when entering nodes with many backlinks (a few seconds at ~150 links). Is there any downside to keeping the idle timer? Even setting it down to .25 seconds, it still makes navigating a bit more responsive especially when I move around files with many frequently linked nodes.

8

@bradmont

Not, really but update the code then, I am giving you a more streamlined code block, replace the previous one with this

(defvar org-roam-buffer-last-node nil
  "Stores the last visited Org-roam node for delayed redisplay.")

(defun org-roam-buffer--scheduled-redisplay-h ()
  (and (get-buffer-window org-roam-buffer)
       (org-roam-buffer--scheduled-fn)))

(defun org-roam-buffer--scheduled-fn ()
  (when-let ((node (org-roam-node-at-point)))
    (unless (equal node org-roam-buffer-last-node)
      (setq org-roam-buffer-last-node node)
      (run-with-idle-timer 1 nil #'org-roam-buffer-persistent-redisplay))))

(defun org-roam-buffer--scheduled-persistent-cleanup-h ()
  (setq org-roam-buffer-last-node nil))

(advice-add 'org-roam-buffer--redisplay-h :override #'org-roam-buffer--scheduled-redisplay-h)
(advice-add 'org-roam-buffer--persistent-cleanup-h :after #'org-roam-buffer--scheduled-persistent-cleanup-h)
9

This happens because I have blocked refresh if you do not change the current node. Otherwise this tries to keep refreshing everytime you do ANYTHING even moving the point across will cause the org-roam-buffer to refresh :man_mage:

Strange… it shouldn’t be so eager to run. It is a design flaw in my opinion. Anyways. Your problem is fixed I suppose. No need to have a cache.

Best.

10

Oops, apologies, I forgot to write back. Yes, this absolutely fixes my problem, no need at all for a cache. Thank you!

1 Like