[TIPS] dynamically add Org-roam files to your agenda file

cool_ran · January 1, 2021, 12:37am

Dear folks:

An org-roam user (lld2001) post a very useful function to dynamically
add Org-roam files that contain TODO to your agenda files.

(defvar dynamic-agenda-files nil
  "dynamic generate agenda files list when changing org state")

(defun update-dynamic-agenda-hook ()
  (let ((done (or (not org-state) ;; nil when no TODO list
                  (member org-state org-done-keywords)))
        (file (buffer-file-name))
        (agenda (funcall (ad-get-orig-definition 'org-agenda-files)) ))
    (unless (member file agenda)
      (if done
          (save-excursion
            (goto-char (point-min))
            ;; Delete file from dynamic files when all TODO entry changed to DONE
            (unless (search-forward-regexp org-not-done-heading-regexp nil t)
              (customize-save-variable
               'dynamic-agenda-files
               (cl-delete-if (lambda (k) (string= k file))
                             dynamic-agenda-files))))
        ;; Add this file to dynamic agenda files
        (unless (member file dynamic-agenda-files)
          (customize-save-variable 'dynamic-agenda-files
                                   (add-to-list 'dynamic-agenda-files file)))))))

(defun dynamic-agenda-files-advice (orig-val)
  (union orig-val dynamic-agenda-files :test #'equal))

(advice-add 'org-agenda-files :filter-return #'dynamic-agenda-files-advice)
(add-to-list 'org-after-todo-state-change-hook 'update-dynamic-agenda-hook t)

I find it is useful and hope it can be useful to you as well.

d12frosted · January 16, 2021, 10:25am

Thank you for sharing it. The problem with this approach is that (a) you can’t plug it into existing pile of notes and (b) messing up with your customize file leads to broken agenda.

This post motivated me to share how I approach this problem that I am using for a month already and super happy about it’s performance. You can read detailed description on my blog. The code is available as a GitHub Gist.

TL;DR - in the before-save-hook current buffer is traversed for existence of any TODO entires and Project tag is either added or removed from #+ROAM_TAGS property. Now, org-agenda is adviced before to set the value of org-agenda-files to the list of files with Project tag (quieried from db, which is enourmously fast).

There is also a snipped to use for migration. This way all required information is always there, in your files and can be easily reconstructed. That being said, proposed solution can be easily adapted to dynamic-agenda-files approach.

Hope someone finds it useful

S_Fabris · April 15, 2021, 10:43am

This code snippet for collecting files with todos has greatly impacted my workflow! Thanks for sharing.

I am just trying to figure out how this would work with org-roam V2. ROAM_TAGS have been dropped and from the code it seems that FILETAGS will only get stored in the database if there are headline nodes with ID that inherit the tag. And even if this would work, every todo in the agenda woud be tagged with “Project”, adding visual clutter.

Have you adapted this for the new version already? Would it make sense to use a note that is being linked to by every file that contains todos and then query the database for backlinks?

nobiot · April 15, 2021, 4:00pm

Hi @S_Fabris

I cannot contribute to @cool_ran’s script, but wanted to respond to two points you commented about V2.

I have been using V2 every day for work for a couple of weeks now; FILETAG just works without headline nodes.
Most of my org files do not have headlines. I use FILETAG as a category, like Meeting, Reference, etc. No issue.

I think this is largely related to how you use/customise agenda (I do not show any tags in my agenda view; I just use them as search keys)… You could also look to control how file-level tags get inherited by the looks of this Org documentation.

To limit tag inheritance to specific tags, or to turn it off entirely, use the variables org-use-tag-inheritance and org-tags-exclude-from-inheritance .

I don’t use these variables so I cannot be 100% certain, but it’s probably worth looking into.

S_Fabris · April 19, 2021, 12:58pm

Thanks for the hint to org-use-tag-inheritance, this keeps the agenda view clean as I do not use tags for other purposes than marking files with todos inside.

S_Fabris · April 23, 2021, 8:14am

After some time testing, this does not work for me. Filetags are somehow not consistently added to the database. It worked for the some daily files with tasks and for some the caching function does not add the filetags to the db entry.

However, the modified code from @d12frosted works flawlessly in adding the tags and retrieving them from the db.

Has anyone else used filetags successfully with V2?

nobiot · April 23, 2021, 6:06pm

This issue you experienced might be caused by this technical issue, which has been resolved.

S_Fabris · April 28, 2021, 11:26am

That’s it! Everything working now!

KSS · May 8, 2021, 9:58pm

This is a very interesting idea for workflow that might work for me well.
I’m currently trying to make the code work at least partially with org-roam v2.

What I want is to add to org-agenda-files files with “Project” tag. This is implemented by those two functions:

(defun vulpea-project-files ()
  "Return a list of note files containing Project tag."
  (seq-map
   #'car
   (org-roam-db-query
    [:select file
     :from tags
     :where (like tags (quote "%\"Project\"%"))])))

(defun vulpea-agenda-files-update (&rest _)
  "Update the value of `org-agenda-files'."
  (setq org-agenda-files (vulpea-project-files)))

When I run vulpea-project-files function, I receive the following error:
EmacSQL had an unhandled condition: "no such column: file"

I tried to investigate the structure of org-roam.db with .schema command to understand which column name should I use. Here is what I got:

Given this structure, shouldn’t it be working as expected? (I haven’t used sql in my life, but it looks that there is a column with name file).

Thank you