SOLVED: Exclude properties drawer from Unlinked References rg

I am using org-noter to annotate PDFs, by calling org-noter-insert-precise-note, which generates an org-heading with a property drawer contraining :NOTER_PAGE: which includes the title of the document.

Currently Unlinked References in the org-roam-mode buffer is returning hits from from :NOTER_PAGE: and other portions of the property drawers on each of these notes, which is not particularly helpful and adds quite a bit of clutter (example below).

Is there a was to configure the rg-command defined in org-roam-mode.el to exclude the properties drawers from the returned hits in unlinked references?

It’s hard to see what you really want to remove from the result.
I’d remove the entire “unlinked reference” section (easiest). I’m on Windows so it cannot even be used – I feel nothing is missing this way. You will remove the entire “clutter”.

If you want to keep the section but remove some text from each line that ripgrep returns, I don’t see any customizing options.

Have a look at variable org-roam-unlinked-references-result-re and function org-roam-unlinked-references-section.

You would probably need to do two things:

  1. Define your own regexp that assign groups for the parts you want to remove from the result (e.g. the text string “#+NOTER_PAGE:”)
  2. Override the function and work out a new logic that do not add the groups you want to remove – see how it constructs the displayed line is constructed (I think it’s f, row and col with using match-string and the number that represents the group).

I can’t test my theory as I don’t have the unlinked ref section altogether. But something like this should work.

Thanks @nobiot I’ll look into the variables you suggested and possibly defining my own regexp and update if I find a workable solution.

What I want to do is override the rg-command defined in org-roam-mode.el to exclude all results that contain the string :NOTER_PAGE: or :id:

I am getting two types of hits from the unlinked references rg-command:

  1. Hits from the actual content of my notes in org headings, which is useful and what I want to see in the Unlinked References section. There is no unwanted information from the properties drawers displayed in these hits.

  2. Hits from :NOTER_PAGE: and :id: in the properties drawers themselves, for which the relevant org heading does not contain any germane reference. This is because the title of the pdf :NOTER_PAGE: it is pointing to contains the string of the new node I am creating. What I end up with is 10’s to 100’s of hits showing me the same directory path from :NOTER_PAGE:

I see…
It could be as easy as adding another condition in this part in org-roam-unlinked-references-section:

(when (and match
           (not (f-equal-p (org-roam-node-file node) f))
           (member (downcase match) (mapcar #'downcase titles)))

Like:

(when (and match
           (not (f-equal-p (org-roam-node-file node) f))
           (member (downcase match) (mapcar #'downcase titles))
           (not (s-contains? ":NOTER_PAGE:" line t))
           (not (s-contains? ":ID:" line t))

I am assuming line contains the entire line of string returned by ripgrep.

EDIT: Sorry, you may not have package called s for s-contanis? function. But I think you get the idea I’m trying to convey.

I have also run into this issue - my org-noter headings have an :id: and so are classified as org-nodes. I have attempted your fix but it doesn’t work for me. Alas I am a complete noob who has little understanding of emacs or lisp, so I don’t know if I have done the right thing. What I did:

Verified that I have package s installed.
Go to my spacemacs packages and edit elpa/27.2/develop/org-roam-20210930.1108/org-roam-mode.el as in your post.
Restart emacs and attempt to use org-roam-insert.

It has been a long time since my post; could you please remind me what the problem is and what “fix” have you tried?

One thing I don’t understand is, why do you have the ID property for Org-noter notes. Do you need IDs? Perhaps Spacemacs automatically adds another package that does it (org-notes-pdf, or something I forgot the exact name). If you don’t need it, could you turn it off or remove it?

My understanding of the problem is this.

  1. org-roam. org-roam uses nodes that are defined as a heading (or file) with “:PROPERTIES:” and “:id:”.
  2. org-noter-pdftools. When annotating a pdf, this tool will add headings with “:PROPERTIES:” data that includes “:id:”. It looks like this:
* My annotated text   
   :PROPERTIES:
   :NOTER_PAGE: [[pdf:PATH_TO_FILE.pdf::2++0.00;;annot-2-36]]
   :ID:       PATH_TO_FILE.pdf-annot-2-36
   :END:

Consequently when searching for org-roam nodes, all of my pdf annotations appear in the search.

As I understood your suggested fix, it was to get org-roam to disregard nodes with “:NOTER_PAGE:”.

@Muff2n
Thank you. I suggest you try one of the options described in this other thread – I think disabling the ID for org-noter-pdftools is the easiest and hence my recommendation.

This thread is about unlinked references, which might look similar but a different issue from what you describe.

Understood, thank you.

sorry I posted earlier but was confusing two different issues

Unfortunately I am still too green with emacs to know how to implement the solution you proposed and I had to abandon using unlinked references altogether. If you could provide some further guidance I would be very greateful!

I don’t have s installed and I’m not sure how to install it on doom/macos (is this an emacs package or a command line tool?) I am also not very well versed in lisp and am unsure how I would go about incorporating your snippet into my config file - would I just place something like this in my config?

I don’t know if I can be of any help; I don’t use Doom and unlinked references.

s is an Emacs package. I suggest you refer to Doom documentation for how you are supposed to install one. I believe you do something in your package.el (some Straight syntax) and then doom-sync to effect the change.

For the linked function, I can’t tell much without context, but perhaps you can try? It is to override the function org-roam-unlinked-references-section, so I think you’d need to evaluate it after you have loaded org-roam. I’d surround the defun with with-eval-after-load 'org-roam like this below. I don’t know if this is the proper way. I think you can put it in config.el (? I guess for Doom??)

(with-eval-after-load 'org-roam
    (defun ...))

Thanks @nobiot, really appreciate your help.

It turns out s.el is pre-installed with doom (my bad). I was able to add your snippet to the org-roam-unlinked-references-section function using with-eval-after-load in ~/.doom.d/config.el, and confirmed that it was changed with helpful-function.

However it seems that the snippet does not change the behaviour of the unlinked-references section in the org-roam-buffer.

FWIW I took a look at the regexp in the varaible org-roam-unlinked-references-result-re but to be totally honest I’m not quite sure what I’m doing.

Sorry, not sure what “snippet” you refer to and what exactly you did?

I added the following to org-roam-unlinked-references function, and wrapped it in with-eval-after-load:

(when (and match
           (not (f-equal-p (org-roam-node-file node) f))
           (member (downcase match) (mapcar #'downcase titles))
           (not (s-contains? ":NOTER_PAGE:" line t))
           (not (s-contains? ":NOTER_DOCUMENT:" line t))

I put the new function in my config.el (full function here). I used helpful-function to confirm that the s-contains? conditions were added to the function. However the org-roam-buffer still shows lines containing :NOTER_PAGE: and :NOTER_DOCUMENT:

I see. Then I’d suggest the best way forward is to use edebug to go through the runtime line by line as Emacs evaluates the function. It’s easy; move your cursor somehwhere in the function and set the break point by M-x edebug-defun. Then try to run Org-roam as normal. If Emacs hits the break point, a new window pops up (it’s an Emacs window within the current frame).

You may need to extract the defun from with-eval-after-load and evaluate the function so that the line that contains defun has no indentation — the rest of the function can have indentation for readability (I think there should be a technical explanation for this but I can’t tell—edebug might just work without doing this).

The result of each function and variable gets displayed in the echo/minibuffer part as you move each line with SPC key.

Thanks again - I was able to use edebug-defun to step through the function as it populated the unlinked references section of the org-roam buffer. When I got to the break points marked by * below…

(when (and match
           (not (f-equal-p (org-roam-node-file node) f))
           (member (downcase match) (mapcar #'downcase titles))
           (not (s-contains? ":NOTER_PAGE:" line t))*
           (not (s-contains? ":NOTER_DOCUMENT:" line t))*
)*

… the output in the minibuffer was Result: t even when the output of the whole function and the unlinked references section contained :NOTER_PAGE: or NOTER_DOCUMENT:.

I’m not sure if I’m interpreting this correctly, but this appears to be an issue with the conditions defined above not correctly identifying the strings :NOTER_PAGE: and :NOTER_DOCUMENT:?

This is weird. I can’t really test it on my end – I can’t use unlinked reference on my machines.
The intention of this part is that line contains each line that rg returns, and only when all the conditions are met and only then does Org-roam adds the line for the unlinked references section in the org-roam buffer.

If I evaluate the code below, I get nil – so in theory, it should work.
When you edebug, it should also display the value of line – can you make sure the property in the code is correctly spelt – e.g. is the _ underscore correct, not a - hyphen, for example?

(not (s-contains? ":NOTER_PAGE:" "this contains :NOTER_PAGE:" t))

I really cannot do much more on this as I can’t really test it it on my end.

I think I figured out the issue, but am unsure how to fix it…

(not (s-contains? ":NOTER_PAGE:" line* t))

The value of line at the * above is:

Result: "/Users/ryantaylor/Documents/GranularityNet/slobodi..."

This is the file path of the node that the org-roam-buffer is getting the results from. However, I was expecting the value of line to be from the contents of the node, not the file path. If I change the function to contain a string from the file path, for example:

(not (s-contains? "slobodi" line t))

I get nil for results with “slobodi” in the file path when I step through with edebug-defun.

It appears to me that s-contains? is acting on the file path of the node, rather than it’s contents, and this is why it’s not detecting :NOTER_DOCUMENT:.

I tried changing line to contents but I get an error that contents is a void variable.

It looks to me that line loops for each line of results, so it’s possible that one line may be the filename (including the path). You will need to check all the lines

When edebug-defun steps through the org-roam-unlinked-references-section it loops for each of the results in the org-roam-buffer's unlinked-references-section.

I looped the function through all results in the org-roam-buffer. For all unlinked references that populate the unlinked-references-section the result of line in (not (s-contains? "EXAMPLE" line t)) is a file path.

Just to be clear: the string I am trying to exclude from unlinked references does contain a file path, but it is a different file path than the one shown in the results of line. The file path I am trying to exclude is the PDF linked by in the :NOTER_PAGE: and :NOTER_DOCUMENT: properties. The file path in the line result is to the .org document that the unlinked references are in.

I may be misunderstanding the issue but it looks to me like s-contains? is only looking at the org document title/file path and not the contents.