Exports and Org-Roam files as a Org-Roam beginner

[laza]

Hi! Coming from plain org-mode I’ve started using org-roam for some while now. Information that I gather comes 99% from webpages. So I use org-protocol to capture information from websites in org-roam files.
These files all have a ROAM_REFS header with the URL to the websites.

Now when I do an export (pdf or ox-hugo), the links all contain the id of the files. What I would like to see is that this would be the URL of the websites, if a ROAM_REFS header is present. Is this a common request with some common solution (sorry, I couldn’t find one)? Or how would I go about it?

[akashp]

Tested lightly on my end - let me know how it goes

(defun org-export-resolve-roam-refs--external (id)
  "Retrieve ROAM_REFS for the given ID."
  (with-temp-buffer
    (org-id-open id)
    (goto-char (point-min))
    (org-entry-get nil "ROAM_REFS")))

(defun custom/org-export-resolve-id-link (link info)
  (let ((id (org-element-property :path link)))
    ;; First check if id is within the current parse tree.
    (or (let ((local-ids (or (plist-get info :id-local-cache)
                             (let ((table (make-hash-table :test #'equal)))
                               (org-element-map
                                   (plist-get info :parse-tree)
                                   'headline
                                 (lambda (headline)
                                   (let ((id (org-element-property :ID headline))
                                         (custom-id (org-element-property :CUSTOM_ID headline)))
                                     (when id
                                       (unless (gethash id table)
                                         (puthash id headline table)))
                                     (when custom-id
                                       (unless (gethash custom-id table)
                                         (puthash custom-id headline table)))))
                                 info)
                               (plist-put info :id-local-cache table)
                               table))))
          (gethash id local-ids))
        ;; Otherwise, look for external files.
        (let ((roam_refs (org-export-resolve-roam-refs--external id)))
	  (if roam_refs
	      (progn
		(message "roam_refs for id %s: %s" id roam_refs)
		roam_refs)
	    (cdr (assoc id (plist-get info :id-alist)))))
        (signal 'org-link-broken (list id)))))


(advice-add 'org-export-resolve-id-link :override #'custom/org-export-resolve-id-link)

To Directly get the ref by querying org-roam-db

Use this helper function instead of the above – might be slightly faster; but dependent on org-roam; the above is database agnostic however.

(defun org-export-resolve-roam-refs--external (id)
  "Retrieve ROAM_REFS for the given ID from the Org-roam database."
  (let ((link (mapconcat #'identity
			 (car (org-roam-db-query [:select [ref]
							  :from refs
							  :where (= node-id $s1)]

						 id)))))
    (if (string= link "")
	nil
      (concat "https:" link))))

[laza]

Hi Akash!

Thanks a lot for looking into this. I just briefly applied your code and it seems that I’m always getting a “progn: Wrong number of arguments: (2 . 2), 1” error. I have to admit, that I’m that kind of emacs user that doesn’t speak elisp too much. So I don’t know what could be the problem here. I use org-roam 2.2.2, org-mode 9.6.15 and emacs 29.3, if that helps.

Maybe to state what I’m trying to achieve again: Let’s say I have two org-roam files like this:

:PROPERTIES:
:ID:       f55b3540-9647-4b90-9338-6e141a961ccb
:ROAM_REFS: https://org-roam.discourse.group/
:END:
#+title: Org-roam - Org-roam is a Roam replica built on top of the all-powerful Org-mode.
 
Just some content

and another one referencing the above:

:PROPERTIES:
:ID:       c4cb1a78-67f7-44b8-aa9d-44f21e452130
:END:
#+title: org-roam example

This should be a link to the [[id:f55b3540-9647-4b90-9338-6e141a961ccb][org-roam website]]

Now when I export the second one in HTML Format I get:

...
This should be a link to the <a href="org_roam_org_roam_is_a_roam_replica_built_on_top_of_the_all_powerful_org_mode.html#ID-f55b3540-9647-4b90-9338-6e141a961ccb">org-roam website</a>
...

What I would like to see there would be something like:

...
  This should be a link to the <a href="https://org-roam.discourse.group">org-roam website</a>
...

Because this is what ROAM_REFS is pointing to and the former does not make any sense in the exported HTML.

I actually want to use this with the ox-hugo exporter, which producing some reference in the MD file, like that:

...
[Org-roam - Org-roam is a Roam replica built on top of the all-powerful Org-mode.]({{< relref "#d41d8c" >}})

##  {#d41d8c}
...

Here I would like to see the following instead:

...
[Org-roam - Org-roam is a Roam replica built on top of the all-powerful Org-mode.](https://org-roam.discourse.group)
...

I hope this makes a bit more clearer what I’m trying to achieve. But maybe this is not the right approach at all for this.

[akashp]

While converting to html I get the problem that the ID is concated with the website. The code I provided is running in my computer it is supposed to step in during the export process and resolve the ID to something - in default the ID is resolved to the relative file path
(cdr (assoc id (plist-get info :id-alist))
This line is responsible for it.

Strange you are receiving that error -

Did you use my latest code? I updated it. We can work on refining the output in html but unless we figure out what is going wrong it is going to be difficult to proceed from here.

Can we work on diagnosing where the code is failing in your setup?

[akashp]

For example if the document is as such

When I export to pdf I get the following

When I export to MD using ox-hugo

When I export to HTML

[akashp]

I can also get the correct MD export using ox-hugo

We need to adjust format here

org-hugo-link () – “Convert LINK to Markdown format.”

With correct html export

adjust : org-html-link () – “Transcode a LINK object from Org to HTML.”

To

 (let ((fragment (concat "ID-" path))
		 ;; Treat links to ".org" files as ".html", if needed.
		 (path (funcall link-org-files-as-html-maybe
				destination info)))
	     (format "<a href=\"%s\"%s>%s</a>"
		     path attributes (or desc destination))))

---

@laza For your convenience I have packaged the changes I made in a seperate “.el” file. The functions
org-hugo-link and org-html-link are quite long – consider byte-compiling this file. You can load the file as you do for any external “.el” file

(load-file "/path/to/org-export-modifications.el")

Code

org-export-modifications.el

These are all the modifications I require in my set up to do what you are desiring to do.

For Debug

(org-export-resolve-roam-refs--external "f55b3540-9647-4b90-9338-6e141a961ccb")

Tell me the output of the above evaluation after you load the file – the ID should belong to a file which contains ROAM_REFS.

If you need anymore help - or still struggling let me know, we can figure a way out.

[laza]

Akash!

What can I say? This is awesome. Thanks a lot!

(org-export-resolve-roam-refs--external "f55b3540-9647-4b90-9338-6e141a961ccb")

This indeed returns the URL I was looking for.

The HTML also now works as expected. The only thing I still have trouble with is the hugo export. But this might be due to my setup. Let me check and I’ll let you know!

Again thanks so much for your help!

[akashp]

Unfortunately - I noticed there is a problem in the current implementation - in that you always have to use https: links in the roam-ref : that property is also used to keep bibliography notes, if in the future you use a org-roam-bibtex you’d fall into a problem - it indiscriminately attaches “https:” to all type of roam-refs

Instead of the current way - we can do better - why not enable emacs to access org-links from outside?
If we have org-protocol enabled - we may convert selectively http/https links to weblinks and convert other ids to a format understandable by org-protocol.

(defun org-export-resolve-id--external (id)
  "Retrieve a suitable link for the given ID."
  (let ((weblink (funcall #'(lambda (id)
				      (let* ((sql `[:select [type ref]
							    :from refs
							    :where (and (or (= type "https") (= type "http")) (= node-id $s1))])
					     (query-result (string-join (car (org-roam-db-query sql id)) ":")))
					(unless (eq query-result "") query-result))) id)))
    (if weblink weblink
      (format "org-protocol://open?link=[[id:%s]]" id))))

;; Enable Org Protocol to open links from outside Emacs
(defun org-protocol-open-link (info)
  "Process an org-protocol://open style url with INFO."
  (when-let ((link (plist-get info :link)))
    (raise-frame)
    (org-link-open (car (org-element-parse-secondary-string link '(link)))))
  nil)

(defun org-protocol-copy-open-link (arg)
  (interactive "P")
  (kill-new (concat "org-protocol://open?link=" (url-hexify-string (org-store-link arg)))))

(with-eval-after-load 'org-protocol
  (add-to-list 'org-protocol-protocol-alist
               '("org-open" :protocol "open" :function org-protocol-open-link)))


;; Modified Functions
(defun custom/org-export-resolve-id-link (link info)
  (let ((id (org-element-property :path link)))
    ;; First check if id is within the current parse tree.
    (or (let ((local-ids (or (plist-get info :id-local-cache)
			     (let ((table (make-hash-table :test #'equal)))
			       (org-element-map
				   (plist-get info :parse-tree)
				   'headline
				 (lambda (headline)
				   (let ((id (org-element-property :ID headline))
					 (custom-id (org-element-property :CUSTOM_ID headline)))
				     (when id
				       (unless (gethash id table)
					 (puthash id headline table)))
				     (when custom-id
				       (unless (gethash custom-id table)
					 (puthash custom-id headline table)))))
				 info)
			       (plist-put info :id-local-cache table)
			       table))))
	  (gethash id local-ids))
	;; Otherwise, look for external files.
;; ---------------------------------------------------
	;(cdr (assoc id (plist-get info :id-alist)))  ;; removed
;; --------------------------------------------------
	(org-export-resolve-id--external id)          ;; modification
	(signal 'org-link-broken (list id)))))


(advice-add 'org-export-resolve-id-link :override #'custom/org-export-resolve-id-link)

Compare with the other method and swap if you’d like.

Ofcourse if we use org-roam with all our ID – imples – there is no ID that is not tracked by org-roam.
We may simply use the org-roam-protocol with open-node

that is change the line in the helper function org-export-resolve-id--external

(format "org-protocol://open?link=[[id:%s]]" id))))

to

(format "org-protocol://roam-node?node=%s" id)

In which case we simply have to activate org-roam-protocol.el but – for my case, I like to define a new protocol and set it as such – this way we will be indifferent if and what IDs are tracked by org-roam

Test

(org-export-resolve-id--external "idvalue")

I have just renamed the function to show it is more general now and as always we plug this helper function in the appropriate function in the place where the ID is being parsed.

For the ox-hugo I tested it by exporting in a temp buffer - if you show me the output that you are getting - maybe we can work more on it.