Org-transclusion -- now on ELPA!

Hi nobiot, quick question that is off-topic - what font are you using in your demo? It looks fantastic.

1 Like

I keep tweaking the fonts I use; I believe these are the ones I used in the demo (and my current setup):

(set-face-attribute 'variable-pitch nil :font "Concourse T3-13")
(set-face-attribute 'fixed-pitch nil :font "Go Mono-11")
(set-face-attribute 'default nil :font "Cascadia Mono PL-11")
(set-fontset-font nil 'symbol (font-spec :family "Segoe UI Symbol" :size 11.0))

Concourse is a paid font – I think it is the one you refer to.

It might look “expensive” on its own, but I have never regretted buying it (together with Equity – used for the title picture). Both are beautiful, and I bought them also to send my appreciation to the creator for his other work (Pollen, and Practical Typography book).

Segoe UI Symbol comes with Windows (used for Org superstar headlines, and some indicators).

The other two are free to get (from Google, and Microsoft, respectively).

I thought it was one of Matthew Butterick’s! I think I am going to have to get it as well; it is too pretty not to. Thanks for the info :slight_smile:

1 Like

This is a really wonderful extension.

I liked the behaviour with in place edit. I wonder if that could be enabled as an option with a deliberate keystroke combination like the default C-c n e ; then a key combination to indicate end of edit. Normal save syncs the background buffer.

Would this avoid the multiple region sync problem, and sync to source on file save?

I think this issue is related to your ask.

Although I believe this feature is many users expected, I am also trying to understand nobiot’s idea first.

:white_check_mark:
That does look like the same issue / feature request.

nobiot’s thinking makes sense from the perspective of offering the filtering option,

I’d like to see a customization to allow toggle r/w, edit in place; sync on save; toggle r/o.

I think, but we can wait to see his response and how the work progresses.

It’s a great improvement to work flows for re-use of and organizing content.

Hi @david, thank you for your interest and feedback.

Just quickly… I might need to come back to this post to elaborate a bit more.

For the “in place edit”, I am afraid I will need to defer to a later time (April 2021).

My motivation and technical challenges are outnlined in the links below. Technically it is very challenging for me at the moment, and I don’t find it essential for my writing process.

Having said this, I would like to come back to it. I have been looking at the following implementations, and got some new ideas about how I might be able to tackle the technical challenges. Especially Org-clones. I think it is brilliant in that it separates the headline between the “body” and “heading”. I might be able to use this idea (I need to see how this goes, though).

Here is the timeline I have in mind.

  • 1-3 more updates within this year, but mainly for UX improvements – I am working on some features to adjust headline levels, and re-arranging the sequences of transclusions – this is for me number 1 priority for the writing process
  • Ask the Org Mode maintainer(s) if they are intersted in including Org-transclusion in the standard Org Mode codeline (“upstream”?) – this has been suggested in the Reddit conversation, and I thought I would lose nothing if I just politely asked :wink:
  • January–March 2021, I will need to focus on my book writing (I am guessing the updates to Org-transclusion will be sparse) – I also want to look at the documentation for Org-roam that I have promised to do, which I have been putting on the backburner (sorry)
  • April 2021, book done (this is the schedule the editor currently wants), I will be back to Org-transclusion; and Org-roam documentation

hmmm. Maybe a bit optimisic, but hey, I will see how this goes.

3 Likes

New video #6 is up 2020-12-06

This is probably my last update within 2020. Thank you for all your advice, feedback, and discussions. I will be quiet and will come back to Org-transclusion after April 2020.

GitHub repo:https://github.com/nobiot/org-transclusion

This version’s highlights are:

  1. Control the headline level for transclusion
  • :hlevel n parameter for #+transclude:
  • Keyboard meta-left/right (equivalently shift-meta-left/right )
  1. Move transclusion up and down ( meta-up/down & shift-meta-up/down )
  2. Option to transclude the first section with filter
  3. Better default theme – switch light and dark themes (thanks to Prot)
  4. More robust failure handling
  • Revamp handling of Org-ID – it should work with Org-roam and normal
  • Avoid deleting links when transclusion fails (including Org-ID)

Default dark theme.

Default light theme.

6 Likes

Nice work!

The question that pops to my mind is why use a property :hlevel instead of using a normal org-mode heading level?

For example
Instead of :

* My book
#+transclude t :hlevel 2
[[Chapter1_link]]
** Chapter2
   Lorem ipsum

Why not do it that way:

* My book
#+transclude t
** [[Chapter1_link]]
** Chapter2
   Lorem ipsum 

Or even this way, which would be, in my opinion, even more clear and “standard org-mode”:

* My book
** [[Chapter1_link]]
   :PROPERTIES:
   :transclude: t
   :END:
** Chapter2
   Lorem ipsum 

That way:

  • when folded it would look like clean org-mode stuff
  • and when promoting/demoting headings would play well together, transcluded or not

Anyway thank you again for your work, as I say, giving ideas is easier than to code them, and as Zaeph showed during emacs conf org-mode code is… supermassive…

And have a good time until next time.

1 Like

I think… the very honest (and historically correct) answer is that this never occurred to me :wink:
You might have given me this idea before, but adding a link to a headline is not something I do, so very foreign to me – that’s probably because it didn’t occur to me.

Restrospectively, though, if I try to think it through and put a more thoughtful facade covering this historical fact, here is my line of thinking.

This is problematic, because #+transclude: keyword is part of * My book section, not part of ** [[Chapter1_link]]. It is very important that the keyword and the link are “always” together. This construction will break them apart, if you re arrange “** Chapter link” with another headline of level 2, for example.

To keep the keyword and link together, this one is a better solution in my book. However, the issue is that it does not work when you are transcluding non-headline elements, such as paragraph, block quote, table, and so on. I do not like to have two different ways for transclusion – a property drawer for a headline, and a keyword for a table, for example…

What do you think?

I see why you went the #+transclude way, it works well with block quotes and tables (and paragraphs I guess).

To me it feels natural to use headings/entries to be transcluded, its the org-mode natural units of knowledge, for example, it can easily be refiled (but tables or paragrahs cannot, if i’m right).

Maybe there is a solution for both way… If I’m not wrong, properties, say like category can be defined
at file level with

#+CATEGORY: loremipsum

and at entry level: inside a property drawer

:PROPERTIES:
:CATEGORY: lorempipsum
:END:

But it is the same thing (it is inherited in fact), not really two different ways, is it?

Also, about this not working for tables and paragraphs, would :hlevel work? (sorry not much time to check myself now)

That’s correct. You can add hlevel to a table, but it does not do anything.

Thanks for all the ideas. I am sure i’ll come back to these points at some point :+1:

1 Like

@nobiot, just checked the github page: great to see you returning to active development on this!

1 Like

The new v0.1.1 brings back live-sync edit feature with much more.

New video #7 is up 2021-05-16

6 Likes

I was intrigued by this cool-looking org-transclusion package and gave it a try today. But I can’t get it to work :frowning:

Here’s the file referring to another node, hoping to get its contents transcluded into it:

:PROPERTIES:
:ID:       210504_080012
:END:
#+title: Carsten Dominick

Wrote [[id:210504_080047][Org Mode]]

#+transclude: t [[id:210504_080047][Org Mode]]

And here’s the pointed to file with the to-be-replicated simple header and line of content:

:PROPERTIES:
:ID:       210504_080047
:END:
#+title: Org Mode

* Summary
Org Mode is a popular package for [[id:210504_075938][Emacs]]

When I call org-transclusion-add-at-point, I get the disappointing message that Nothing done. No content is found through the link at point 110, line 8.

Am I using it wrong?

It’s probably because the ID is not recognized by Org-ID.

I could reproduce your issue and could also fix it.

Try this to see if you get a marker returned.

  (org-id-goto "210504_080047")

;;; EDIT sorry, I meant this function for marker
(org-id-find "210504_080047")
;;; org-id-goto takes you to the buffer -- it might give the same insight but not sure ;)

If not, you would need to update the hashtable and locations file:

  (org-id-update-id-locations (list (buffer-file-name (current-buffer))))

We discussed a solution to update Org-ID for externally created files or manual ID elsewhere. It does not seem to work within Org-roam any longer. I now have the following to ensure org-id-locations-file is in sync with Org-roam cache for my IDs.

(advice-add #'org-roam-db-insert-file :after
            #'my/org-id-update-location-at-org-roam-db-insert-file))

(defun my/org-id-update-location-at-org-roam-db-insert-file ()
  "Update `org-id-locations-file' and hash table.
It's meant to be used with `advice-add' :after
`org-roam-db-insert-file'.  We can assume that this function is
run wihtin a buffer visiting a file being inserted, as
insert-file is run within `org-roam-with-file' macro."
    (when-let ((id (org-entry-get 1 "id")))
      (org-id-add-location id (buffer-file-name))))

This is a really neat module. I’m trying to build a workflow based on it: I annotate documents with org-noter and want to keep the notes in separate files, displaying them through the transclusion. I came up with the following function so far:


(defun insert-transclude-note ()
  (require 'org-roam-protocol)
  (interactive)
  (let ((node-title (read-string "Enter node Title: ")))
    (org-roam-protocol-open-ref (list :template "t"
                                      :ref "" ;(buffer-file-name)
                                      :title node-title
                                      :body ""))
    (org-id-get-create)
    (save-buffer)
    (let ((node-id (org-entry-get nil "ID")))
      (message node-id)
      (kill-this-buffer)
      (insert (concat "#+transclude: t [[id:" node-id "][" node-title "]]"))
    )))
(setq org-roam-capture-ref-templates
      '(("t"
         "transclusion ref"
         plain ""
         :if-new
         (file+head
           "${title}.org"
           "#+title: ${title}\n* ${title}\n- ")
         :immediate-finish t
         :jump-to-captured t))

It prompts for a name for the note and creates it in the background, and includes a corresponding #+transclusion t [...] link at current position. So far so good. However, when I try to enter live-sync edit mode it errors with No live-sync can be started at: <pos>. Now when I go to the buffer that was created, append a little bit of text, save, and go back, the livesync editing will suddenly work. Does anyone know why?

On a side-note, the livesync edit currently opens a side buffer (overwriting the roam buffer for me). Is it possible to turn that off?

Thanks for trying the package. Author here :slight_smile:
There is only one location where we would see that error message at the moment. The cause of it is that the program could not find the beginning point of the source for live-sync.

It looks to be an elaborate workflow you are building and I can’t really follow it. Does the source buffer (the buffer you are transcluding from) exist as a regular buffer at the point in time when you call live-sync?

I wonder if it is a timing issue, or “appending a little bit of text” changes the behaviour (I am assuming it’s a timing issue).

Regarding your second question, no at the moment. It is done byorg-transclusion-live-sync-display-buffer; remember the current window configuration, delete other windows, and pop-up the source next to the live-sync transclusion buffer. The only way currently is to override this function.

I will see if I can do about the windows down the line.

Yes indeed, the source buffer is just a regular buffer. My goal is that when I annotate a file (pdf) with org-noter, I don’t want to have one huge monolithic file that contains all the notes. Instead I want the org-noter document to be a collection of #+transclude references to small atomic notes. But I’d prefer to not leave the org-noter mode in which I have the corresponding pdf open side-by-side with the document including the notes (in the form of transcludes.) So for being able to stay in that side-by-side view, I’d like to use the live-sync feature. Hope that makes it clear.

In this part, try inserting more than a single “-”.
I think what’s happening is that a single “-” is parsed as the beginning of a list but without any content in it. I don’t think a list with empty content can work with live-sync.

For example, if it is “A” – it would be a paragraph. No problem in starting live-sync.
It could be like “(insert here)” or whatever, but not just a hyphen “-”.

1 Like