Thank you. I’m not familiar with org-babel-tangle, but I like how org-edit-src-code works on a block of source code / example within an org file. I am guessing org-babel does something similar with blocks of code within an org file. My current understanding is org-babel-tangle is to export code blocks of various program languages; where org-edit-src-code enables you to edit languages within Emacs.
My initial inspiration came from org-edit-src-code, went away to try text-clone for real-time sync. I am now coming once again back to org-edit-src-code way with additional learning from this journey.
As @YveksK, says above, I would like to focus on the “one thing it does well”. Need to define and refine this “one thing”
I’m thinking of an alternative approach. Below is a quick sketch to open it up for people to see, and also to record my current thinking (for myself).
I am glad that I have tried the real-time sync with text clone. But as I mentioned in demo #3, there are technical challenges. I have a feeling that we don’t have to solve them. As @YvesK and alphapapa suggest, I am thinking that real-time sync is not part of what this implementation of transclusion should strive for.
I have essentially the same use case in mind as @YvesK does (for myself too):
I am currently thinking of trying out an approach illustrated in the diagram below. The buffer at the bottom is going back to the original idea of org-edit-src-code with a twist; can we open the source or clone buffer properly narrowed? This way, you would be directly updating the source by yourself – org-transclusion would not have to be the middle person working hard to keep the copy and source in sync.
“Do one thing, and do it well”… Syncing does not have to be “it”.
I don’t this is a very good idea for the following reason: the point of transclusion into one file is to view the file as a whole, to view the context around the transcluded part, opening the transcluded part into another buffer somehow defeat the purpose, you could as well open the file itself then.
Though about it during some sleepless night… What I would try if I had the time and the skill:
simply refile the entry and prevent removing it from the original file
the function already exists!: org-refile-copy
Use a file directives #+PROPERTY: transclude or org entries :PROPERTIES: :transclude:
when some file is saved/modified, look for backlinks, when one has the transclude property and has a transcluded link to one of its entry, on save, do the update
when some file or entry is saved, and has a transclude property, look for the forward links and do the updates
use refiling mechanics for doing the update (delete old entry, org-refile-copy new)
Option to add a warning/confirm dialog to modify forward (and backward?) links
transclude property could have different flavours/options like
nil (don’t transclude)
t (transclude all links)
siblings (only links inside same directory links)
children (only links inside subdirectories links)
family (siblings+children)
heading (only links inside subdirectories with the name of the parent heading or filename if without heading)
limit to one level transclusion or make it recursive but then what about infinite loops
and/or have a variable defining max number of sub-transclusions
transclusion works only on begining of line or heading links
transclusion works only on file org heading links or org ids to org entries
since it is not full real transclusion I would chose another name for it:
transdex, transdexion, transdexing
relative to the idea of the index/toc file
I agree. This “clone buffer” at the bottom of the diagram is only for editing the source. This way, you would edit one source at a time, directly in the (clone of) source.
You will see all the transcluded text contents in the buffer that is doing transclusions.
I will consider to keep the transclusions in the transclusion buffer (left) when you are editing the source (right). Currently, because moving the cursor does “remove-all”, you edit without the rest of the context.
Yes, in your example, say if the content of your table repports to the content of Heading 1, not seeing it any more is kindof defeating the point of putting it together in one file in the first place.
Maybe your first idea of updating the parent/child transclusion with a hook at save time is better because you see all you want to see (either the mother transcluding file, either the child transcluded file) at all times. (even if because of not real time updating you don’t see both at the same time, but what for, since it is the same data?)
Now of course it is much more easy to have ideas than to code them
Now that I have basic ingredients laid out on my kitchen countertop in a more or less modular way, I can assemble them and cook up different dishes rather quickly.
Figure 1. Source edit buffer (right) with keeping the transclusions (left) with my annotation
I am still swinging between the ideas I have already explored, so your comment is really relevant (thank you!).
I find that the major problem of the first approach (and also to some degree, the second one with real-time sync) was that Org-transclusion had a piece of functionality that saved the original source from the transcluded copy – it deleted the part, and inserted the updated content over. You didn’t see it on the surface, but I was not convinced that it was the best for “data integrity” (well, it still could be “good enough”).
This new approach is that Org-translusion just opens a clone for you, and you yourself are editing the source directly; no middleman.
I am biased to this new idea, of course, but my subjective “user experience” is that it feels “solid” – it does not feel like holding a flimsy plastic case that can break any minute, but solid material like the type used for an iPhone, if this analogy makes sense.
Changing approaches is now rather quick, so I am still flexible. I should be able to merge the code in GitHub and do the next show & tell soon.
One question: maybe I misunderstood, but is it not possible anymore to have normal links in the buffer that are not transcluded if you start the transclusion mode? I use quite a lot of links and I would only want transclusion for some of them.
A follow-up on what @wuqui is pointing at as well. Maybe transclusion would benefit from an extension to the current Org mode syntax? Something akin to how Markdown-links are transformed to images in … Markdown maybe? By prepending an exclamation point to the link itself.
Normal link: [[id:reference]]
Transcluded link: ![[id:reference]]
(At the same time, the current handling of image links should potentially be extended as well, to also require an exclamation as a leading character… For congruency. In a way the semantics would be in line between images and transcluded text that way. ![[link to text/image]] = “Include [[link to text/image]]”.)
The current main branch feature is that the link must be at the beginning of a line.
So… If you want to “disable” a transclusion for a certain link, then you add some text (or even space) in front of it.
In the demo, I had this. I didn’t point your attention to it, but it is a detail.
There is a good debate with alphapapa on Reddit on this very point.
I am not committed to either of the ideas: (1) use standard org file: link type; or (2) use a custom link type such as org-transclusion:.
I am leaning towards (1) at the moment, with the addtional syntax like ![[file:path/to/file.org::*headline][description]].
I feel that this way you can make use of (potentially) Org-roam’s backlink (not sure if it really works yet) or Org’s standard functions such as org-store-link and org-insert-link without any additional functions or extensions…
Left a few comments in the linked thread. I hadn’t noticed the space sensitivity that you mentioned. Maybe that is enough for v1 of this. Though I personally would still advocate the leading !.
Regardless of the space or exclamation point issues, this is great work!
1/ Having a look at org-paste-subtree, you understand emacs (and org-mode) (relative) success at being a self documented - extensible without limit, err text editor: you need something, and most building blocks are already in there…
2/ Anyway, in the demo you showed many links types but you didn’t with the simple org heading link of the kind:
3/ Having some links being transcluded and other not, I think too, that it is a good feature.
I prefer to use the standard org-mode functionalities for that matter and others too. Using standard org-mode stuff insure that your files are compatible even when you stop using the transclude package. It also provide the whole already existing lisp ecosystem. Therefore I would suggest using properties, at file or org-entry level :transclude: t/nil. You could also have a configuration variable org-transclude-always t/nil that would set the default for all files.
Loading /home/yves/org-transclusion-paste-subtree.el (source)…done
Loading /home/yves/org-transclusion.el (source)…done
if: Symbol’s value as variable is void: window-selection-change-functions
I could not find nor in emacs describe, nor in google what/where is this window-selection-change-functions variable.
I believe it would come from a package you have on your system?
