Is there a mechanism to easily identify broken links in V2?

How To Requests
28

Then it visits it.
If you press enter on a roam link you can create a node -

if you decide to create a node, you may want to repair all incoming links, on save org-roam will automatically convert roam links to id links if the node exists - we accelerate this process in the repair function.

29

Oh, my apologies. I was totally wrong.

What happens is that I have some IDs that are actually the same as the title.

So yes, roam links should point to valid IDs.

I think the links table is used to build the graph of connections.

1 Like
30

I had missed that specificity of roam: links, I thought they were using IDs too. That makes me completely reconsider your repair function.

Simply adding a common title (even a temporary one) instead of leaving it empty might actually be beneficial considering the minor interruption in the writing flow it constitutes when adding multiple links to the same ‘future node’ before creating it.

I am gonna have to do more testing tomorrow…

1 Like
31

Leaving here the relevant portion of [[roam:]] type links protocol in org-roam for future readers here.

Further note this variable – which may be customised to turn off this auto replace behaviour – in which case the repair function would break. It assumes this is t (which is by default)

(defcustom org-roam-link-auto-replace t
  "If non-nil, replace \"roam:\" links to existing nodes with \"id:\" links."
  :group 'org-roam
  :type 'boolean)
32

Correcting my error. roam links use only the ID as a destination.

But you can use any use an ID for a destination that does not exist if you want to create that “future link”. Then when you follow it, org-roam will ask you for the template to use for this new node, create that node with the ID that you specified.

When you save this new node, the link to it will no longer be dangling (and still use a meaningful id).

33

How does it use ID?

(defun org-roam-link-follow-link (title-or-alias)

It defines itself to accept title-or-alias.

34

I think it is better to simply send the user to the location of the link. If the user follows it, org-roam takes over and prompts to create a node with that ID.

If the user does not like the link, the user can replace it with a valid one, or remove it altogether.

But I see a very valid use case for your processing: changing a link that appears in many documents to a new destination everywhere it appears. that would be awesome.

I guess what I am saying is that there are different use cases that require different processing:

  1. Destination does not exist, create it.

  2. Replace a link destination (whether it exists or not) with a new destination everywhere it occurs.

  3. Destination does not exist, fix only this link.

35

I am starting to think you have not even tried the functions I gave – because the find-broken-link does exactly what you are saying…
Please try the functions atleast :sob:

The functions workflow exactly match what you are saying :sob:

Simpler to use a temp title – then create a node with the temp name – then repair all incoming links

then just simply change the title – ID links are resilient to this change.

…But it would break link descriptions.

I will work on this, should be easy to do.

36

Yes, I didn’t try them. But I looked at their name and description:

(defun org-roam-find-broken-links ()
  "Find broken links in Org-roam"

I assumed this function returns the list of “broken links” (though the SQL query returns all links).

But now that I am looking at the code, it does not do that.

Looking at the code it seems that your implementation indeed does what I suggested.

37

I will implement changing all [[roam:]] links with a common title to something else- without creating a node. This will allow OP to not spend time thinking what to name them when writing. Ideally a random common title should be used – if a user spends time what to name it from beginning - it would obstruct the writing workflow.

But I want to add this point – [[roam:]] should be used with title-or-alias only and it does not use ID, look at its implementation in the code I posted above.

OR,
The repair mechanism may be augmented with this. I will think more tomorrow.

38

Thanks for the link to the variable specifying whether links should be changed to ID or not. I didn’t know about it. It seems that it is possible to have links by title or alias then. it is that by default, when the file is saved, the link is replaced to use the id.

I am not sure i like that automatic behaviour. I think titles are more descriptive, but, they are more likely to become invalid (specially if the of the node is changed). I wonder if roam deals with this case already (if I change the title of a node, and the old title is a destination, fix the links to the new title). Titles are problematic because they not guaranteed to be unique.

There are soo many potential issues with using titles or aliases for destinations. I see why org replaced them by default.

40

I think this would be great. I think it can be a “stand-alone” since you don’t need to change anything inside org-roam to make it work. A suggestion? why don’t you create a repo and work on providing functions with each of the use cases?

I think that the only people who get affected by the hooks to file saves are the people who have a HUGE file.

by the way, which package handles an ID type of link?

41

I finally had some time to test org-roam links behavior and your repair function.

First of all, thanks for clarifying the actual functionality of roam: links, it makes the whole discussion clearer.

After a few tests, I am definitely convinced by your repair mechanism. It works very well, I was surprised that all my test links were repaired without any action on my part, just by saving the newly created node’s buffer. I originally thought that I would have to call it interactively, or add it to a ‘buffer save’ hook myself. That must be what you refered to as the ‘buffer-save’ protocol in some of your posts. I was not aware of this mechanism. It makes the process completely transparent so there is actually no reason I can think of not to use it. Except maybe the buffer saving performance issues you talked about, but I have not experienced this kind of inconvenience so far.

As I was saying earlier, this repair mechanism only makes sense to me now that I understand how to use roam: links properly. It would integrate very well with the first instinct I had to give temporary ‘placeholder’ titles as the roam: target ID for my dangling links while still in the early writing/canvassing process. This does not constitute a major interruption of the flow of writing IMO because, even if I need more consideration to find a descriptive and meaningful title, I instinctively always have some kind of variable name(s) in my mind to address the content of that ‘future node’ when I think about it.

I should add that where the repair function would really shine IMO, is when completing a writing project the ‘Zettel’ way using many distributed notes. When I started this thread, I was writing a structured manual in on single file, and wanted to link sections not yet created which I also use as a reminder/task for the need of a future section. But I was always reluctant to write structured pieces in a distributed manner, like a note per chapter or section by fear of it being impossible to maintain. Writing is an iterative process that requires a lot of reviewing, restructuring and modifying (depending on the nature of the piece of course), and I never felt comfortable starting a project this way. Maybe I will give it a try using roam: links and these two ‘dangling links’ functions.

So thanks for this addition, I think that was a very good idea, and really improves the writing and reviewing process in my opinion.

This brings me to your ‘node title change propagation’ project. If I understood correctly, the idea is to update all id: (and :roam ?) links pointing to a node which title is being modified. The description part of those links would then be updated to reflect the new node title state. I also assume that a mechanism would be in place so that only links descriptions are only modified if their description matches the old title state of the modified node.

To illustrate my assumption more clearly I hope:

  • Modified node: id = 999, old title = ‘foo’ , new title = ‘bar’
  • link 1 = [[id:999][desc:foo]]
    • link 1 becomes [[id:999][desc:bar]]
  • link 2 = [[id:999][desc:something else]]
    • link 2 remains [[id:999][desc:something else]]

If my understanding of your ‘title change propagation’ idea is correct, that would answer a persistent frustration of mine across all my attempts at ‘Zettelkasten’ implementation. Namely, the difficulty of maintaining structure notes link descriptions when modifying notes titles (here is a link to an article describing the concept of strucuture notes in case that helps some readers).

Even if I rarely use the node’s titles in links descritptions, I always do in structure notes, and it is an important part of the ‘Zettelkasten framework’. Personally, I think this could be a nice addition as well to keep notes up to date and organized with less effort.

I would have reservations though for a user who actually manages to use node titles as is in their link descriptions, and semantically integrate them in sentences. For them, that could end up messing up their notes meaning…

44

I spoke too fast when I said ‘never’, I actually do use links the way you describe a lot as well for less formal notes, more technical ones, or those that are meant for me only :slight_smile: .

Those functions are very useful for both use cases anyway IMO.

45

Okay done

  • Created a branching execution path “RENAME” that avoids renaming non-standard (custom) descriptions - to do this use C-u prefix or with RENAME set to non-nil

C-u M-x org-roam-repair-broken-links OR
(org-roam-repair-broken-links t)

The main branch logic remains the same to rename all - not sparing even custom descriptions, because by default when we insert a node: org-roam formats the description as the title – so this should be the default, whereas the branching RENAME logic makes sure to not rename custom descriptions.

  • Made formatting of candidates while finding broken links better.

  • Misc fixes

ezgif-5-564cf3f9c5

Let me know if you face any problems.

Best,

46

Thanks. I will test all the modifications as soon as I can.
Hope I can get to it tonight and report back.

47

I realised something…

[[roam:title][description]]

is a valid way to make these links – and it also incidentally behaves very well. I did not plan for it - but its not broken – repair wont destroy the description.

So while writing this strategy can be used – namely: use description to write what’s on mind very fast and the title to something temporary that won’t tax cognition too much.

48

I agree with this logic.

Thanks for including that use case as well.

I have tested both new functions, I have no issue with find-broken-links.

For the repair function, I have some issues, but it is getting late, and I have limited time to do better tests today. I am gonna try to describe what I did so far, I will try to answer more questions tonight if you have some, but testing further is going to be complicated until tomorrow.

  1. Testing (org-roam-repair-broken-links)
  • New node: id = 999, title = ‘foo’
  • link 1 = [[roam:foo][desc:foo]]
    • link 1 becomes [[id:999][desc:bar]]
  • link 2 = [[roam:foo][desc:something else]]
    • link 2 becomes [[id:999][desc:something else]]
  1. Testing (org-roam-repair-broken-links)
  • Modified node: id = 999, old title = ‘foo’ , new title = ‘bar’
  • link 1 = [[id:999][desc:foo]]
    • link 1 becomes [[id:999][desc:bar]]
  • link 2 = [[id:999][desc:something else]]
    • link 2 becomes [[id:999][desc:bar]]
  1. Testing (org-roam-repair-broken-links t)
  • Modified node: id = 999, old title = ‘foo’ , new title = ‘bar’
  • link 1 = [[id:999][desc:foo]]
  • link 2 = [[id:999][desc:something else]]
  • I get a prompt Enter new node title, and what I thought was the usual default value between brackets appears on the next line Currently [ bar ] :
  • I hit Enter leaving the input field blank,
  • The #+title: bar line at the top of the current node is removed, nothing else happens, except nil is printed at the bottom.
  1. Testing (org-roam-repair-broken-links t)
  • Modified node: id = 999, old title = ‘foo’ , new title = ‘bar’
  • link 1 = [[id:999][desc:foo]]
  • link 2 = [[id:999][desc:something else]]
  • I get a prompt Enter new node title, the usual default value between brackets appears on the next line Currently [ bar ] :
  • I input bar and hit Enter,
  • Every single https:, id: and roam: links I had in the file where I added my link 1 and link 2 gets modified to bar except link 1 and link 2 :scream:

Tests 1 and 2 went as I expected, I was surprised to get a prompt for test 3 but looking at the GIF you posted, I understand now you redesigned the function as a complete renaming process; I think it is fine as well, but I would have expected the function to try and repair links with the current value if I entered nothing. As for test 4 I gotta say, I did not check how the links were affected beyond the description displayed. I kind of panicked and rushed to hit undo.

I was sloppy I tested code on the actual file I was working on these last few days, this is my bad. Actually I do not have any safe process in place to test new stuffs on Emacs, I did not get around it yet, I was mostly racing to get a minimal setup and start using my new toy. I only hope other links in the rest of my notes were not affected the same way or that at least, the undo command reverted all of them if it they were.

What is troubling me the most, is that you posted a GIF of the same process working fine so I do not really understand how it can behave that differently for me. The previous version worked fine too… Let me know if you have any idea what could be causing such unexpected behavior on my end.

49

This is exactly how I intended to do it. When I started the thread, did not understand roam links turned into id links, and did not have the repair function, I was using empty roam links for the same purpose i.e. [[roam:][semantic description]]

50

I think I failed to get across the design – maybe I shouldve broken it to three functions instead of two…

The way the rename works is that, you shouldn’t leave it blank – I did not plan for this, the intended workflow is that you already have everything correctly in place – no roam: links and so on – and you just want to rename the links and the title itself.

what is shown inside is the current value – the value that is to be replaced. I did not expect youd run rename when trying to execute repair. There is actually 2 functions inside that one, its compacted - both these functions have different workflows.

Without a rename - it is not possible to save custom descriptions – because somehow we need to get the value to compare against and also make modifications from there – so it is not possible to save custom descriptions in the first case.

case 4 should be impossible – it doesn’t affect any links except “id” and “roam”