Hello,
Newbie to the whole Zettelkasten / PKM / Emacs / Org-Roam world, so apologies in advance if I sound silly (I did try searching here first though)
I have put in close to 200 notes into org-roam over the past week now. And am linking them slowly.
My main issue has been finding entry points and for that I need a way to search my notes.
Like I have several notes on my photo management workflow and to get into it, all I remember is the term icloudpd.
Am I correct in assuming that I need another tool like deft or org-roam-search to get at them?
Or is there something really obvious that I am missing?
If you want to do it the Zettelkasten way, you would have one note as an index which lists key words and links to their related notes.
Other words for somewhat similar concepts exist, like āmap of contentā (MOC).
Simultaneously, you could work on outlines for potential publications (blog posts, papers, whatever), which would link to the notes that would make up the gist of the publication.
Of course, deft or org-roam-search can be much more convenient to discover and maintain content.
In the end, everyone needs to find their own system. You should discover, try, and adapt various things to find your way.
@schemar, thank you for your advice. this was very helpful in the long run.
my main source of confusion currently is, does org-roam let me search?
for e.g. like i said, if i have forgotten what the name of my index file is (which currently is the case. i am sure i will get better at this as i go along), then how do i go looking for it?
if so how? do i use a separate tool? or do use some obvious command that i have overlooked?
From these, Iām assuming you are looking for full-text search, not just searching the title of notes.
Org-roam does not have full-search capability and itās a good thing
There are tools dedicated to full-search and they work well with Org-roam in your Emacs environment. You might say these are āanother toolā; yes, but itās just an additional āplug-inā to Emacs and Org-roam to enhance your workflow. To me, this composability of dedicated tools in a single application (Emacs) is one of its attraction.
One popular full-text search toolkit is a combination like these:
grep and consult-grep
ripgrep and consult-ripgrep
grep and ripgrep are both command line programs that work in your terminal. consult-grep and consult-ripgrep are functions that are included in a popular Emacs package called consult.
Both work well on my Ubuntu machine and I use ripgrep on my Windows. You can look for files within your Org-roam directory (or wherever in your file system) that contain text string āicloudpdā for example.
Depending on your OS, grep might already come with it. Itās a classic full-text search program that has stood the test of time and still is widely used by many software engineers and system administrators who live in command line terminals.
ripgrep is a new incarnation of grep and is purported to be faster and seems to be gaining popularity. You would usually need to install it separately as instructed in its GitHub page ā in most OSās itās very easy to do with a package manager.
For additional information on consult-ripgrep with Org-roam, see the post below for a good detail and video demonstration.
The post talks about a custom function to fine-tune consult-ripgrep but you can use it (and consult-grep) with no custom functions.
This was comprehensively helpful, and cleared so many cobwebs
Thank you muchly @nobiot!
I am now using Deft to do exactly what you demonstrated, and Iām sure that Iāll be leaning less on it over time, as I gain a better insight into my entry points.
That leaves me with one curious question though
You mentioned
Org-roam does not have full-search capability and itās a good thing
Which means normal Org-roam folk have a different mental model to go start searching and writing and that is the something I wish to understand
How do you know where to start? Do you know / have in your head / have memorised you primary entry points?
Or do you like me use grep/Deft to start somewhere and then work your way throught the links?
What would be your workflow, if for eg. some term Photo management or Syntopical Reading just popped into your head and you wanted to write a blog post or article about it?
Would be grateful to hear your thoughts and workflow.
What @nobiot means is this: each tool should focus on its strength and only its strength. Org-roam is not a search tool. Grep is. Simultaneously, grep is not a tool for knowledge management. By that separation of concerns, each piece can make sure it is the best at what it does, instead of being mediocre at everything.
So what most people do is what you call a ātwo tool processā: they use, for example, org-roam with ripgrep.
However, I donāt consider that a ātwo tool processā, but a āone tool processā, the one tool being Emacs.
As @nobiot mentioned above, Emacs gives you the ability to compose small tools which are focused on one thing. And by that composition, something greater emerges. As a trivial example: org-roam and consult-ripgrep are great tools at what they do. The combination of both gives great synergies.
Hope this helps. Emacs plug-ins are not necessarily made to be used in isolation within Emacs, but rather to be used in combination. The concrete combination depends on the user (you) and their workflow.
Before responding to your question, I need to correct myself: āfull-searchā was meant to be āfull-text searchā. Apologies if this caused any confusion.
@schemar Thank you for your elaborate clarification.
I agree and this may be (only) one mental model that most Emacs users, with their diverse backgrounds and usages, might share. If you do not mind reading a long-form article, this blog and video presentation eloquently discusses this point. Iāll quote one paragraph that echoes with @schemarās āone tool processā statement, which I agree with.
Your focus should be on the workflow. On the whole system that helps you get things done with Emacs. Because that is what your actual experience is about. You benefit from the set of /emergent qualities/ that become available in the combination of otherwise disparate pieces of functionality.
On the question of a mental model of ānormal Org-roam folkā, I have been drafting a response, which has become too long. Iāll try to write a short version below and might make a separate post with the full version, if I get round to finishing it.
Itās perhaps not precise to think of āOrg-roam folkā having a single mental model and I think that each one of us has a different, somewhat overlapping, mental model about their notes in Org-roam. I find it useful to think of three typical modes of usage of Org-roam as follows. They are based on my observation on this forum.
Task management
Information / knowledge repository
Thinking tool
From what I have observed, everyone uses Org-roam with one or more modes of usage. In relation to your question on search capability, depending on the combination, our requirement for āsearchā capability differs.
As for myself, I have been using only the āinformation / knowledge repositoryā mode for work and striving to use the āthinking toolā mode for private (like this post). I donāt use the ātask managementā mode at all.
Task management mode.
I think one of the most elaborate workflows is that of @d12frosted. He has a great walk-through of his task management usage with Org-roam in his blog series with explanatory GIF animation and images. Most users of this mode seems to use Org-agenda as the way to retrieve and display TODOs from various notes in a dashboard-like presentation (refer to images from the blog post linked above).
Information / knowledge repository mode.
I think the key for this mode is quick search and retrieval of relevant notes through a large collection of them. Examples of this mode includes this post, which talks about a 15 million text files and 15 thousand OCRed books. Itās by far the largest information / knowledge repository I have heard on this forum. Mine for work is around 365 in the last 1.5 years of daily use (less than one note per day). If I remember correctly, I have seen 10,000+ notes mentioned a couple of times in Slack and this forum, and it appears 1,000-3,000 notes in less than 12 months are rather common.
For my usage of this mode, I personally use mainly the following:
org-roam-node-find with titles, tags, and the number of backlinks displayed in the list of candidates. I also use savehist-mode so that recently worked notes appear on top of the candidate list (there are several ways to achieve this, depending on your ācompletion frameworkā, such as ivy-mode, helm-mode; I use vertico-mode).
My candidate list looks like this (the configuration is described in our community wiki).
I only use four tags: #creation, #reference, #index, #meeting. Only the #creation tag is for āthinking tool modeā and the other three are different categories of information / knowledge. Tags can be made appear in the list of notes with org-roam-node-find, so itās easy to just to look for meeting and index files. The index note is one way to achieve the āmap of conceptsā as @schemar describes above. I have one index note per topic and have 16 index notes for my 365 work notes at the moment. The screen shot is for my private Org-roam. This one does not really contain useful information or knowledge; they are meant to be more for the āthinking toolā usage. I have 5 index notes for 217 notes.
ripgrep and consult-ripgrep for full-text search and retrieval. Itās really a great toolset to find a āneedle in the hayā ā a specific word, name or phrase that would be unlikely to be part of the title.
I use the ātime stampā as the key to look for something. All my note files are named with a creation time stamp in the format of yyyy-mm-ddThhmmss.ext. If I want to find a record of a meeting, I can easily find notes and other related files around the same date or month. This goes beyond Org-roam and Emacs. Most my work files have date prefix (yyyymmdd_). It is much more useful than it might sound. Much of my work related information is based on specific events ā trade shows, customer meetings, launch of a new product, etc. ā so it is a great way to look for all the files in the context of a specific event.
Thinking tool mode.
I think āthoughtsā should be differentiated from information / knowledge. The key differentiator is my own perspective and judgement beyond the record of facts; a good thought captures my own perspective built on a collection of information about relevant facts. My hope for me as a user of Org-roam is to be able to use it better as a tool that helps me think better. One good example is probably the usage of Jethro himself (the author of Org-roam).
As for my usage for the āthinking toolā mode, most of the elements I describe for the āinformation / knowledge repositoryā mode also apply, but I find myself using search by ripgrep less. The title and links usually take me back to the chain of thought I had when I was writing the idea and thought in the note.
In addition, I am considering of using more of org-roam-ui ā like this:
I would like to use it not to connect thoughts, but to prune (remove) stray notes, so that I can focus on the good ideas that I can develop over time. I find 217 thoughts / idea notes are bit too many at this stage ā some of them are those I made for even debugging and forgot to deleteā¦ As you can see in the graph, I have a lot of stray notes around the connected notes.
So this is it. This has become very long (even for a short version). Thank you for indulging meā¦ I hope some bits can be helpful to others.
@nobiot, this is probably the single most, comprehensive, helpful and kind reply to a query I have ever received, in my life. So thank you!
I donāt. Iām curious about what other folk in this world do. Ergo the persistent prodding at you folk
I am new to Emacs, new to Org, new to Zettelkasten, and then new to Org-roam which I have chosen as my weapon of choice to implement my garden of thoughts. There is tonnes of tacit knowledge in each of those domains that I lack and am slowly building
I think it is worth it, because this is critical to me and my life, and proprietary tools have burned me repeatedly (Iām looking at you Evernote, as I stand in the ashes of my 70gb database)
So I am pretty much exploring and asking folk how they do things.
Yes experience and doing it is the best teacher, but I donāt want to be hacking my way with a machete in the rainforest that is all my stuff, when all the while there was a highway that passed a few metres away from me or there was a zipline right above the tree canopy
My needs are primarily 2 and 3. (will stick to plain org for task managment, when life gets busy. right now it is just pen and paper)
And your explanation (someday, please do write the long version) and notes on what you do, have given me enough scope to learn and explore
I did not know i could org-roam-node-find tags. will do a lot more tagging now
I learnt we have a wiki full of tips and tricks! woohoo!
On using only four tags, I do both, tag them with nearly everything i can think off, as well as a few structured tags that I use most of the time to filter through. So I was, happy, that my approach is similar to yours
ātime-stampā is a neat trick! iām copying this!
Thinking tool mode, is why i am learning zettelkasten, because I have years and years of notes with very little to show for it. Two days of connecting them and I donāt have the blank page problem anymore. I might not write what I think I want to write (maybe that will come with experience) but something else (better or interesting) always pops out.
your idea of āback to the chain of thoughtā is what iāve experienced over the past few days too, which has led me to stop stressing about how Iād search, to a great extent.
my org-roam-ui mode is still a vast constellation of unconnected dots I am assuming that building the connections is the rest of my lifeās work
Your idea of not treating ideas as precious, of pruning them goes so much against the grain, against my mental model of hoarding, that I had a physical reaction But yes, it is an interesting contrasting thought to hold in my head, that this is a good thing too
Thank you, so much for your time.
P.S.
how did you handwrite on that screenshot? iād love to know!
Thank you for your kind comments. Iām happy that it was helpful to you. This discussion has been great to me too; I had been meaning to do a write-up of this sort for some time. Your question gave me the impetus, and it helped me clarify some of the things I wanted to clarify for myself.
Just a piece of paint software called Krita on Linux and an old pen tablet. Iām self-teaching digital drawing at the moment and I use (this pen tablet) ā as an aside, Iād advise against any pen tablets with no screen unless you are very keen to learn how to use them. Even many professional digital artists (on YouTube) say they need getting used to. Outside Linux, I use iPad and Apple Pencil for more casual drawing.
Two more comments: on use of tags, and on pruning
On tags, there has been some good discussion threads. I find the ones in which @rodelrod is involved are always informative (this on Slack, and relatedly, this on this forum). Just to cut the long story short, Iāll jump to my crude take of the gist:
Use tags sparsely to denote types of notes rather than liberally as topics
Use index notes and (back)links for topics instead
Of course, you are free to develop your own tags usage. Personally, I have been happily following this tag usage. My personal reasons are mainly as follows:
I have tried topics tags and realise I get inundated by too many useless tags
I had already been sticking to the usage of ātagsā as types (the only four categories I noted above) informed by this old article from 2009 ā it then refers to yet older discussion, which has been relocated to this forum post.
On pruning, you do not have to really delete the files from your system. I have a folder named āfleetingā outside Org-roam as a holding bay (keeping notes from 2011). I might just move pruned notes to it and come back for review later. Alternatively, you could have a subfolder for fleeting notes within Org-roam and show its name in org-roam-node-find so that you can differentiate your āthoughtsā notes from those āfleeting ideaā notes.
There is a bit more personal story behind why I am starting this āpruningā experiment, but Iāll stop here for now
Iām also keen to know the story of the 70GB of notes in Evernote.
Nothing really to it Like I said, Iām a hoarder. I read a lot.
Every thing I read, every beautiful turn of phrase in a poem, every moment in a novel that strikes me, every lightbulb moment in a non fiction book, every aha in an article gets clipped and put in.
The pileās been growing ever since a rich friend gave me a hand me down windows tablet in 2002, with Ritepen installed on it. Ritepen got to become or morph into Evernote and I moved from Windows to Mac and it moved with me, until 2017.
Thatās when some sync error, blew up the database. And no matter what I did, I could not get it back. My backup (on an external) was corrupted too. I had to learn the hard way, that sync is not a backup either
That was my last straw with the ānew fangledā stuff.
Evernote was going through too much turmoil those years to pay attention to client needs.
Google had killed Reader a few years before that and the one email client I loved, Sparrow even before.
Wordpress bought Simplenote and left it to wither on the vine.
It was almost as if every tool that I loved, seemed doomed.
That was also the time I learnt about the āLindyā mental model.
So I just flipped everything and began using old tools, slowly but surely.
And began putting in the effort to learn them, even if they were quirky
My transition to Linux took two years.
Slowly but surely I have tried to stick to old tools and open formats.
So that I am never locked in again.
Thatās what does not happen with me. The tags donāt bother me muchā¦
My primary usage is similar to yours. Most of my queries would be on types of notes, think about 5-7 at most
Iām just glad the other tags are there if I ever need them. (and I do, every once in a while )
Iām old enough to have seen the rise of tags as a solution against fixed categories and types and folders Operating systems got tags, so I could search for something common across folders. my original introduction to it was the labels as folders thing in Opera Mail in the early 2000s. So that is my mental model
But I agree with your sentiment, to each their own
Besides, the idea of Zettelkasten and Org-roam and the bi-directional linking is something I have just stumbled across recently, and am investing my time into. I hope it will reduce my reliance on tags over time.
(Iām doing this because I hope it will let me write more easily.
I want to write, I love reading my own words, but the effort kills me. I want something that will aid me.)
Aah. this is nice.
I told you mine I will wait to hear yours, someday.
Itās perhaps not precise to think of āOrg-roam folkā having a single mental model and I think that each one of us has a different, somewhat overlapping, mental model about their notes in Org-roam. I find it useful to think of three typical modes of usage of Org-roam as follows. They are based on my observation on this forum.
Task management
Information / knowledge repository
Thinking tool
This is an awesome write-up. This will be very helpful for new users. Do you think you could extend / move this some more discover able place (separate thread, blog, ā¦)? Or now that I think about it, I actually believe that the manual would be the right place. The manual currently has Section 2 āTarget Audienceā, which touches upon the flexibility of org-roam (āIt is our belief that note-taking workflows are extremely personal, and there is no one tool thatās perfect for you.ā) but then mentions only Zettelkasten. Expanding this with a few examples of use cases like the ones you mentioned here would be helpful for some users who may think that org-roam is primarily targeted at the Zettelkasten workflow. I donāt think itās primarily targeted at any workflow, while it really doesnāt dictate a workflow.
Iām somewhat biased here because this discussion reminds me of myself when I started to get in touch with org-roam (see my thread: Is it wrong to think of org-roam as a wiki?). Back then I felt org-roam is āorg with better linkingā and I still feel that way. My use cases of org-roam are really primarily 1 and 2 and only a tiny bit of 3, so back then I wasnāt sure if org-roam is the right tool for me.
Thank you. I was thinking of creating a new post with a longer/modified version of it. I donāt have capacity at the moment to discuss the feasibility of this passage for the user manual and create PR but Iād be happy for you or others to do so.
Haha, task management is not the only use case for me. Wait until you discover how https://barberry.io is built I hope to write a post about it Small teaser: these are private notes in my org-roam-directory
I also use it for taking litnotes. Though I canāt say that I have too many notes here.
My wine-related notes are numerous. Overall I have around 10k notes, despite me cleaning up time to time.
I am assuming that building the connections is the rest of my lifeās work 
