The State of Org-Roam for Actual Beginners

[cobblepot]

I’m giving up on Emacs, Org-Roam, Org-Ref, and ORB and since I’ve spent a lot of time in the community during my 6-week experiment I wanted to leave you with my thoughts in case they are useful to anyone. None of the following is intended as a criticism.

Who should try org-roam?

TLDR; only people who are already comfortable with emacs or want to learn lisp programming (and maaabye others but only if they are not tied to Windows OS).

So you have some context: I am a Windows user, tied to that ecosystem, and I’m probably in the top 1% of technical ability for people who are not engineers, computer scientists, mathematicians, or in related fields. I build my own PCs from parts, I can do very basic scripting in AutoHotKey, and I live and die by keyboard shortcuts. But I generally use MS Office and have never previously used emacs or vim or Linux.

In the org-roam community and the emacs community in general, I am probably in the bottom 1% of technical ability. And it seems that only about 10% of the users in this community are on Windows. Many times someone trying to help me troubleshoot gave up when learning that their non-Windows solution didn’t fix my Windows install.

Routes For Emacs Newbies

People who have not used Emacs before and want to try it for org-roam have two obvious routes: vanilla Emacs and Doom Emacs. Getting vanilla Emacs set up and workable with these packages is not feasible for someone who does not want to really learn programming as an end in itself. As helpful as people are in various channels, no one is willing to spend the absurd amount of time needed to troubleshoot someone else’s Emacs setup and get it working (or at least, not in my case when using Windows).

Doom Emacs is not the Answer

As others have mentioned elsewhere, Doom Emacs cannot be used without lots of configuration requiring specific knowledge, even when the org + roam flag and biblio module are used (at least currently, maybe this will change). Many people have tried to help me get a very basic working set-up–nothing fancy, just getting the basic insert link etc. commands working. And they failed. Recently, I went looking for a set of Doom config files that would work on my machine. No luck. Everyone has modified their specific config in ways that prevent simply copying those files into my setup; syncing, updating, launching Doom Emacs; and ending up with the basic functionality working.

I recently tried to use @nobiot`s config files since he is on Win 10. They didn’t work because he uses custom modules. He suggested that I try his guide to get org-roam working on vanilla emacs (because Doom emacs hides many things that might cause problems) and discouraged me from using anyone else’s config straight because individual configs are complicated and hard to troubleshoot and because making my own config is considered a central part of learning Emacs. Nobiot has been the singular most helpful person so far in getting me going, but his point shows that Emacs isn’t good for someone that just wants a notetaking solution. I’m not interested in learning Emacs for its own sake. I have no use for learning Lisp outside of org-roam. I just want a notetaking and writing setup. I’m willing to spend a lot of time learning how to customize things to my liking, and I’ll learn scripting or lisp as needed to make that happen later, but the point is that I cannot get even a basic functional setup right now without learning Emacs programming.

As a side note, as great as Doom is, its dev Henrik has been gone from the Doom Discord server for a week (on vacation, I think) and it has really highlighted how central he is to the enterprise. When he is there I can be confident that I will at least get some direction on system-breaking problems within a day or two. Now, it’s nearly impossible.

Suggestions for the Org-Roam community

There was recently a hubbub on the Slack channel from someone’s rant about how hard it is to get org-roam installed. He suggested that the org-roam community “can’t be happy” that their solution isn’t accessible to the masses. What’s become clear to me is that the org-roam community (and I think the Emacs community in general) does not care that much about whether their tools gain mass acceptance (which is fine) From my perspective, what they/you want is to have tools that work for you in the way you want, and you like solving problems and sometimes working together on them, and that many of you like helping others in general, and would be perfectly happy if newbies got into emacs, but these are secondary benefits and not the real goal.

This is clear to me because several people have offered to write “org-roam for beginners”, including me, and the offers were generally met with collective silence. People say “Great!” but generally the people who want to write for beginners don’t have the knowledge to create the necessary manuals, and the people with the knowledge don’t want to write for beginners or spend the time working with those who do. Nobiot’s “Zero to Emacs to Org-Roam” is an exception, but it’s for vanilla Emacs, and thus only appropriate for beginners who want to learn vanilla Emacs - i.e. people who want to (essentially) learn to program first and note-take and write second. I’d write up (actually have been writing up) Doom emacs and org-roam and org-ref for beginners, but I don’t know enough to get it working, and no one with the knowledge has been available enough to move me forward in gaining that knowledge.

I’ve had maybe a dozen people offer to contribute to the Doom/org-roam/org-ref documentation that I have already spent at least 15 hours writing, and every one of them bailed. That’s fine, it’s their free time. But people are often more interested in sharing their solutions with others rather than they are in getting other people to a point where they have working setups. I’ve seen many people post configs and others write “great write-up! I learned a lot.” But when I try to duplicate the config, although many of these config posters will answer some specific questions, they lose interest after posting, because they have solved their problem and shared their solution that’s what they are primarily interested in doing (again, not a criticism). No one I have met so far in the community with the technical ability to get these packages working is actually interested in spending the time writing up truly basic, general documentation that covers many use-cases because they are more interested in the technical challenges of making the software work for them. And that’s fine, I’m not saying anyone owes anyone anything, but it means that at least at this point of development this community should discourage anyone who doesn’t want to learn programming from using org-roam.

I encourage those writing the Org-Roam manual and doing tutorials to be clear up front about their audience, which (whether they know it or not) is people familiar with Emacs, and they should say that up front. The project maintainers suffer from the ‘curse of knowledge’ - they are so familiar with the material that it is not clear to them that the manual and documentation, as good as it is, is absolutely not organized in anything approaching ideal teaching/learning structure. I assume the authors are students with limited experience (i.e. a few years at most) teaching this type of material, and the organization of the documentation reflects that. This is not a slam, it’s just impression of a user with many years of teaching experience.

Lastly, from my perspective, the single most useful thing that the community could do that would allow people like me to onboard into Emacs and org-roam is to create some Doom config files that were plug-and-play and would just work (on Windows). Forget about individual configuration–that can come later. Just have some bare-bones config files that someone like me can download, run Doom sync, and be ready to actually use emacs to take notes with org-roam, org-ref, and org-roam-bibtex. If this existed, I am sure that I would be enthusiastically staying and learning how to config the system. But I have spent, I don’t know, maybe 30-50 hours trying to get basic Emacs functionality working, and I have failed. Many people have kindly donated 1-4 hours of time to help me, but all of them (very understandably) disappear, and apparently having expert help for that amount of time is not nearly sufficient to set up a working Doom Emacs / org-roam system on Win 10.

My sincere thanks to all that gave me individual help these past weeks!

[rodelrod]

I’m sorry that this didn’t work out for you @cobblepot. I think your points are valid.

I’m a programmer with many years (decades, oh shit!) of experience whose main platform is Linux, so I’m firmly in the camp that would have a hard time helping you with your Windows setup. If that offers any consolation, that’s how Linux users feel about the much larger world of proprietary software that either doesn’t care or is actively hostile to the Linux platform.

Org-roam is an extremely young package by Emacs terms (a few months) and it will take some time to stabilize the ecosystem to the point of being accessible to newbies.

I’d just like to add a word about the options you mention. I’m a recent convert to Doom Emacs and although I’m extremely happy about it as a sweet spot of flexibility vs batteries-included, it required many hours of fiddling and research to make it usable, mostly due to the lack of documentation. Spacemacs, however, was practically a plug-and-play experience(*).

Now, Spacemacs started breaking on me when I tried to customize it too much, so it’s not perfect.

In short, if you want the most stable, easiest out of the box experience of Emacs including org-mode and its classic packages, such as org-agenda and org-capture, Spacemacs is probably your best bet.

I would not claim this to be the case if you are using not only the “classic” org-mode packages, but you intend to use org-roam. Org-roam it’s just way too recent and still in flux. However, I think that beginners will have a much better time once the Spacemacs org-roam layer settles down and is fully integrated in Spacemacs stable branch.

(*) Granted, on Linux, and I’ve had the vim keybindings burned in my fingers when VAX/VMS was still a thing. I have no idea if the Spacemacs experience is any easier on Windows.

[mshevchuk]

Dear @cobblepot, thank you very much for your great feedback. Every single word you’ve written is true.

First of all, since you’ve already spent 6 weeks on Emacs, I’d bet you would return to it after this first initial frustration. So let me tell you shortly my Emacs story, which will hopefully complement yours and give you some encouragement.

It all began more than 10 years ago with me looking for a digital task planner and organiser. The GTD system was very popular back then and the articles about it flooded the Internets as it is happening now with the Zettelkasten method. This is how I came to Org-mode. I was on Windows, a fresh graduate with sciences background although far away from any programming except for a limited Pascal experience in the high school. Hey youngsters, anyone heard of Pascal?

Anyway, my acquaintance with Emacs and Org-mode made me eventually leave Windows for Linux, maybe within a year, because I realised from the very beginning that it was barely usable on Windows. That was a tough decision because my occupation required regular collaboration with others — across the Atlantic ocean and the Himalaya mountains, as well as the neighbouring office — in Microsoft Word and a piece of software called ChemDraw, both are not available for Linux. Don’t even try to explain that to your boss.

Those were tough times of dual boot and virtual machines. But during that period I also learned a lot about Linux and system maintenance. I gained a superpower of being able to help people, my co-workers, friends and relatives to quickly fix their Windows (!) related problems because I learned how computers and operating systems work in general, and how to look for and find the solutions to the problems. I installed Linux on every toaster I could reach for. I also grew comfortable with configuring and troubleshooting something called nuclear magnetic resonance spectrometer, a scientific instrument my peers tentatively use everyday but otherwise are really scared to breath on from a wrong direction. Dear times of youth maximalism.

And all this thanks to Emacs, so back to it. My Emacs (mainly Org-mode) configuration grew over time. There was no Spacemacs nor Doom back then, John Wiegley hasn’t even started to write his use-package, and people, guess what, used Customize to set up their environment. Hassle-free®. This would be my first suggestion to you if/when you return to Emacs: get a simple config file with just package.el initialised properly and then use Customize and package.el to configure your Emacs environment. Don’t listen to hippies who say these interfaces are bad. They say it but never explain why. These things are at least maintained by Emacs developers, well-bearded people, so they are guaranteed to work, perhaps slowly but surely.

Eventually I got tired of Linux because there are always these new shiny things, a new version of a package you use that you must install immediately at 2 a.m. just before going to bed. So I bought myself a Mac, which is a reasonable compromise between Windows and Linux, unless you are a strict orthodox Stallmanist of course. It gives you a lot of Unix power, the command line and Emacs included, and can run virtually any commercial software Windows can run. I encourage you to give it a try, although I myself am a bit tired of MacOS now and am looking towards a new Windows computer in a couple of years. So maybe I’ll be in your camp then. The problem is, of course, they all make their best to tie you to their proprietary systems, but this is not something one cannot overcome.

I also get tired of Emacs from time to time. I mean, I still continue to use it but during those periods I almost completely lose interest in developing my system further. Which is by the way the central point of using Emacs, at least for me. It all started as a todo list tracker and organiser, the features I do not use extensively anymore. Instead, my system transformed into a writing and time-tracking environment, and recently, with the advent of Org-roam, into a knowledge repository. This was also the first time I was knowledgeable enough to make my small contribution back to the community in the form of org-roam-bibtex.

In place of conclusions. Emacs is a very special application, unlike no other. People do not typically use it to post pictures to Instagram in one click, although that is probably doable. Instead, Emacs provides a reasonably friendly way to learn the basics of real programming and dive into the world of technically advanced computer usage. You see immediately the results of your program — that function you wrote to shortcut a few things is your first program — and you can use your program to make “real world” things - your notes, tasks, files, memories, whatever. If your profession does not require special computer knowledge, but you yourself are a computer enthusiast, then Emacs will become a really rewarding and simultaneously useful hobby for you.

[zaeph]

Hi there.

First and foremost, thank you for taking the time to write out your thoughts on your experiences with the Emacs ecosystem. I’m going to address those points in-line, and I’ll try as much as possible to clarify our intentions with Org-roam regarding them.

@cobblepot

Nobiot has been the singular most helpful person so far in getting me going, but his point shows that Emacs isn’t good for someone that just wants a notetaking solution. [emphasis added]

I agree that @nobiot’s documentation effort has been stellar, and I’ll use the opportunity to thank him for this. As for your second assertion, I would agree with it, but I’d also consider it as a problem to resolve. More on this later.

@cobblepot

What’s become clear to me is that the org-roam community (and I think the Emacs community in general) does not care that much about whether their tools gain mass acceptance (which is fine).

I agree that this is a sentiment shared by the Emacs community at large, and most notably by its giants, i.e., those with the technical expertise to maintain and expand the ecosystem reliably.

However, I’d argue that this sentiment is the result of many years spent trying to convince people to try out Emacs, often with varying approaches, and coming to the conclusion that the efforts expounded in that direction just weren’t worth it. It’s not that we’re averse to writing documentation, since our guidelines for writing Emacs software are quite stringent in that regard, but I don’t think we’re doing as good a job as we could.

We expect our users to demonstrate the same level of rigour finding the information as we did writing it, thereby underpinning the RTFM attitudes which, in Emacs, we advocate by pointing users to the built-in docstrings (C-h v/f/a) and info-pages (C-h i).

The fundamental problem with this model is that, even though all the information is there when you know where to find it (= basic help-commands + ease of discoverability via appropriate naming), it is supremely overwhelming for those starting out with Emacs.

There are so many paradigms to familiarise oneself with, often without having the slightest clue where to start. Some of us strive in those environments, but I doubt any of us would say that getting out of the woods didn’t require a significant amount of time and efforts, both of which are valuable resources which not all of us can expand, regardless of our initial motivations to improve our workflows or change our ways.

@cobblepot

From my perspective, what they/you want is to have tools that work for you in the way you want, and you like solving problems and sometimes working together on them, and that many of you like helping others in general, and would be perfectly happy if newbies got into emacs, but these are secondary benefits and not the real goal.

It might be that I’m too green as an Emacs developer, but I still believe the introduction of new users to Emacs to be a hill worth dying on, not out of practical concerns for the survivability of the project, but for popularising and legitimising the use of Free/Libre open-source software (FLOSS). Your remark seems to imply that we’re in it for ourselves, out of utilitarianism, and whilst it might be true for some, I can guarantee that a lot of our giants are keenly aware of this political dimension.

@cobblepot

[…] generally the people who want to write for beginners don’t have the knowledge to create the necessary manuals, and the people with the knowledge don’t want to write for beginners or spend the time working with those who do. [emphasis added]

Developing FLOSS means leveraging what people want to do with how much time they can sink into it. It’s a common trend in our fields to direct new contributors to writing documentation because ‘anyone can do it’, but I believe this to be fundamentally wrong. Finding a good technical-writer is hard enough as it is, but finding one with enough time to do a thorough job is even harder.

A lot of us who develop FLOSS also develop software for a living, and this is especially true for the giants of our communities. Early on, when we’re just getting started, it’s easy for us to be a one-man-army and address all issues as they crop up. However, as our careers progress, often as a result of the rigour we demonstrate within our personal projects, it becomes increasingly hard for us to justify the time investment.

More specifically, our jobs put us in contact with actual technical-writers who make us realise how much work it actually takes to write good documentation. This problem directly feeds our reluctance to provide direct support to people because, as developers, we’d rather solve the source of a problem rather than its effects.

@cobblepot

No one I have met so far in the community with the technical ability to get these packages working is actually interested in spending the time writing up truly basic, general documentation that covers many use-cases because they are more interested in the technical challenges of making the software work for them.

I don’t think this is true and, once again, it seems to to stem from a strictly-utilitarian, self-centred view of FLOSS development which I do not share.

@cobblepot

The project maintainers suffer from the ‘curse of knowledge’ - they are so familiar with the material that it is not clear to them that the manual and documentation, as good as it is, is absolutely not organized in anything approaching ideal teaching/learning structure. [emphasis added]

We are keenly aware of the problems with the documentation, and we’ve stated it both here and on the GitHub tracker. The effort is underway, but its advancement is subject to the requirements of our professional and personal lives. Originally, I’d intended to start the work on the documentation right after the release of v1.2.0 in early June, but I was delayed by the requirements of my job, as well as by the desire to find a solution that would allow me to secure more time to invest into Org-roam. I intend to get the project moving ASAP.

@cobblepot

I assume the authors are students with limited experience (i.e. a few years at most) teaching this type of material, and the organization of the documentation reflects that.

Just as with technical-writing, being a developer does not presupposes an ability to teach well. However, I’d like to stress the fact the project is only 7 months old, and for half of which @jethro was alone in maintaining the project. Now that we are a team, it’s easier to maintain the project and get it moving, but the documentation remains one project amongst others.

For instance, before we could get into the content of the documentation, we had to improve the infrastructure to minimise our maintenance efforts, so as to optimise our time. During the adjustment, a lot of the previous manual had to be retired, which might account for some of organisational problems you’ve highlighted.

However, the chiefest hold-up on the documentation has been the development of the core features of Org-roam. As simple as the tenets of the slip-box method might be, mapping out the realm of possibilities within Org-mode is something that requires us to constantly reconsider our implementations, a key example being the inclusion of headline-links. When we discuss priorities with @jethro, as salient as our documentation effort might be to us, we simply cannot justify investing time into something that will be made obsolete within the fortnight.

We’re incredibly happy to see the success that Org-roam has accumulated over the months, and whilst we’re not feeling pressured in any way to improve the software, we still feel a sense of responsibility towards our users, both the old and the new. It might feel at times that we’re expounding more efforts towards the old, but I’d like to reiterate that we’re not doing it at the expense of the new.

Over the last 10 years, within the Emacs community, I think it’s fair to say that Org-mode has reeled in the most souls. Org-mode is a fantastic toolbox that gives us plenty to work with to organise our workflows.

Org-roam is different. It’s not just another tool for the toolbox as org-backlink may be. The modularity of our code makes it able to fit that rôle, but it’s not something that @jethro and I anticipated. Instead, Org-roam was meant to be a way to implement the slip-box method in Emacs with the Org-mode toolbox, which makes it inherently opinionated. Where Org-mode provides freedom, we impose restrictions, of which there are two types:

1. The restrictions linked to the slip-box method.
2. The restrictions linked to the technology.

As opinionated as Org-roam might be, we still strive to provide as much freedom as possible with regards to type-1 restrictions. For instance, if you want to organise your notes in groups, you can put them in a subdirectory and have Org-roam consider the name of the directory as a tag. Also, if you prefer to have non-atomic notes, we’ve created headline-links to make it easier for you to reference a precise point in a note.

As fruitful as a discussion on type-1 restrictions is, the reality of software-development means that we have to keep type-2 restrictions in mind, both in terms of feasibility and performance. Whilst every user of Org-roam can reflect on type-1 restrictions, the same cannot be done for type-2 restrictions. Being an expert user of Org-mode might be enough to make you gauge the feasibility of a feature, but you’ll need an expert developer of Org-mode to gauge the impact on performance and to implement it. I’d argue that there are even fewer of the latter than there are technical-writers with time on their hands.

@cobblepot

[…] it means that at least at this point of development this community should discourage anyone who doesn’t want to learn programming from using org-roam.

I think Org-roam has a rôle to play here.

I won’t surprise anyone by saying that the slip-box method has the wind in its sails at the moment, and I doubt this is going to change any time soon. For us, this means that plenty of people are going to stumble upon Org-roam, which will lead them to Emacs and Org-mode. Since the slip-box method is a way to organise your knowledge, to learn by writing, I think we have a golden opportunity to have people appropriate the method by applying it to the very tool they’re discovering.

…I’d like you imagine a different world.

---

You hear about the slip-box method, and upon investigating it, you discover Org-roam. You find out about what it can do through videos, and it tells you a little about Emacs. You decide to give it a shot.

The Org-roam webpage has a nice quick-start guide to get you started on your journey. It mentions that you should have an up-to-date installation of Emacs, and it points you to a neophyte-friendly guide to installing it, whatever your OS may be. This second guide recommends that you use a vanilla version of Emacs (as opposed to Doom-Emacs or Spacemacs) to familiarise yourself with the default tool-set, and it warns you of the risks of not doing so. Setting up Emacs was a breeze thanks to its setup wizard.

Now that Emacs is installed, you return to the first guide on Org-roam which explains to your how to install it via package.el and MELPA. It only tells you as much as you need to know whilst covering the edge cases, and it points you to the relevant sections in the Emacs manual for details. Once you’ve installed the package, the last thing that the quick-start guide mentions is that you should try typing M-x org-roam-tutorial RET in Emacs.

After a series of question to confirm your level of proficiency with the slip-box method, Emacs, and programming in general, you are dropped into a tutorial that teaches you how the implementation works. Rather than just being an info-page, it guides to through the creation of your first files, your first links, and it invites you to take notes on what you are learning. A final quiz allows you to confirm that you’ve learned the fundamentals of the three pillars, and it directs you to the relevant sections if you’ve missed some of them.

At the end of the tutorial, you are directed to the built-in info-pages for Org-roam, so that you may explore them to your heart’s content. You’re also directed to Discourse if you want to talk with other users.

Even though you headed into this project thinking that you only wanted to use a new piece of software that looked interesting, you get out of the experience with the feeling that you’ve learnt more than you intended. You look at the notes you’ve taken on programming, and you realise that it isn’t as intimidating as you thought it would be at first. Even if you find Emacs a little bit archaic, the tutorial has helped you realise that it is a façade for something incredibly powerful. It makes you wonder how many books you’ve misjudged by their covers, and how much software you’ve dismissed because it was free.

You know that you’re only at the beginning of the journey, but rather than feeling overwhelmed, you feel a sense of adventure. You’ve already written a few notes, but you realise that this could be the beginning of something much greater. You’ve already started the journey, and equipped as you are with your tools and your newly-found knowledge, you get ready for the next step.

---

I’d like to usher in that new world, and I’ll be striving towards that goal in the coming months.

Thank you again for your contribution, @cobblepot.

[bruce]

@zaeph

Developing FLOSS means leveraging what people want to do with how much time they can sink into it.

I just want to pull this out as a key point to consider. I don’t even have time for the things I intensely want to do in the open software world, much less everything else I think might be valuable for other people to work on.

[nobiot]

@cobblepot, you owe me this (half serious):

Zero to Emacs and Org-roam: a step-by-step guide on Windows 10

OK, thanks, I’ll try it.

I have done a little “script” to automate steps I describe in my Zero to Org-roam guide.

If you have 10 minutes when you come back to Emacs, try it. If I’m still around when you do, I will be happy to help you get over the line.

I can’t support Doom (it’s massive and way beyond me), but vanilla is much more tamable if you only do the configuration in the guide.

The auto-install script is in my guide repo, and got only two files:

• .emacs.d/init.el
• install-org-roam.el

I have tried it a couple of times; it works. Maybe it will work on macOS and Linux (I don’t know yet).

It will install the packages I mention in the guide up to the chapter on ORB.

• org-roam,
• modus-operandi-theme & modus-vivendi-theme
• ivy, counsel, & swiper
• olivetti
• org-ref
• org-roam-bibtex

It will check if Emacs recognises SQLite3 installation

Instruction can be added later.

Brief instruction (will do a bit more later)

1. Make sure you have no .emacs file or other configuration files in your home folder, and take a backup of your .emacs.d
2. Get the two files from the repo
3. Place install-org-roam.el to your home folder, and replace .emacs.d – make sure your .emacs.d contains only the init.el from the repo
4. In Powershell, from your home folder, call this:
   emacs -l .\install-org-roam.el --batch

Your Emacs installation needs to be in your PATH environmental variable (for PowerShell to recognise emacs command; if not you just need to specify the full path), but I’m pretty sure you are fine with this if you can deal with doom sync.

[nobiot]

@cobblepot

Getting vanilla Emacs set up and workable with these packages is not feasible for someone who does not want to really learn programming as an end in itself

I am sorry to hear of your frustration and struggle.

For this point about vanilla, though, we have a counter example: me.

I am not a software developer or professional programmer. I have used Emacs only for note-taking and (academic) writing when I was studying on postgraduate courses as a (very) mature student. I started Elisp programming on Emacs only after I started with Org-roam (only a few months ago). I have been in the software industry for long time, so I guess I am rather familiar with some peculiarities of software development cycle and support. I can program a little, only as an extension to my normal use of PC – Outlook, PowerPoint, and Excel. I don’t use VB or macro on them. I use Atom, Visual Studio Code, or Jupyter Notebook for some rudimentary coding on the side as a hobby, or to complement my daily work.

I don’t think you and I are very different in our motivation to use Emacs.

The reason I think I was happy to come back to Emacs (I had left it for years until I discovered Org-roam) is, and the difference between you and me is, that I did not start with Emacs with Doom (or Spacemacs for that matter; I left it after a few days of use). From my experience, Doom requires intermediate to advanced knowledge of Emacs.

I am not trying to persuade you to come back or anything. But if you do, try vanilla on a clean slate. Like I said, if you find me still around, I’ll be happy to help you get over the line.

[mshevchuk]

There have been developed a number of more or less sophisticated frameworks over the years. They all come and go. Only Emacs remains.

[cobblepot]

Thanks for the earnest responses. I’ll share a few thoughts before I respond to individual comments.

First: I’m an academic, so my writing and notetaking is a central professional activity for me. I came to org-roam via the zettelkasten concept, which I pursued only because my existing notetaking/writing system was not working as well as it could be. That system involved 4 applications, and a main draw of emacs was the possibility that I could do nearly everything in one, well-integrated system. (Little did I know how poorly emacs packages are actually integrated until the user makes them so.)

However, the centrality of writing for my job also means that I can’ develop a different system slowly over time as a side project. If I’m taking notes, I need to access them in my main active writing system. So a move to emacs for me requires a major shift into a new system, and I must dive in rather than dipping in a toe when I have the free time. I am/was willing to take a temporary productivity hit during that shift, but I can’t stop all productive writing until I get things working. So my inability to get the basic functionality working essentially meant that I could not work in emacs. All my time was spent configuring. I’m willing to spend 50% of my time configuring for a while, but not 100%.

Second: when emacs is not working, for me at least, it was totally broken. If the config file has a missing parenthesis, emacs won’t work. That’s a problem for new users. It’s not like it was basically working but I couldn’t get some minor workflow optimization working. I effectively can’t write when it’s not working. That’s why I suggested offering new users a basic working setup.

@nobiot, I appreciate your script. I did try vanilla emacs following your guide and got it installed and working. But vanilla emacs, unlike Doom emacs, doesn’t work out-of-the-box at the minimum functionality level I need/want. I use Dvorak rather than Qwerty, so C-x and other emacs keybindings are awful for me. So I’ll need to figure out emacs keybindings. I need to read pdfs and annotate them. Org-noter to the rescue! But I have to figure that out, and that’s not part of your guide (yet). That being said, I have now used your script to install vanilla emacs and go through the tutorial (which is surprisingly well-written for helping emacs beginners, and even more surprisingly seems to be almost the only thing well-written for helping emacs beginners that are not aiming to learn Lisp or programming, with the exception of some very limited tutorials).

Although @nobiot’s and my use cases are similar in some sense, the key difference is that my writing system is my “daily driver”, not something I can deal with in my free time as a rewarding hobby.

I’ll admit that @nobiot’s comment to me about his custom Doom modules did prompt my post. At first I thought, ok, I’ll try vanilla. And then I saw what that meant. And re: Doom custom modules, when @nobiot told me to check the Doom documentation about custom modules, I thought, “if even the indefatigable, helpful @nobiot is telling me to go read the manual, then I’ve exhausted everyone’s patience too much - this is unsustainable.” In any case, I have a few more comments about vanilla emacs at the bottom of this.

Now I’ll address @zaeph’s points. Like me, @zaeph is/was a teacher (I think) and I love @zaeph’s enthusiasm about bring emacs to the masses. It’s great that you recognize the off-putting nature of RTFM. The truth is that it is frustrating to have people ask you to explain information that already available. But what most Emacs helpers fail to understand is that RTFM is only reasonable when the manual is written in a way that someone could reasonably learn from it. But, currently, it’s not. It’s a reference book. As I posted elsewhere, asking someone to learn Emacs using the manual and internal documentation is like giving an English speaker a French newspaper and an English to French translation dictionary and expecting them to teach themselves French. All the information is there, right? Just read it!

No. The information must be presented in the right order, in a way that allows users to link things they don’t know with things they already know, so they can learn what concepts mean and build on this new knowledge. It is REALLY hard to do well.

@zaeph wrote:

Your remark seems to imply that we’re in it for ourselves, out of utilitarianism, and whilst it might be true for some, I can guarantee that a lot of our giants are keenly aware of this political dimension.

My comment has nothing to do with the notion of individual interests versus social interests or utiltarianism vs idealism. It’s just psychology. It is a lot more fun for most developers to solve programming challenges that stretch their brains the way they want them to be stretched than to write documentation for people who don’t know what they already know. That is why there are a million jokes about the frustrations of people trying to provide technical support to their aging parents over the phone. It’s frustrating! It’s hard! It’s annoying!

Most programmers would be supremely intellectually challenged in trying to write documentation that general users could understand, but that is not the type of intellectual challenge they signed up for (which is why they are not teachers, but programmers). I would venture to say that even people like @zaeph, who are really politically committed to open source software and who spend lots of time trying to make it more accessible, on any given day find it a lot more fun figuring out how to make the program work better than struggling to figure out what is confusing someone else that seems so obvious to them. The developer of Zettlr is another example. He’s really committed to open source software, has extensive documentation about his software, and spend a huge amount of his free time trying make the software better. But when he has to make a choice, the documentation falls behind.

I am quite aware that org-roam is very young, which is why in my original post I did explicitly note that things were not ready for people like me at this time. But I do think that the disconnect between Emacs projects’ inaccessible documentation goes deeper than the current schedule on this project. For as long as Emacs has been around, and for all the great tutorials and videos that exist, there is still nothing that gets users from zero to actual, functional, usable document creation in Emacs (@noboit’s guide being a welcome partial exception, and I have seen the “Emacs for writers” presentation and others). I have a significant list of links to tutorials and “Emacs for beginners” webpages, and all of them stop very early in the usability process. There is just enough there to get you to a place where you can start to learn how to configure Emacs yourself if you want to learn lisp. But there is nothing, as far as I know, to get you to a place where you can actually perform job duties or complete other projects as a first-order goal. Maybe that is the key point and one that is unusual to me: I’m trying to use emacs to better achieve a central professional activity for me, when it is really designed to do that only for people who want to learn programming, either as a central professional activity for them, or as a hobby.

@nobiot, I have now tried vanilla emacs using your script (BTW, your post’s step 4 didn’t work for me - I had to amend it to emacs -l .\install-org-roam.el --batch). I may be willing to take a productivity hit and try it as my writing daily driver if someone can help me get it working for me with what seem to me to be minimum functionality tools (in addition to the tools already in @nobiot’s script):

• a tool that lets me easily create and manage Dvorak-friendly keybindings
• org-noter or other tool to create and extract pdf highlights that can be easily put into org-roam notes
• pandoc with working .docx export (I know this is in your guide) and proper citation (which your guide is not clear about - seems like you have it partially)
• window-select, ace-window, or some other one-character window selector
• emacs which-key-mode

…and the following which are perhaps less essential but seem very useful to me:

• dired icons
• “normal for Win users” undo
• hydra
• helpful
• md-roam (@nobiot why is this not part of your guide?!)

Of course, no one owes me this. But I would probably finish my documentation if I actually was able to get everything working.

The thing is, I can currently do almost everything I need in those 4 separate programs I mentioned earlier, except keep a ZK, and I can add a 5th program to do that (maybe Obsidian). Emacs is worth the switch, maybe, if it can do everything almost those other programs can do, and can do some things better, but it’s not worth learning a new system and set of keybindings (a MAJOR mental investment) if it is just an addition to the 4 other programs I use.

My best to all!

[mshevchuk]

@cobblepot, Emacs was designed in the times when computer access was a privilege rather than a self-obvious entertainment resource for masses. It is older than most of the members of this small community here. In fact it is as old as the parents of some of us. Back then, accountants, managers and fiction writers let alone teachers, engineers and scientists had to learn programming—punched cards included—to use computers to help them achieve their first-order professional goals. Emacs still bears that legacy, simply because it is Emacs. If it didn’t, it wouldn’t be the Emacs then—“at its core an interpreter for Emacs Lisp, a dialect of the [Turing-complete] Lisp programming language with extensions to support text editing.”

To support text editing. Emacs can support many more activities, in fact anything, since it can do anything a computer can do. In loose terms, it is a software computer, not a computer application like Microsoft Word that addresses a single task and tries to do it well. Having just a computer you can’t edit text. You must install an application for that. To build a graph, you need another application, say Microsoft Excel. The common denominator here is Microsoft, which brings these two applications to you and makes them play well together, so you can integrate your graph into your text document. In exchange for a share of your income to sustain itself.

Emacs is geared towards programming and programmers. This is not a secret, it is written everywhere the name Emacs is mentioned. Moreover, it is an open source tool, which means it often lacks professional beginner-level support for the reasons you’ve extensively outlined yourself. Using Emacs (and any other computer programme) is a deal: I will get this and this and this, if I learn that and that and that. Even becoming proficient with Microsoft Word requires a rather big investment into learning. It may seem trivial to us because a large portion of the today’s world population have grown up with computers on their desks and in their pockets, absorbing portions of computer knowledge with breast milk. But what would happen if Victor Hugo were to face a computer? Would he pursue to learn Microsoft Word or decide he’d be more productive with an ink pen to finish his “Notre-Dame de Paris”? I don’t know, but I’m sure he’d definitely be disappointed with the lack of accessible beginner-level tutorials for Microsoft Word.

Emacs can potentially make you more productive if it fits your way of thinking and if your way of thinking fits the Emacs paradigm. Otherwise it can become a time sink and a huge disappointment. Again, Victor Hugo didn’t need Emacs nor Microsoft Word to achieve his accomplishments. Neither did Niklas Luhmann known for his lifelong elaboration of the Zettelkasten method, our contemporary, who did not use computers at all.

Computers are good at processing large amounts of data in short times. They’ve made such tasks as acquiring and fitting the data from the Large Hadron Collider experiments or coordinating hundreds of millions of Amazon deliveries per day possible. They are helpful in instant communication and are also good at storing large amounts of data in a small physical volume, so that the Library of Congress can fit into your suitcase. But they offer little if any advantage for “single mind” tasks such as creating notes, compared to analogue tools. Thinking the opposite is an illusion and self-deception. Using a personal computer, you inevitably run into many random problems starting from a recent Windows update failure through configuring Emacs up to a cryptolocker ransomware infection, which require you to spend varying amounts of time fixing them rather than on your writing. And even if you happen to manage your work done slightly faster by being more productive thanks to some automation and lesser number of tedious and boring mechanical steps needed, you’ll probably spend your freed time reading news or hacking Emacs than doing more work. So the decision whether to use a computer-based system for writing should largely be governed by personal preferences. The decision regarding what tools to use is even more personal, just use what is good for you, they are all the same from this point of view.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1311997/?page=1

[amatsukawa]

Have you tried zettlr or obsidian? It seems to be more what you are looking for.

My goal is not to send you away, and I certainly hope you can stay and try some of the responses, but it sounds like you are looking for alternatives anyway.

But as a new emacs user, I definitely feel your pain. I am a programmer, I even know a different variant of lisp, I’m on OS X, and it still took me tens of hours to get a working setup.

[cobblepot]

Thanks for the suggestions. I have tried both and am currently trying Obsidian for the linked notes part of my workflow. But it is not quite right to say that it’s what I’m looking for, because what I was really looking for is an integrated system, and Obsidian potentially fills one part of the system that was falling through the cracks before, but not the other parts - outlining, pdf annotations, reference management, and outputting articles with proper citation. Emacs potentially offered this all together. I’ll check back in a few months to see what’s happening.

Incidentally, a belated hat tip to @nobiot about one thing - I chose Doom Emacs due to the availability of its developer but as a total newbie to both Emacs and Vim, I didn’t realize how much more complicated it was to jump into both Emacs and modal editing together. I now agree that it would have been easier from the start to do Emacs without Vim/evil keybindings. Org-roam should keep this in mind if/when aiming to appeal to people with no programming experience - if they have no modal editing experience, steer them to vanilla Emacs or at least away from evil keybindings.

[mshevchuk]

An excerpt from Stallman’s 2002 speech.

It was Bernie Greenberg, who discovered that it was (2). He wrote a version of Emacs in Multics MacLisp, and he wrote his commands in MacLisp in a straightforward fashion. The editor itself was written entirely in Lisp. Multics Emacs proved to be a great success — programming new editing commands was so convenient that even the secretaries in his office started learning how to use it. They used a manual someone had written which showed how to extend Emacs, but didn’t say it was a programming. So the secretaries, who believed they couldn’t do programming, weren’t scared off. They read the manual, discovered they could do useful things and they learned to program.

[zzz]

Total newb Emacs/Roam and Windows user here. Just signed up for Discourse to add this 100% rambling comment.

I’m a junior dev, very wet behind the years, who uses VS Code on a mac full time (for about two years now). And have taken full running starts at Emacs via Vanilla, Spacemacs and now Doom a few times now over the past year or two only to find that setting up a really cozy coding environment similar to what VS gives you out of the box is REALLY difficult to do. And if you don’t have a good reason for using something, that something becomes VERY challenging to learn. I never learned Emacs.

Fast forward to today. I recently switched my daily driver non-work Thinkpad from Linux back to Windows to have a little more out of the box stability, hi-res display support and just overall features, but most of all, to test out WSL2. About a month ago, a friend told me about Roam Research, and I took a deep dive on the Zettel method and along the way discovered there was a free Emacs alternative called org-roam. I decided to give it a shot… My life is a hodgepodge of OneNotes, physical and virtual stickies, Apple notes and Firefox notes. Pure pain, and I’m never really able to find what I need.

As it’s been said above, taking a package like org-roam for a quick test drive in Windows, as a non-Emacs user is basically impossible. You’re fighting an uphill battle of a vast amount of keybindings, people talking about long-term pinky finger damage, people talking about how much the defaults suck and how you should use vim, poor documentation of adding packages on a Windows system and a lot of hacks using git-bash, scoop, etc. Where the hell does Vanilla emacs store the config file, what is this lisp language, what is a major mode, etc.

I couldn’t figure out how to add org-roam to the native Windows Emacs package, but I found a tutorial on setting up Doom Emacs with WSL2 on Windows and how it could be run as a GUI app using a Windows X server freeware package. After a little bit of trial and error, I got a sexy Doom Emacs opening up on my Windows machine. My Doom config files are located where they should be- in my Ubuntu home folder. I can run Emacs in the terminal, or I can run it quasi-natively as a gui though it’s a little bit of a pain in the ass to make sure my x-server is running, open the terminal and type my emacs command just to take a note. BUT IT WORKS!

Why Doom. I know just enough Vim to get by. I don’t really feel like learning the Emacs document-altering bindings yet. Doom looks really badass out of the box and has exactly what i want as the default items on the first page. I can press spacebar and slowly learn the commands… not just the commands but what these commands are calling. It’s easier to use than Spacemacs, though a little less documented.

Still have no idea what I’m doing, but I was able to add the org-roam package to Emacs using +roam and magically, it seems like a lot of the stuff actually works!

All of a sudden, I’m writing notes, inserting roam-links and I can click those links and it actually opens up the other file!

And that weird backlink sidebar (buffer?) that never seems to disappear actually shows the backlinks! I have no idea how to create a daily template and am just copying (yanking?) and pasting from one document every day, but it’s okay! It’s on my TODO list to learn how to make a template and make it do the thing I want it to.

I even discovered the package that runs the roam-server graph thing! I can actually open up Edge or Firefox on WINDOWS and see that badass chart and hover over things and see my org documents display. It’s awesome! Now, of course it would be awesome to be able to CLICK those things in Windows and have them open in Emacs, but that’s a TODO for another day.

But I’m in Emacs!!! It’s an absurdly preconfigured Emacs, but that’s okay, because it works and I am doing things in it! Doom is holding my hand as I’m now learning Vim (haha), while I discover the beauty and features people have known for years of org-mode, and at the same time discovering how truly magical the TODO and agenda system is, and org-roam is just icing on the cake- it’s like a sexy glue between all of these note files I’m creating. I still need to understand how Tags work in org-roam… no idea if they’re different than regular org inserts, and hell if i can find anything about them in the fledgling documentation, but that’s okay, because this is clearly a very new package, and they don’t have great docs yet. I even threw Doom on an Emacs-Plus install on my Mac (of course it works much better there).

From my total newb perspective, I think the problem is that code editors and applications in general have come such a long way that the average person expects quite a bit of functionality out of the box. Most people don’t want to have to download Notepad and add in all sorts of functionality to get it to work like VS Code when they can just download VS Code and add the extensions they need (using a very simple config JSON). It seems like Emacs can have incredibly powerful sane defaults, but the maintainers of Emacs choose to provide the ‘Notepad’ version of the app to people (sure, this is good for people with years and years of config they’ve constructed) rather than additionally providing a ‘hey here’s a really sexy and usable example of Emacs for newbs to quickly start working with’ version.

From my perspective, I just want more actual-use examples… preferably from a place where you have very little custom config.

Things I’d like in Org-Roam docs as a beginner:

1. Here’s how I take notes on a book
2. Here’s how I take daily notes while I work
3. Here’s my daily template and how I get it to preload when I run org-roam-today (or whatever it is )
4. Here’s how I utilize tagging.
5. Here’s how I utilize TODOs
6. Shortcuts

Just want to say a quick thanks to everyone who contributes to this project!

[mshevchuk]

@zzz

I’m a junior dev

@zzz

what is this lisp language

You should have attended your CS lectures instead of, I even don’t know, what.

[zzz]

@mshevchuk

You should have attended your CS lectures instead of, I even don’t know, what.

Imagine thinking that every junior dev has a CS degree in 2020 and then imagine poking at someone’s self-admitted lack of knowledge in a discussion thread involving absolute beginners and Emacs/Org-Roam … There actually IS a problem in here, but let’s just say it’s not the people trying to learn/fill in their knowledge gaps.

[mshevchuk]

@zzz

There actually IS a problem in here

A huge problem here in 2020, I would say.

Emacs is Lisp, but Lisp is not Emacs — as in subset of a set. Lisp is something as basic and fundamental to programming, as say, the relativistic theory to physics. One can be a junior physicist and work in the field of electromagnetism, for example, and do not know all the details and methods of the relativistic theory, of course. But if such a junior physicist said “what is this relativistic theory”, that would have had negative consequences for their reputation. It doesn’t matter whether they graduated from the physical department or linguistics. As soon as they start working in physics, they should know the basics. Would you also like some analogies with doctors? Probably not, you’d like your doctor be a specialist.

The problem is not the lack of knowledge, or being a beginner, or asking for advice — the problem is that people recklessly admit their lack of knowledge. They are proud of it. It’s like something self-obvious, no one knows everything, so why should I, right?

[jethro]

@mshevchuk

You should have attended your CS lectures instead of, I even don’t know, what.

I just completed my CS education and there was no mention of lisp whatsoever. This is not uncommon these days.

Sensationalizing lisp doesn’t help here. It’s certainly esoteric and takes some getting used to.

@mshevchuk

the problem is that people recklessly admit their lack of knowledge.

Isn’t this the first step to getting better? OP admits a lack of knowledge here, and is working to fix it. What’s wrong with that?

[mshevchuk]

@jethro

I just completed my CS education and there was no mention of lisp whatsoever.

This is another contemporary problem of many universities and colleges teaching only the most recent developments in the field, be it computer science, physics, chemistry, anything. They do not give the background. They give fish but do not give the fishing rod and do not teach how to use it.

That’s why the young generation grows with this:

@zzz

From my perspective, I just want more actual-use examples… preferably from a place where you have very little custom config.

Actual-use examples so that I can copy-paste them and do not bother.

@zzz

It seems like Emacs can have incredibly powerful sane defaults, but the maintainers of Emacs choose to provide the ‘Notepad’ version of the app to people

The maintainers of Emacs give you the fishing rod and explain in their extensive manuals how to use it.

Frameworks like Doom, Spacemacs or Prelude use that fishing rod to build something they think will help other people with fishing their Emacs config.

@zzz, please excuse me if I insulted you, I’m a nasty person when it comes to knowledge. I believe, knowledge is something on should struggle to get. Knowledge requires one to put efforts to gain it, to work hard on it, in contrast to plain facts that can be copy-pasted. Knowledge is thinking, it is a creative work, while copy-pasting is a robotic work. All this is dangerous because it leads people to fall into a cargo cult — when they are doing something but do not understand what they are doing. And they eventually fail in their doing if not provided with exact instructions — just like robots.

@jethro

OP admits a lack of knowledge here

Admits in a careless way showing they do not care what Lisp is, as if it were some minor nuisance that prevents them from using Emacs. There is an inherent logical contradiction here because as I said Emacs is Lisp.

It’s not about Lisp per se, of course. It’s about the growing generation’s fragmentary knowledge, the desire to have everything ready in a digestible form that would require little effort to consume it.

Despite the commonplace sense of safety cultivated in the modern world, the world itself is a very fragile place. This premise is very well demonstrated by the current pandemic. One strong enough Sun flare and all your computers become rotten pieces of plastics and metal. The world counts on you guys being able to fix it, and hopes you won’t say “we didn’t learn how to do that in the university”. I won’t be able to fix your computers, but I’ll be able to provide you with basic medicine.

This is a joke and exaggeration of course, but I hope I made my point clearer.

[jethro]

@mshevchuk

It’s not about Lisp per se , of course. It’s about the growing generation’s fragmentary knowledge, the desire to have everything ready in a digestible form that would require little effort to consume it.

@zzz

From my perspective, I just want more actual-use examples… preferably from a place where you have very little custom config.

This is a common desire. Most people want things that just work, and would gladly pay for it (like 15/month for Roam Research). That’s fine, they’d much rather spend their time doing the actual work. There is nothing wrong with copy-pasting. Maybe it’s not the Emacs way, but the Emacs way is unique and hard to appreciate. That doesn’t make it better than everything else.

The barrier of entry for realising Org-roam’s full potential is high. One must learn how to wield Emacs, tame Org-mode, and then become adept at taking good notes. All of these tasks require a significant time investment. So maybe it’s moot to come here and complain that Org-roam isn’t friendly for beginners because it requires all these prerequisites as none of these are solvable by Org-roam, but the point still stands that we didn’t issue any warning on the perils that lie ahead in this journey.

The only actionable I can takeaway from this long thread is that we need a section in the manual somewhere that let helps people figure out whether Org-roam is for them before they invest time in the ecosystem. That’s the least we can do.

@zzz

Things I’d like in Org-Roam docs as a beginner:

1. Here’s how I take notes on a book
2. Here’s how I take daily notes while I work
3. Here’s my daily template and how I get it to preload when I run org-roam-today (or whatever it is )
4. Here’s how I utilize tagging.
5. Here’s how I utilize TODOs
6. Shortcuts

These are all extremely personal, and should be figured out on your own. How I use Org-roam is different from how other people do (you can look at all the Roam tours on youtube, everyone does things differently).