@liam makes some good points:
a bunch of org-roam users without a history in the Org community flooding the Org mailing list calling for large refactors is only going to breed animosity
Absolutely, I think we should avoid that. However, I think a lot of what would be required for an effort like this is smaller-scale, incremental changes that are a definite win regardless of whether we ultimately adopt any of the bigger ideas. For instance, to do such a thing would require more separation of concerns, a more thorough test suite, etc.
However, I’m also looking to other projects that do have a formal RFC procedure and the good, larger-scale changes that come out of that – in contrast with the quite incremental approach taken so far. A formal proposal makes no sense coming from someone without a history of contribution. But it doesn’t mean we shouldn’t dream big in the long term!
I don’t think Elisp is as bad as it seems, either.
I can definitely get over the ergonomics, but Elisp seems to encourage a highly untestable, stateful approach to programming that makes modularity a lot harder. That’s my big complaint (also, just that it’s never going to be as fast as C for some tasks).
That said, there are some incredible Elisp programmers out there doing very impressive (and useful) things.
On a different note, I’ll echo @agibragimov here:
First of all, let’s emphasize the notion of “optional”. I’m talking about the optional indexing layer, which is never intended and never will be “the source of truth.”
I think this is the key thing that would keep these changes backwards compatible. One way to look at it is that there have been multiple independent inventions of a DB caching layer for org, which suggests that perhaps we want one semi-official one. (Or at least an interface with org core that unifies a lot of the common tasks.)
Perhaps the right first steps would involve:
- More prototyping (Jethro and the rest of the org-roam contributors are doing an excellent job of this; I have some orthogonal ideas)
- Refactoring work to make the org-roam / org boundary very nice, clean, and modular, and at the right abstraction level (including lots of testing work).
- Can we look at org-db, org-ql, and org-roam to see what they have in common? The old software adage is to not try to make something general until you’ve done it 3 times. If there are nice, shared abstractions, perhaps bringing something into org contrib would make sense (but definitely not at the cost of agility downstream)
So I guess where I come down is: yes, the skepticism is entirely warranted (and healthy/helpful; I appreciate pushback). But I think we can make some unambiguous, incremental improvements to org, perhaps with an eye on some longer-term, ambitious goals.
And a final (quite relevant) objection: talk is cheap, where’s the code? I’m guilty here (my org contributions have been minimal, though in my defense I’ve had patches get ignored a couple of times on the mailing list), and so my first steps in a project like this will look a lot like trying to get more involved in both these projects on a normal contributor basis, while simultaneously trying out crazy, orthogonal experiments. It’s going to be longer-term because I have a day job. So I guess you should consider this me volunteering to do chores for org-roam I have the same username on GitHub. Feel free to tag me in, but I don’t want to impose management on the maintainers so I’ll probably pop in on the issue tracker as soon as I get a grip on the org-roam rewrite.
And one final, pretty out-there thought: I get the impression that org-mode contributors are doing most of the work in their spare time (which is heroic and I thank them). But there are some important institutions that depend on org-mode. I can think of at least one multi-billion (USD) trading company, not to mention lots of scientists. So there might be some grant/contract funding in the world (from an organization that supports scientific infrastructure, like the Simons Foundation) to support someone working part/full-time on these things (pending lots of paperwork, networking, etc.).