Has anyone migrated away from zotero/mendeley to just an org-based solution (probably org-ref)?

How To Guides
1

Would you have any tips? These days I use, essentially, org-protocol (but actually a rewrite of grasp) to capture notes from the web. The pain points I’ve had in attempting this migration are the following:

  • grabbing from the web into bibtex – I believe can be done with org-ref’s drag-n-drop, but using something like org-protocol would be preferable to not have to have an open bibtex file all the times.
  • grabbing the associated PDFs from a website (org-ref seems to only grab the bibtex)
  • (bonus) adding extra notes to items like an archive.org link (which is done via zotero-memento to combat link-rot).
  • (bonus) doing captures on mobile

Any tips would be appreciated! Even just “don’t do that” with an explanation of why.

2 Likes
2

@stites,
I am no academic or student (anymore), but I use Zotero and .bib, plus I have just written up an article on how to use PDF-Tools, Org-noter, and Org-roam-bibtex together in Emacs on Windows (Org-ref is also part of the overall guide); Zotero is a central part of it, without explicitly mentioned as such (I just assumed it would be for many of the readers of the guide).

… so, I may be able to contribute a perspective.

I would not say “don’t do it”; I am assuming you might feel that it was better if you moved away from Zotero/Mendeley to an “org-based soltion” (?)

I am trying to understand your reasons why you might want that. If you are comparing org-protocol with Zotero/Mendelay, I feel that this comparison may be comparing an apple to an orange (not a valid comparison).

I see the whole “ecosystem” where Zotero (and its plug-ins), Org-based packages in Emacs (e.g. Org-ref, Org-noter, Org-roam, and Org-roam-bibtex) work nicely together, like this in the crude illustration I have just done below…

Image002
Image0021523×857 158 KB

You have at least 4 “functions” to think about:

  1. Capture
  2. Write notes
  3. Write up [something] – academic paper, blog, etc…
  4. Manage bibliographic database

To me, org-protocol is (only) one of the tools for capture. Zotero is mainly a bibliographic database, with capture functionality (works on mobile browser, too). I didn’t realise that Org-ref works as a capture tool, too (“grab the bibtex”); I use Zotero for this purpose. Zotero does work well for grabbing / capturing PDF documents for bibliographic database, so that would address one of your pain points (and on mobile browser, too; so maybe two pain points?).

What would be reasons for you to prefer “just an org-based solution” to using different tools such as Zotero / Mendeley?

5 Likes
3

I had previously used purely org-ref to capture everything into a single bibtex file, but I found it lacking compared to Zotero’s connector. The bibtex org-ref generates lacks information compared to when using Zotero.

Org-ref is has some ability to fetch PDFs of the files, and works for most of the stuff I refer to (arxiv, mainly).

If the capturing aspect of org-ref is improved I can see myself moving to a pure org solution.

1 Like
4

Org-ref is not a bibliorgaphy manager I would say. It just extends Org and BibTeX modes with some (a lot of actually) custom functionality and sort of connects the two.

I use Org mode to write documents that are then exported to LaTeX. I, however, don’t like too much literal TeX in my Org documents, so for me Org Ref provides a convenient way to have a non-LaTeX-style markup in Org. And it is also very convenient to open PDFs and URLs from the cite: style links, which I am used just to type manually, by the way.

Since Org Ref uses BibTeX as the bibliography management backend, you would inevitably end up editing BibTeX files by hand. Org ref provides some custom functions to streamline this as well as to fetch BibTeX from DOI and even PDF files but, similar to Jethro, I found they do not suite my needs and I couldn’t find a quick way to tweak their behaviour.

I personally use BibDesk (MacOS only) to manage bibliography. It also uses BibTeX as the database format, so integration with Emacs tools that use BibTeX is pretty straightforward - they are all just different interfaces to interact with the same .bib file.

Switching from Zotero to BibDesk was a conscious choice because I wanted to have a single database that I can work with using different tools. Exporting from Zotero to BibTeX, and most importantly, re-importing it was too much hassle for me back then.

Windows and Linux users seem to have only one full-featured option that uses BibTeX as the native database backend - JabRef. It’s rock solid and can do many things, basically everything one needs, but I can’t force myself to use any Java GUI.

There is also Ebib - a powerful native Emacs BibTeX manager. I’m currently exploring its functionality and when I find a way to replace its own note management with Org Roam, I will probably switch to it.

Rephrasing what Nobiot said, if you need something more complex than just retrieving the BibTeX metadata and the associated PDF files on a desktop computer, then you probably need to stick to Zotero.

My workflow — 1. copy a DOI, 2. paste it into BibDesk, 3. run “retrieve BibTeX from DOI”, 4. copy the BibTeX key, 5. download PDF, 6. rename PDF with the citekey, 7. update the database — may seem too tedious compared with the fully automatic Zotero style, where you have a magic button that captures the page, retrieves everything, files the PDF and exports to BibTeX. I, however, have a full control of each step and enough time to reflect on what I’m doing.

1 Like
5

Thanks all! I think this gives me a good lay of the land and points of consideration.

@nobiot: In the context of these functions, I would say that Zotero handles all the functionality you listed, but (I similar to yourself) I only use it for capture, reference management… and a miscellaneous curl request to archive.org, so it seems a bit heavy-handed.

It sounds like I’m on the same track of exploration as @mshevchuk right now in that I (mostly) just need a reference manager for a single (albeit large) file, and running more browsers seems to be an unsatisfying default for most solutions these days. Ultimately, it sounds like if I have personal, niche demands, I’ll have to write personal, niche software or deal with the current status quo.

3 Likes
6

Long ago I was using Pybliographer when working on a (biography) book using LyX/LaTeX…now I want to start taking notes using ZK method using org-roam (and then possibly write using LyX/LaTeX or ConTeXt) and most of my references, based on the material which I study. are going to be from the books I personally own and majority of them are also available online in wiki-like web site and/or some PDFs here and there…

I want to annotate my notes/articles/presentations with proper references and since I’m on (Debian) Linux I also do not want to use JabRef…

…and wonder how you are satisfied with ebib and whether it could be sufficient for my needs over Zotero?

7

I don’t use Ebib as my primary reference manager yet. I occasionally do some batch processing with it and it’s very good. I’m on MacOS and there is a native free and OSS app BibDesk, which was among the reasons I switched to this operating system.

8

FWIW, if you want to use Zotero as your frontend, you can install Better BibTeX, and configure it to automatically maintain a bibtex/biblatex file.

9

Yes it’s for me the better way…

10

Thank you.

Heh, considering the prices of MacOS hardware here in Croatia, I bet that for the foreseeable future I’m destined to stay on Linux. :smile:

11

I did install it and wonder if the appropriate setting is:

Preferences --> Better BibTex --> Automatic Export --> On Change

or there is something else as well?

12

The only other thing you might need to configure is how to generate the citekeys.

13

Do so, please. Although I like MacOS quite a lot because it offers the familiar Unix experience and one can also run natively many commercial programs such as MS Word, I must admit that the hardware and support is pretty shitty. Can you imagine that a key falls out of your Dell or HP or Asus or … laptop keyboard while you are typing? This is precisely what happened to my 3-year-old MacBook Pro (sic!). Ok, Apple seems to have this issue covered by offering a free service for the affected laptops. Yet I had to wait several months for an appointment, fine, because of corona. It took their affiliated partner a whole week (!) to fix the issue once I got the appointment though. And I’m still waiting for more than a week now for my laptop to be sent back to me.

Their hardware is unreliable, their partners are unreliable (in Germany at least), I’m really frustrated because I can’t do my job properly for two weeks now. So when you are ready to spend 1300 Euros for a new laptop (it’ll happen sooner than you think), don’t give them to Apple. Give them to Dell or HP. Install Linux, or use Windows, which is though crappy in many aspects but is being constantly improved at admirable rates. For example, in these past two weeks I discovered that Windows offers an OpenSSH server as a system package. I wonder what other hidden surprises it has.

14

Hmm, this is quite strange…it must be that “The Times They’re Changing!” since I always had impression that people are using Macs due to superior quality of its hardware. :confused:

15

That’s a lasting impression and company image from the 2000s I guess. There is a growing number of complaints about newer laptop models from around 2016. The company’s striving to make the devices lighter and thinner has apparently reached the physical limits. One cannot have a millimeter-thin plastic cap on tiny hinges that is at the same time mechanically durable - that’s just with regard to my issue. At the same time, the company obviously tries to keep the prices as low as possible - at least their notebooks are not much more expensive than notebooks from Dell or HP or Lenovo of a comparable configuration, if at all. They’ve been using the same Intel processors, the same memory and ssd components as others. One cannot expect some ultra quality from their products, but it should be decent and a device should not fall apart easily. Although their operating system is currently solid enough, they’ve been constantly and without much advertising moving away from it being UNIX for technical users among others with a nice and consistent desktop interface (Emacs was shipped pre-installed with MacOS until relatively recently, for example) to a media consumption mobile interface for masses. Microsoft, on the other hand, has made correct conclusions from their Windows 8 experiment and is moving in the right direction now.

But enough about it, sorry for this off-top fueled by my personal frustration.

16

Hey, did you make some progress with it ?

17

No. I studied the internals of Ebib a little bit and had an impression that this would require quite a lot of work. BibDesk is currently doing its job, I’m used to the workflow and have little incentive for changes because of much more pressing matters.

1 Like
18

Ok, thanks for reply. Same here. I’am curious about it but it’s difficult to work without Zotero :wink:

19

Hi folks,

I’m going to pick this thread up because I have something to add :slight_smile:
I was an “early” adopter of Zotero where I was using it already in 2012 (i know it had been around a bit already). I liked it … it did the job well.
I had a huge database of PDF files to the point where I needed more than the free 1GB (or was it 2GB at the time …?) space allocation so I ended up purchasing the the options that allowed me to store more PDFs.
However, I got to a point where:
a) I thought that I shouldn’t be paying for a service like this or at least did need to
b) I had privacy concerns around my notes, etc. being stored on someone else’s server
c) I wanted to be able to share my bibliographic database.

That is when I entered the world of WIKINDX which allows me to overcome all the above. Its a LAMP based solution and I have it running on an old server. Now that I have discovered the wonderful world of ORG-ROAM, my next adventure is to figure out how I substitute Zotero in @nobiot 's schematic at the beginning of this thread.

Another important aspect is that the LAMP database is stored in one place with WIKINDX rather than synchronized across various machines. I’m not saying this is the perfect solution. It satifies my needs just like Zotero does for many. The great thing about the world of the Internet is that we can all find the solutions that best work for us and share out experiences with everyone.