Help for installation of org roam in windows

[ianian]

I use Emacs only for org-mode for several years. But I still think I am just a newbie for Emacs.

First, please allow me to describe my installtion process. My computer using Windows 10.

1. I installed the package with melpha.
2. I installed the emacs-sqlite3 package, downloaded the sqlite-tools and configured the path.
3. I modified the org-roam-db.el as https://org-roam.readthedocs.io/en/master/installation/ says.
4. I created the org-roam file directory, and put the following line (setq org-roam-directory "~/org-roam") in the config.
5. I installed the graphiz and configure the path.
6. M-x org-roam-db-build-cache OK. The org-roam.db was sucessfully created.
7. I add (add-hook 'after-init-hook 'org-roam-mode) in the config and restart Emacs

The problem:

1. The start up process was slower than before.
2. After start up there is message in the mini buffer: Query timeout error: "Query timed out", 30
3. C-x b prompt let*: Symbol’s function definition is void: destructuring-bind, which I was binded to switch-to-buffer

[nobiot]

Hi @ianian,

I would not claim to be an expert, but I have been using org-roam on Windows 10 myself, so let me try. I invite others to jump in where my attempt falls short.

I think Problem 1 is a bit expected, and Problems 1 and 2 in your case seem to be related. I would think Problem 3 is a separate issue, and may not be directly caused by org-roam or your implementation of emacs-sqlite3.

For 1.
How slow? Perhaps your Emacs won’t launch and return control back to you until you see the “Query timeout error” message? (30 seconds or more?).

The reason I say Problem 1 is expected is that, I think, org-roam tries to launch during the Emacs launch process because of your after-init-hook setting. As part of the starting process, org-roam tries to see if there is any update in your org-roam-directory that updates your dababase org-roam.db.

I believe how long this startup process takes depends mostly on the number of updated note files. But if the udpates are none or minimal (let’s say, you might have changed one or two org files when org-roam was off), then this should not feel like taking too long (my experience confirms this). And you should see a message like this: (org-roam) files: 0, links: 0, tags: 0, titles: 0, refs: 0, deleted: 0. If this takes too long, then it is probably because of your Problem 2.

For 2… This needs a bit more analysis. I tend to use view-echo-area-message to see more messages around the timeout error message you get. I hope others may give us a better way to get you more information. Like this method..

• Do you see any more information around the timeout error message?
• How big (in terms of kb or mb) of your database (via Explorer)?
• After the timeout error message, can you open org-roam-buffer for one of your org file, and see the backlinks properly shown?
• Can you change the org file, and see the backlink properly updated (in some cases, saving individual files works, but org-roam-db-build-cache does not).
• Can you run org-roam-db-build-cache manually, and successfully?

These would be the questions I would be asking myself if I saw the same timeout message (which I have seen a couple of times; but most of the times, I could just ignore it).

For 3… This is weird. I suspect it may not be related to org-roam. Do you see the list of buffers when you do C-x b, or you just immediately get the let*: Symbol's function message? Is there any location in your configuration where you might have bound C-x b to a command other than switch-to-buffer?

[nobiot]

Also, as a side note:

Jethro and some of community members are working to make the use of emacs-sqlite3 as the standard, mainly to make it easy for Windows users to start with org-roam. I don’t think this change will influence the issues you currently have. But for the long-run, we won’t have to manually maintain our “hack” on our own. I think it will be a change we can embrace as Windows users, when this goes through to the next update.

See https://github.com/org-roam/org-roam/pull/701 for detail, if you are interested.

Thanks.

[ianian]

Thank you @nobiot for your detailed reply. Let me answer the questions one by one.

1. How slow?

Now the start up time is about 38 seconds. Before is about 15-20 seconds.
The message is: (org-roam) files: 0, links: 0, tags: 0, titles: 0, refs: 0, deleted: 0

2. More messages for the echo area message

Loading cua-base…done
Loading d:/XXXX/config.el (source)…
org-super-agenda-mode enabled.
Loading d:/XXXX/config.el (source)…done
Loaded d:/XXXX/config.el
(org-roam) files: 0, links: 0, tags: 0, titles: 0, refs: 0, deleted: 0
Query timeout error: “Query timed out”, 30

2.0.1 The result of (let ((debug-on-error t)... command.

This command is only sucessful at the first time I run it, the minibuffer retures:

(org-roam) files: 3, links: 1, tags: 0, titles: 3, refs: 0, deleted: 0

If I run it again, it returns

Debugger entered–Lisp error: (file-error “Removing old name” “Permission denied” “d:XXX/org-roam.db”)
#(“d:XXX/org-roam.db”)
apply(# “d:XXX/org-roam.db”)
delete-file(“d:XXX/org-roam.db”)
(progn (delete-file (org-roam-db–get)))
(if force (progn (delete-file (org-roam-db–get))))
org-roam-db-build-cache(t)
(let ((debug-on-error t) (org-roam-list-files-commands nil)) (org-roam-db-build-cache t))
eval-buffer() ; Reading at buffer position 98
funcall-interactively(eval-buffer)
call-interactively(eval-buffer record nil)
command-execute(eval-buffer record)
#f(compiled-function (cmd) #<bytecode 0x1f1edb5>)(“eval-buffer”)
ivy-call()
ivy-read("M-x " [rng-ipattern-after connection-failed ansi-color-apply-overlay-face epg-context-signers locals 0 rng-what-schema message-required-headers denied tls-format-message epg–status-VALIDSIG org-publish–store-crossrefs bbdb-current-record 0 org-emphasis-regexp-components mml-secure-smime-sign-with-sender url-cookie-multiple-line url-scheme-registry gnus-agent-group-covered-p nnmail-fix-eudora-headers gujr mixal gnus-summary-recenter avl-tree–stack-store–cmacro cl-print–preprocess matched-string tls-success :examined -some? bibtex-SuppCollection 0 post-deadline cl–struct-epg-signature-p ! " f-directory? message-kill-actions nnheader-translate-file-chars scale ' nnoo-import-1 gnus-summary-catchup-from-here :contents rx-trans-forms request-list elfeed-search-date-face not-at-beginning-of-line helm expired message-send-mail-function …] :predicate commandp :require-match t :history counsel-M-x-history :action #f(compiled-function (cmd) #<bytecode 0x1f1edb5>) :sort t :keymap (keymap (67108908 . counsel–info-lookup-symbol) (67108910 . counsel-find-symbol)) :initial-input nil :caller counsel-M-x)
counsel-M-x()
funcall-interactively(counsel-M-x)
call-interactively(counsel-M-x nil nil)
command-execute(counsel-M-x)

2.1 Do you see any more information around the timeout error message?

Yes. The result is as above in answer 2.

2.2 How big (in terms of kb or mb) of your database (via Explorer)?

36kb (36,864 bytes)

2.3 After the timeout error message, can you open org-roam-buffer for one of your org file, and see the backlinks properly shown?

Yes, the backlinks properly shown.

2.4 Can you change the org file, and see the backlink properly updated

Yes, I can change the org file, and the backlink properly updated.

2.5 Can you run org-roam-db-build-cache manually, and successfully?

The result is strange. The first time I run org-roam-db-build-cache. The mini-buffer reture apply: Selecting deleted buffer . And I run org-roam-db-build-cache again. It build correctly and return: (org-roam) files: 0, links: 0, tags: 0, titles: 0, refs: 0, deleted: 0

And if I run org-roam-db-build-cache again(third time), still: apply: Selecting deleted buffer . The fourth time: (org-roam) files: 0, links: 0, tags: 0, titles: 0, refs: 0, deleted: 0

3 The C-x b problem

For now I also think it’s a seperate problem. Now I bind it to counsel-ibffer. It works.

3.1 Do you see the list of buffers when you do C-x b, or you just immediately get the let*: Symbol's function message?

No, I didn’t see the list of the buffer. It immediately get the message.

3.2 Is there any location in your configuration where you might have bound C-x b to a command other than switch-to-buffer ?

No I have no config for bounding C-x b to switch-to-buffer. But when I M-x switch-to-buffer, in the minibuffer It shown the key was binded and It works for me before.

Even if I bound C-x b to switch-to-bufferin the config. It still returns the error message. But the M-x switch-to-buffer works ok.

4 New problems after enable the roam mode.

4.1 The scratch buffer problem.

After I start the Emacs and switch to the scratch buffer. The major mode is “Fumdamental” not “emacs lisp mode”. And the whole buffer is empty. There were 2 lines of commented description at the top of the buffer. But now it’s gone.

4.2 The buffer list problem

Before if I opend 4 buffers, I could switch to them. The numbers of active buffers seems has no limit, even after restart the Emacs.

But now every time I restart Emacs, there will be only 2 active buffers in the buffer list.

Thank you again for your time.

[nobiot]

Hi @ianian,

Let’s focus on what you would really like to resolve.
You might see problems 3 and 4 important for your day-to-day Emacs life (), but shall we look at org-roam since that’s what connects us here, isn’t it? To be frank, I don’t think 3 and 4 are related to org-roam (I can be wrong, so I’ll keep mind open here).

For slow launch, let’s see if we can do something about it after perhaps fixing what looks to be a bit more grave issue of problem 2.

And… for problem 2, you may not like to hear this, but perhaps I’d suggest that you start again with a clean(er) slate.

org-roam is now officially based on emacsql-sqlite3. See the melpa package list (image below) – this is from 2020-05-28, today in my time zone.

I don’t use melpa, but I believe you can use the new version available there to override the hack you did for sqlite3 (I mean, if you can hack your way in the code in Emacs, then I don’t think you qualify as a newbie, really ).

Looking at the errors you reported, I think your org-roam is mostly working, at least in a very similar way to mine. The only thing to solve looks to be the “Permission denied” error in 2.0.1. I have seen it a couple of times in my set up, and every time, it was my coding error of my own plugin for org-roam. I tend to clear the db (org-roam-db--clear) or close Emacs and delete the db file manually. And fix my code, and rebuild the db. It’s hard to see what causes your problem (e.g. is there anything special about your D: drive? Why D: instead of C:? Do you really have full admin authorization there? etc. etc.). By using the melpa version, you can eliminate the possibility that your hack is the cause. That will be a cleaner starting point for further analysis.

The “strange” result in 2.5 is something I have always had. I have learnt to live with it now. Everything I want to work has been working with this issue. So for me, it’s low priority. It’s relieving to see that another Windows user also has the same issue. It may be Windows specific. I might dig deeper to see if I can resolve it, some day. But for now, I would like to use org-roam to create more meaningful notes, rather than fix the tool.

[ianian]

Thank you again for your reply. I think I will keep on using org-roam with these problem, and wait the for the next stable release.

[Klord]

I’m encountering a similar issue on Windows 10. I just updated all MELPA packages, cleared the database (org-roam-db--clear) and restarted.

Loading time was 2 min 30 seconds (with timer). Loading ends with the message: Query timeout error: "Query timed out", 30. Only one buffer is restarted, despite the fact that I activated desktop-save-mode.

Without org-roam, loading time was about 30s. I have always had the query timeout and failure of the desktop-restore with org-roam; however, the loading time has recently increased dramatically (from ~55 seconds to the current 2min30) – I suspect, from a MELPA update.

Like the original poster, my database is small, and I only have ~ 20 files in org-roam.

During the installation process, I encountered the error org-roam cannot find executable sqlite3, which I solved by downloading this binary of SQLite for Windows and putting it in a PATH-referenced folder.

I’m happy to try any suggestion.

[nobiot]

Rebuilding the database always takes much longer than doing a smaller incremental update. Do you experience the long-time loading time after the initial db build?

I don’t use desktop-save-mode, but one suggestion to improve experience of loading might be to move the loading of org-roam to after loading of desktop-save-mode.

If you followed the documentation, you’d have add-hook ‘after-init-hook ‘org-roam-mode to specify when Org-roam should be loaded. You can move this to another hook more appropriate for your loading sequence (perhaps after desktop-save-mode).

Is there a hook like after-desktop-save-mode-hook or something like that?

[Klord]

Thank you for your help!

I experience the long-time loading each time I start Emacs. I just re-did the test now, without re-building the database, and moving (desktop-save-mode 1) to last in my init.el. The loading seems to occur normally up to around 30 s, then Emacs lags with an empty buffer, and a minibuffer set to:

(org-roam) files: 0, headlines: 0, links: 0, tags: 0, titles: 0, refs: 0, deleted: 0)

It lags for 1 m 30 s, and then, 2 minutes after having been launched, resumes the loading of just one buffer, becomes usable, and displays this in the minibuffer:

Query timeout error: "Query timed out", 30

There is no hook to after-desktop-save-mode-hook or anything similar.

[nobiot]

Hope you can resolve your issue soon, but I’m not sure how much of help I can be.

I do not have the same issue as you do.
One thing I could suggest is to isolate the Org-roam issue from others.

Comment out your (add-hook) where you load org-roam-mode.

;(add-hook 'after-init-hook 'org-roam-mode)

See how my Emacs starts up in this video clip below. The total length is about 38 seconds; 21 seconds or so are used to just start Emacs. I then manually call org-roam-mode, which finishes almost instantaneously (I have about 5 files for this).

2020-06-26_17-02-29.mp4 (video)

My .emacs for this video is located in my GitHub repo; I just commented out the part where org-roam is loaded at start up as above.

This config is used only for writing Org-roam guide. For my usual one, I delay loading of Org-roam until I manually call it.

You might like to load up both desktop and org-roam (and others) at startup, but maybe some sort of selective loading is one way you might like to consider…

If the timeout issue still persists, then getting a backtrace would help the project look at the problem. I’ve written up a quick guide here – see section 6. Call M-x toggle-debug-on-error manually before you call org-roam. You will get a backtrace if you hit the error.

[Klord]

Thanks a lot for your help. I followed your advice and did some tests. Here are the results:

• It’s (desktop-save-mode 1) which creates the issue. If I remove it, everything works fine.
• Starting org-roam-mode manually or as a hook to after-init makes it complete in a few seconds.
• Having desktop-save-mode set to 1 creates the issue only if at least one buffer has to be restored. If I clear the desktop (M-x desktop-clear) and immediately switch off Emacs, it restarts normally. If I open init.el in the buffer, exit Emacs, and then re-open it, I have the Query Timeout error and the veryyyy long hang.

To be more specific, I can re-create the problem with this minimal init.el:

(require 'package)
(add-to-list 'package-archives (cons "melpa" "https://melpa.org/packages/") t)
(package-initialize)
(require 'use-package)


(use-package org-roam
  :hook
  (after-init . org-roam-mode)
  :custom
  (org-roam-directory "~/org/org-roam")
  (org-roam-completion-system 'helm)
  :bind (:map org-roam-mode-map
              (("C-c n l" . org-roam)
               ("C-c n f" . org-roam-find-file)
               ("C-c n g" . org-roam-graph-show))
              :map org-mode-map
              (("C-c n i" . org-roam-insert))))
			  
			  
(desktop-save-mode 1) 

[nobiot]

… So it’s an issue of decktop-mode?

Does this issue interact with Org-roam. In other words, does this issue happens only when combined with Org-roam?

If so… Maybe putting (desktop-save-mode 1) before (use-package org-roam … Would that change anything?

I don’t use desktop-mode, so my idea is running out.
An extreme alternative might be not to use desktop-mode.
Or perhaps you could live with the problem, and enable Org-roam later manually or with a hook to Org-mode, etc.?

I hope you can find your own solution to achieve a good-enough balance between convenience and frustration…

[Klord]

No, as you guessed, it’s an issue of org-roam, which appears when I use it with desktop-save-mode. It happens when Emacs tries to restore buffers into which org-roam was activated. I would be interested in knowing if others can replicate the problem, or if it’s just me.

For the time being, I will stop using desktop-save-mode and, if needed, call it manually with desktop-save and desktop-clear (curiously, both work).

[nobiot]

Just one last thing…

I tried desktop-save-mode by adding (desktop-save-mode 1) at the end of my init file.
I do get similar error, but I am not sure I am doing it the way you do – I don’t know how this mode should work and get confused where I should save the desktop file.

One thing that you might like to do is to put desktop-save-mode in after-init-hook after Org-roam.
Like this:

(add-hook 'after-init-hook 'desktop-save-mode)

Or with using use-package, perhaps like this (?)

(use-package org-roam
... all the code for org-roam)

(use-package desktop-save-mode
  :hook
  (after-init . desktop-save-mode)
  ... other config ...)

Or maybe yon can look to use the after option (directive?) to make it explicit to load desktop-save-mode after org-roam.

This way, at least I could avoid the long loading time.

I suspect one of the reasons why you have the loading issue is, by putting (desktop-save-mode 1) regardless of where in init file it is, you are enabling destop-save-mode before org-mode has been enabled. This is because it looks like you are enabling org-roam-mode after init file has been loaded in this (after-init . org-roam-mode).

[ianian]

Thank you for solving my problem!

I don’t know what leads you to get the source of the problem : desktop-save-move !

I disable the desktop-save-move. My emacs start up speed back to normal, and the Query timeout error: “Query timed out”, 30 message gone.

There comes a new error about start-up screen. And I solved it by (setq inhibit-splash-screen t)

It also solved my scratch buffer problem!

Now the only error is Eager macro-expansion failure: (error "Unknown rx form ‘anychar’"). This error came out after I upgrade the org-roam 1.2.0. I saw some user also has this problem from a previous post. I will stay with it and waiting for the new upgrade.

[nobiot]

@ianian, it looks like the issue was fixed 16 days ago. Try upgrading Org-roam if you haven’t since before that.

[Klord]

@nobiot: Thank you, you are being very resourceful. Unfortunately your suggestions do not work. I tried all combinations and others (note that the syntax for use-package is desktop and not desktop-save-mode), to no avail.
It seems that desktop-save-mode needs to be called directly after init (as per https://github.com/jwiegley/use-package/issues/296), so it cannot be called after org-roam.

@ianian: I’m happy to see we at least identified where the issue comes from, and that removing desktop-save-mode solves it for you too!

[nobiot]

Yes… I think I am able to reproduce your issue.

I did a couple of things, but I’m afraid this is beyond me.
As a record, I’ve written some notes in Gist.

I believe this is Windows only issue (not verified yet), and probably beyond Org-roam. I am suspecting this is Windows specific in relation to emacsql, or emacsql-sqlite, but not entirely sure.

[ianian]

The “any char” issue was fixed after the release of 1.2.0 version. I used the melpha stable. I could wait for the next stable.

[cookyzed]

I had the 2nd issue on both of my two Windows 10 computers after adding org-roam to Emacs:

After start up there is message in the mini buffer: Query timeout error: "Query timed out", 30

and I’m not enabling desktop-save-mode in my configuration.

I got it fixed after removing the line that enables org-roam-mode under custom-set-variables:

(custom-set-variables
 ;; '(org-roam-mode t nil (org-roam)) ;; <--- comments out or remove via the `customize` command
)

I have no idea why that line was added automatically. Here is part of my org-roam related configuration using use-package:

(use-package org-roam
  :hook
  (after-init . org-roam-mode)
  :custom
  (org-roam-directory "d:/sync/wiki/"))

Anyway, removing that line fixes the issue for me, and org-roam related functionality works just fine.

P.S. I have some org files under the org-roam-directory, but every time it shows 0 files on startup:

(org-roam) files: Δ0, headlines: Δ0, links: Δ0, tags: Δ0, titles: Δ0, refs: Δ0, deleted: Δ0

Still, everything works fine like org-roam-find-file.

---

The following are something I found while investigating the issue, hope it helps fix the “real” issue.

After searching for the error message, I think it came from the emacsql package:

(cl-defmethod emacsql-wait ((connection emacsql-connection) &optional timeout)
  "Block until CONNECTION is waiting for further input."
  (let* ((real-timeout (or timeout emacsql-global-timeout))
         (end (when real-timeout (+ (float-time) real-timeout))))
    (while (and (or (null real-timeout) (< (float-time) end))
                (not (emacsql-waiting-p connection)))
      (save-match-data
        (accept-process-output (emacsql-process connection) real-timeout)))
    (unless (emacsql-waiting-p connection)
      (signal 'emacsql-timeout (list "Query timed out" real-timeout))))) ;; <---- here

The org-roam-db-build-cache function (which should be the first function getting emacsql-wait called on startup) completed normally as it prints the (org-roam) files: Δ0, headlines: Δ0 ... message.

I think the Query timeout error: "Query timed out", 30 issue (causing delay of 30 seconds) happens when loading the *scratch* buffer, as the buffer is empty on startup. But before adding and enabling org-roam, or after fixing the issue by removing that line under custom-set-variables, the *scratch* buffer would have two lines of comments:

;; This buffer is for text that is not saved, and for Lisp evaluation.
;; To create a file, visit it with and enter text in its buffer.