Last weekend I set up a 2018 MacbookPro from work in the same manner as my daily driver, a 2022 M1 Macbook Pro. Both run the latest macOS (Ventura) and Emacs 29 via Emacs+Plus. Both have the same .emacs, the same .emacs.d (via Git) and the same PATH environment. Except for org-roam (I use the latest version of v1) and Helm, every package is up to date.
Yet, on the newly configured 2018 machine I encounter this weird bug:
Error setting org-roam-mode: (emacsql-timeout Query timed out 30)
I tried a lot already and now am at my wits’ end. My gut feeling is that this has something to do with permissions in macOS - but it could also be something completely different.
I found these three articles online but could not solve the issue.
Helpful pointers would be really appreciated!
This is the result of (debug (org-roam-mode)):
Debugger entered--Lisp error: (emacsql-timeout "Query timed out" 30)
signal(emacsql-timeout ("Query timed out" 30))
#f(compiled-function (connection &optional timeout) "Block until CONNECTION is waiting for further input." #<bytecode 0xdcdbb9fef8ddf18>)(#<emacsql-sqlite3-connection emacsql-sqlite3-connection-1ffb2e760900>)
apply(#f(compiled-function (connection &optional timeout) "Block until CONNECTION is waiting for further input." #<bytecode 0xdcdbb9fef8ddf18>) #<emacsql-sqlite3-connection emacsql-sqlite3-connection-1ffb2e760900> nil)
emacsql-wait(#<emacsql-sqlite3-connection emacsql-sqlite3-connection-1ffb2e760900>)
#f(compiled-function (connection sql &rest args) #<bytecode -0x1eff5dcf1872eb1c>)(#<emacsql-sqlite3-connection emacsql-sqlite3-connection-1ffb2e760900> [:pragma (= busy-timeout $s1)] 15000)
apply(#f(compiled-function (connection sql &rest args) #<bytecode -0x1eff5dcf1872eb1c>) #<emacsql-sqlite3-connection emacsql-sqlite3-connection-1ffb2e760900> ([:pragma (= busy-timeout $s1)] 15000))
emacsql(#<emacsql-sqlite3-connection emacsql-sqlite3-connection-1ffb2e760900> [:pragma (= busy-timeout $s1)] 15000)
#f(compiled-function (conn slots) #<bytecode -0x118a18c66d0e1827>)(#<emacsql-sqlite3-connection emacsql-sqlite3-connection-1ffb2e760900> (:file "~/.emacs.d/org-roam-db/org-roam.db"))
apply(#f(compiled-function (conn slots) #<bytecode -0x118a18c66d0e1827>) (#<emacsql-sqlite3-connection emacsql-sqlite3-connection-1ffb2e760900> (:file "~/.emacs.d/org-roam-db/org-roam.db")))
#f(compiled-function (&rest args) #<bytecode 0x1d0de56ae1e88461>)(#<emacsql-sqlite3-connection emacsql-sqlite3-connection-1ffb2e760900> (:file "~/.emacs.d/org-roam-db/org-roam.db"))
apply(#f(compiled-function (&rest args) #<bytecode 0x1d0de56ae1e88461>) #<emacsql-sqlite3-connection emacsql-sqlite3-connection-1ffb2e760900> (:file "~/.emacs.d/org-roam-db/org-roam.db"))
initialize-instance(#<emacsql-sqlite3-connection emacsql-sqlite3-connection-1ffb2e760900> (:file "~/.emacs.d/org-roam-db/org-roam.db"))
#f(compiled-function (class &rest slots) "Default constructor for CLASS `eieio-default-superclass'.\nSLOTS are the initialization slots used by `initialize-instance'.\nThis static method is called when an object is constructed.\nIt allocates the vector used to represent an EIEIO object, and then\ncalls `initialize-instance' on that object." #<bytecode -0x1e27f44ed359e4fa>)(emacsql-sqlite3-connection :file "~/.emacs.d/org-roam-db/org-roam.db")
apply(#f(compiled-function (class &rest slots) "Default constructor for CLASS `eieio-default-superclass'.\nSLOTS are the initialization slots used by `initialize-instance'.\nThis static method is called when an object is constructed.\nIt allocates the vector used to represent an EIEIO object, and then\ncalls `initialize-instance' on that object." #<bytecode -0x1e27f44ed359e4fa>) emacsql-sqlite3-connection (:file "~/.emacs.d/org-roam-db/org-roam.db"))
make-instance(emacsql-sqlite3-connection :file "~/.emacs.d/org-roam-db/org-roam.db")
emacsql-sqlite3("~/.emacs.d/org-roam-db/org-roam.db")
org-roam-db()
org-roam-db-build-cache()
org-roam-mode()
(debug (org-roam-mode))
eval((debug (org-roam-mode)) nil)
elisp--eval-last-sexp(nil)
eval-last-sexp(nil)
funcall-interactively(eval-last-sexp nil)
call-interactively(eval-last-sexp nil nil)
command-execute(eval-last-sexp)
Just to eliminate the most common error I have seen in the forum and GitHub issues after upgrading macOS.
Permission for the executable. You use emacsql-sqlite3 so the executable is the one from your “package manager” – I don’t know which one you use, but I supposed you can locate the one your Emacs uses (not sure if it is the same one as where sqlite3 in Terminal [1]), and see if you have executable permission to it.
Thanks for the reply and the pointer. The idea about the permissions is not bad, the good people at Apple have added a lot of additional security walls in the last few years. But it might not be the answer for this issue.
For some reason I just checked the activity monitor on both machines. On the working machine there is one sqlite3 process. On the one with issues there is not zero nor one but two sqlite3 processes. I don’t know what to make of this.
I checked permissions and even gave Emacs, emacsql and sqlite full disk access. Still no improvement. Is there any way a connection from sqlite to the database can be blocked?
For what its worth: If I delete the org-roam.db, a new (and empty) one is created.
Sorry that I have no idea how macOS works. I don’t use it.
It’s very weird when you can create a new file but can’t read it — is this even possible? Not sure if commandline sqlite3 can open the db file directly without Emacs
Why not try the emacsql-sqlite and compile the executable with gcc or clang? (Instead of emacsql-sqlite3 and sqlite3).
Thanks a lot for following up on this and the link, I really do appreciate the effort!
I’ve experimented with different versions of emacsql and emacs. Downgrading does break other parts of my config, so it would not be my first choice - but it also did not work. The strangest thing about this is that one machine is running perfectly fine with this setup, whereas the second (which is setup identically) fails.
Btw I also used “console.app” on both machines to see whether there are unique sqlite error messages for the struggling Intel Macbook Pro.
This is the list of unique error messages for the machine with the working setup (M1 MBP):
error 21:14:58.781938+0100 syspolicyd open flag(s) 0x01000000 are reserved for VFS use and do not affect behaviour when passed to sqlite3_open_v2
error 21:14:58.782051+0100 syspolicyd cannot open file at line 46922 of [554764a6e7]
error 21:14:58.782390+0100 syspolicyd os_unix.c:46922: (2) open(/private/var/db/DetachedSignatures) - No such file or directory
sqlite3_open_v2
error 21:14:58.832065+0100 taskgated cannot open file at line 46922 of [554764a6e7]
error 21:14:58.832089+0100 taskgated os_unix.c:46922: (2) open(/private/var/db/DetachedSignatures) - No such file or directory
sqlite3_open_v2
[…]
standard 21:15:02.084909+0100 searchpartyd automatic index on KeyGenerationBeaconKeys(beaconIdentifier)
[…]
error 21:15:10.250469+0100 Emacs
error 21:15:10.250505+0100 Emacs cannot open file at line 46922 of [554764a6e7]
error 21:15:10.250520+0100 Emacs os_unix.c:46922: (2) open(/private/var/db/DetachedSignatures) - No such file or directory
And this is the list for the struggling machine:
standard 21:19:50.284727+0100 searchpartyd automatic index on KeyGenerationBeaconKeys(beaconIdentifier)
[…]
error 21:19:51.606525+0100 tccd open flag(s) 0x01000000 are reserved for VFS use and do not affect behaviour when passed to sqlite3_open_v2
error 21:19:51.606642+0100 tccd cannot open file at line 46922 of [554764a6e7]
error 21:19:51.606668+0100 tccd os_unix.c:46922: (2) open(/private/var/db/DetachedSignatures) - No such file or directory
[…]
error 21:19:54.354862+0100 taskgated open flag(s) 0x01000000 are reserved for VFS use and do not affect behaviour when passed to sqlite3_open_v2
error 21:19:54.354935+0100 taskgated cannot open file at line 46922 of [554764a6e7]
error 21:19:54.354960+0100 taskgated os_unix.c:46922: (2) open(/private/var/db/DetachedSignatures) - No such file or directory
[…]
error 21:19:54.664608+0100 ContextStoreAgent open flag(s) 0x01000000 are reserved for VFS use and do not affect behaviour when passed to sqlite3_open_v2
error 21:19:54.666307+0100 ContextStoreAgent cannot open file at line 46922 of [554764a6e7]
error 21:19:54.666347+0100 ContextStoreAgent os_unix.c:46922: (2) open(/private/var/db/DetachedSignatures) - No such file or directory
error 21:19:54.703269+0100 progressd volume does not support data protection, stripping SQLITE_OPEN_FILEPROTECTION_* flags
The only real difference between is:
error 21:19:54.703269+0100 progressd volume does not support data protection, stripping SQLITE_OPEN_FILEPROTECTION_* flags
Where I am going with this question is a wild guess as follows.
Is it possible you share an external SSD drive (or internal) between the two machines? For example, you have the M1 Macbook Pro and the Intel-CPU MacBook Pro 2018, and you share the Org files and Org-roam db files in an external SSD drive.
Somehow, googling your error message has led me to these Apple support articles:
The scenario I am speculating is something like this:
You use Org-roam on your MBP 2018, which creates the DB file in the external SSD drive
Somehow (wild guess), the SSD drive is also connected to M1 MBP, and detects the new file just created and encrypts it for M1 Apple Silicon (different CPU architecture and/or different machine)
MBP 2018 is unable to decrypt the DB file because its encrypted by the M1 Apple Silicon (even though it is MBP 2018 that has created it)
I don’t know if this is even plausible. But you seem to be experiencing something that is very unique to your use scenario and perhaps unique to macOS…
Again, thanks for this. I appreciate the creative approach here (and you don’t even own a Mac!) but sadly this does not seem to be the issue. The setup is somewhat similar because I have my .emacs.d in Github. I forgot that I had the org-roam.db there as well - but moving this file somewhere else does not help.
Running v1 doesn’t make it easier, I guess.
I have solved this issue in the most overkill and ridiculous way possible. In the past few days I have re-written the core functionality of org-roam-v1 (find, insert, show-backlinks) with ripgrep and a lot of regex replacing emacsql. I still have to add “tags” but otherwise “orgrr” (for org-roam-replica) is surprisingly fast and only depends on rg being installed. The idea was to become independent of the future fate of v1 and also to learn more elisp.
