Slightly off-topic, but still related 
I’ve found this tool called orgNv (OrgNV - Org Navigator), which sounds really promising to me - supposed to be lighter than deft, better integrated into the working environment, and should also work with non-english languages.
I’ve installed the .el and included everything, and I see the functions that it supplies - but I’m not getting any results from the search.
I wonder if there’s anyone who tried it here/is interested in trying and could share some insight…
Thanks!
Made 4 minor changes, and worked on my Ubuntu.
Emacs 29.0.50 (sorry, I just compiled a new Emacs).
If you set orgnv-directories correctly, it should work.
The whole modified source below.
;; -*- lexical-binding: t -*-
;;; orgnv.el --- notes database based on grep
;;;
;;; Copyright (C) 2020 Juan Jose Garcia-Ripoll
;;
;; All rights reserved.
;; Redistribution and use in source and binary forms, with or without
;; modification, are permitted provided that the following conditions are met:
;; 1. Redistributions of source code must retain the above copyright
;; notice, this list of conditions and the following disclaimer.
;; 2. Redistributions in binary form must reproduce the above copyright
;; notice, this list of conditions and the following disclaimer in the
;; documentation and/or other materials provided with the distribution.
;; 3. Neither the names of the copyright holders nor the names of any
;; contributors may be used to endorse or promote products derived from
;; this software without specific prior written permission.
;;
;; THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
;; AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
;; IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
;; ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
;; LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
;; CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
;; SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
;; INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
;; CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
;; ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
;; POSSIBILITY OF SUCH DAMAGE.
;;
;;; Version: 0.1
;;; Author: Juan Jose Garcia-Ripoll <juanjose.garciaripoll@gmail.com>
;;; Keywords: org mode, plain text, notes, Deft, Simplenote, Notational Velocity
;; This file is not part of GNU Emacs.
(require 'org)
(defgroup orgnv nil
"Emacs OrgNV mode."
:group 'local)
(defcustom orgnv-directories (list (or (expand-file-name org-directory)
(expand-file-name "~/.orgnv/")))
"List of directories where OrgNV looks for notes."
:type 'directory
:safe 'stringp
:group 'orgnv)
(defcustom orgnv-recursive t
"If true, recursively search the orgnv-directories and its subfolders looking for notes."
:type 'boolean
:group 'orgnv)
(defcustom orgnv-extensions '("org")
"File name extensions to consider when looking for notes."
:type '(repeat string)
:group 'orgnv)
(defcustom orgnv-grep-command (or (executable-find "grep")
"grep")
"Executable file that implements the GNU grep utility."
:type 'file
:safe 'stringp
:group 'orgnv)
(defcustom orgnv-description-limit 1000
"Maximum number of files to display in the browser."
:type '(choice (integer :tag "Limit number of files displayed")
(const :tag "No limit" nil))
:group 'orgnv)
(defconst orgnv-buffer "*orgnv output*"
"OrgNV buffer name")
(defcustom orgnv-context-size 3
"Maximum number of lines to consider after the title."
:type 'integer
:safe (lambda (x) (< 0 x 10))
:group 'orgnv)
(defcustom orgnv-title-pattern "#\\+TITLE\\:"
"Pattern used for searching titles in notes."
:type 'string
:group 'orgnv)
(defcustom orgnv-display-limit 300
"Maximum number of items do display in the browsing buffer"
:type 'integer
:group 'orgnv)
(defcustom orgnv-database-sort-predicate 'orgnv-compare-titles
"Either NIL, if we do not sort the database, or a function with
two arguments that can be passed to SORT."
:type '(or function symbol))
(defcustom orgnv-install-keybindings t
"Whether to install OrgNV under C-x g"
:type 'boolean
:group 'orgnv)
(defgroup orgnv-faces nil
"Faces used in Orgnv mode"
:group 'orgnv
:group 'faces)
(defface orgnv-title-face
'((t :inherit font-lock-function-name-face :bold t))
"Face for OrgNV file titles."
:group 'orgnv-faces)
(defface orgnv-separator-face
'((t :inherit font-lock-comment-delimiter-face))
"Face for OrgNV separator string."
:group 'orgnv-faces)
(defface orgnv-summary-face
'((t :inherit font-lock-comment-face))
"Face for OrgNV file summary strings."
:group 'orgnv-faces)
(defun orgnv-build-database (&optional pattern context directories)
"Scan the notes in the orgnv-directories, creating a temporary
buffer with the output of the command. PATTERN, CONTEXT,
DIRECTORIES default orgnv-title-pattern, orgnv-context-size and
orgnv-directories."
(with-temp-buffer
(delete-region (point-min) (point-max))
(setq context (or context orgnv-context-size)
pattern (or pattern orgnv-title-pattern))
(let ((grep-args `(,@(unless (zerop context)
(list "-A" (format "%s" context)))
;;"-s" ; Suspend errors
"-E" ; Extended syntax
"-Z" ; Null separated names
"-m" "1" ; One result
,@(mapcar (lambda (extension)
(format "--include=*.%s" extension))
orgnv-extensions)
,(if orgnv-recursive
"--directories=recurse"
"--directories=skip")
,(concat "^" pattern)
,@(mapcar (lambda (d) (expand-file-name d))
(or directories orgnv-directories)))))
(message "Invoking %s with %S" orgnv-grep-command grep-args)
(apply 'call-process orgnv-grep-command nil t nil grep-args)
(orgnv--format pattern))))
(defun orgnv-compare-titles (record1 record2)
(string< (cadr record1) (cadr record2)))
(defun orgnv--format (pattern &optional sort-predicate)
"This function processes the current buffer, scanning for an
output from GREP in the form of
file-name null-character line-matching-title
file-name null-character summary-line-1
...
--
It returns a database in the form of an association list,
((filename-1 . (title-1 . description-1))
...)
This database will be sorted if SORT-PREDICATE is a function
that can compare pairs of records."
(goto-char (point-min))
(let ((file-pattern (concat "^\\([^\x0-]+\\)\x0[ ]*\\(" pattern "\\)?[ ]*\\([^\n]*\\)$"))
(records '())
lastfile
record
description)
(while (re-search-forward file-pattern nil t)
(let ((file (match-string 1))
(string (match-string 3)))
(cond ((not (equal file lastfile))
(setq lastfile file
description ""
record (cons string description)
records (cl-acons lastfile record records)))
((or (null string) (zerop (length string)))
;; Nothing in the description
)
((null description)
;; No more text fits into the description
)
(t
(setq description (concat description string " "))
(if (>= (length description) orgnv-description-limit)
(setf description (substring description 0
orgnv-description-limit)
(cdr record) description
description nil)
(setf (cdr record) description))))))
(if (setq sort-predicate (or sort-predicate orgnv-database-sort-predicate))
(sort records sort-predicate)
records)))
(defun orgnv--filter-database (pattern database &optional size-limit)
"Take an OrgNV database and filter the records that match
PATTERN. If SIZE-LIMIT is not nil, return a database with at most
SIZE-LIMIT elements."
(cond ((length pattern)
(let ((i 0)
(output nil))
(while (and database (or (null size-limit) (<= i orgnv-display-limit)))
(let ((record (pop database)))
(when (string-match pattern (cadr record))
(setq i (1+ i))
(push record output))))
(nreverse output)))
((null size-limit)
database)
(t
(seq-subseq database 0 size-limit))))
(defvar orgnv--database nil
"Database which is currently being interrogated")
(defvar orgnv--filtered-database nil
"Subset of elements in the database that are displayed")
(defvar orgnv--buffer nil
"Buffer where the database is displayed")
(defun orgnv--display-matches (buffer filtered-database &optional first)
(with-current-buffer buffer
(delete-region (point-min) (point-max))
(dolist (l filtered-database)
(insert (propertize (cadr l) 'face 'orgnv-title-face)
(propertize " -- " 'face 'orgnv-separator-face)
(propertize (cddr l) 'face 'orgnv-summary-face)
"\n"))
(when first
(toggle-truncate-lines +1)
(hl-line-mode +1))
(goto-char (point-min))
(hl-line-highlight)))
(defun orgnv-previous-line ()
(interactive)
(with-current-buffer orgnv--buffer
(hl-line-unhighlight)
(forward-line -1)
(hl-line-highlight)))
(defun orgnv-next-line ()
(interactive)
(with-current-buffer orgnv--buffer
(hl-line-unhighlight)
(forward-line +1)
(hl-line-highlight)))
(defvar orgnv--selection nil
"Record selected by the user or nil")
(defvar orgnv--next-action nil
"Actions to perform once the ORGNV buffer is closed.")
(defun orgnv--cleanup ()
"Quit the OrgNV and close its window."
(pop-to-buffer orgnv--buffer)
(quit-window))
(defun orgnv-completing-read (database)
"Prompt the user to select a record in the OrgNV. The user will
input text in the minibuffer. This text will be used to refine
queries on the database, with output displayed in
orgnv-buffer. The user can press M-p, M-n, <up> and <down> to
change the selected entries. Finally pressing <RET> implements
the selection."
(let ((orgnv--database database)
(orgnv--filtered-database database)
(orgnv--selection nil))
(with-temp-buffer
(let ((orgnv--buffer (current-buffer)))
(rename-buffer orgnv-buffer t)
(orgnv--update t)
(display-buffer orgnv--buffer)
(unwind-protect
(let ((map (orgnv--make-minibuffer-map)))
(read-from-minibuffer "Keywords: " nil map))
;; We cleanup our buffers both for natural causes (e.g. pressing
;; <enter>) or when aborting (pressing C-g)
(orgnv--cleanup))))
orgnv--selection))
(defun orgnv--make-minibuffer-map ()
(let ((map (make-sparse-keymap)))
(set-keymap-parent map minibuffer-local-map)
(define-key map "\t" 'minibuffer-complete)
;; Extend the filter string by default.
(setq i ?\s)
(while (< i 256)
(define-key map (vector i) 'orgnv--insert-and-update)
(setq i (1+ i)))
;; M-TAB is already abused for many other purposes, so we should find
;; another binding for it.
;; (define-key map "\e\t" 'minibuffer-force-complete)
(define-key map [?\M-p] 'orgnv-previous-line)
(define-key map [?\M-n] 'orgnv-next-line)
(define-key map [?\C-h] 'orgnv-help)
(define-key map (kbd "<up>") 'orgnv-previous-line)
(define-key map (kbd "<down>") 'orgnv-next-line)
(define-key map (kbd "<tab>") 'orgnv--update)
(define-key map [?\C-c ?\C-l] 'orgnv-link)
(define-key map (kbd "RET") 'orgnv-select)
map))
(defun orgnv-select ()
(interactive)
(orgnv--select (lambda (selection)
(switch-to-buffer (find-file-noselect (car selection)
nil nil nil)))))
(defun orgnv-link ()
(interactive)
(orgnv--select (lambda (selection)
(org-insert-link nil
(car selection)
(cadr selection)))))
(defun orgnv--select (the-action)
(let ((filter (minibuffer-contents)))
(with-current-buffer orgnv--buffer
(let* ((l (min (line-number-at-pos)
(length orgnv--filtered-database)))
(selection (elt orgnv--filtered-database (1- l))))
(push
(if selection
(lambda ()
(funcall the-action selection))
(lambda ()
(let ((new-note (orgnv-create-note filter)))
(when new-note
(funcall the-action new-note)))))
orgnv--next-action))))
(exit-minibuffer))
(defvar orgnv-help nil)
(defun orgnv-help-text ()
"OrgNV (Org Navigation)
----------------------
Enter keywords to refine search through the OrgNV database. Use the special
keys below to affect the selection of records, move through the database,
create notes, etc.
key binding
--- -------
C-g abort
C-h show / hide this help message
up / M-p move to previous record
down / M-n move to next record
<enter> select current record if it exits, or create new note
other keys continue extending the database query
")
(defun orgnv-help ()
(interactive)
(if orgnv-help
(orgnv--clear-help)
(with-current-buffer orgnv--buffer
(setq-local orgnv-help (point))
(delete-region (point-min) (point-max))
(insert (orgnv-help-text)))))
(defun orgnv--clear-help ()
(when orgnv-help
(with-current-buffer orgnv--buffer
(orgnv--update)
(goto-char orgnv-help)
(setq-local orgnv-help nil))))
(defun orgnv--insert-and-update ()
(interactive)
(let ((c (vector last-command-event))
f)
(cond ((setq f (lookup-key minibuffer-local-map c))
(funcall f 1))
((setq f (lookup-key global-map c))
(funcall f 1))
(t
(self-insert-command 1)))
(orgnv--update)))
(defun orgnv--update (&optional first-time)
(interactive)
(setq orgnv--filtered-database
(orgnv--filter-database (minibuffer-contents)
orgnv--database))
(orgnv--display-matches orgnv--buffer orgnv--filtered-database first-time))
(defun orgnv-create-note (tentative-title)
"Create a note, prompting the user for the title."
(let ((title (read-from-minibuffer "Note title: " tentative-title)))
(when title
(let* ((filenames (mapcar (lambda (dir)
(orgnv-make-file-name title dir))
orgnv-directories))
(filename (completing-read "Filename: " filenames
nil nil (car filenames))))
(when filename
(orgnv-create-template filename title)
(cons filename (cons title nil))
)))))
(defun orgnv-make-file-name (title dir)
(expand-file-name (concat title ".org") dir))
(defun orgnv-create-template (filename title)
(let ((buffer (find-file-noselect filename nil nil nil)))
(when buffer
(with-current-buffer buffer
(insert "#+TITLE: " title "\n\n"))
buffer)))
(defcustom orgnv-relative-links nil
"Control how to create org-mode links.
- nil means insert full paths
- t means paths relative to org-directory"
:type 'boolean
:group 'orgnv)
(defun orgnv-browse ()
(interactive)
(let* ((orgnv--next-action nil)
(record (orgnv-completing-read (or orgnv--database
(orgnv-build-database)))))
(dolist (action orgnv--next-action)
(funcall action))
record))
(defun orgnv--relative-path (file)
(if orgnv-relative-links
(file-relative-name file org-directory)
file))
;; `bind-key' function is missing from my Emacs
;;(bind-key (kbd "C-x g") 'orgnv-browse org-mode-map)
(provide 'orgnv)
Tested on Windows with 27.2 (grep provided by MSYS2).
It seems to be case-sensitive for “#+TITLE” but otherwise it works… Don’t know how to make grep case-insensitive, though.
I only have two notes in the directory. I wonder how it fairs with more notes.
Created 1,000 test notes with using the test data generator the author also provides in the website.
There can be some usability improvement for my taste, but the performance for the 1K notes is good – no lag experienced.
Yeah, it can be useful for some folks.
Hey, you’re amazing for looking at it, thanks!
Even with your version, though, I’m still getting an empty results buffer, and I noticed it’s re-building the database every time I’m running orgnv-browse, like it’s not assigning it to the orgnv–database var, which is indeed nil…
Do you have any idea what I might be missing?
Thanks again!
It might be that your org files are not meeting the requirements. You can easily test this hypthesis.
- Create a file with
#+TITLE:with upper case, not#+title:; I think you need to have at least one file with TITLE - Check
grepworks with the following Elisp code (evaluate in Emacs, in scrath buffer for instance – change the path to your org file - Pay attention to the trailing “/” in the definition of the path
orgnv-directories– it may be relevant; I have this – it must be a list, which was one of the fixes:("/home/nobiot/org")– no trailing slash - You can also try the same in Terminal with grep with the same option
(let ((grep-args (list "-A" "3" "-E" "-Z" "-m" "1" "--include=*.org" "--directories=recurse" "^#\\+TITLE\\:" "/home/your-user/your-org")))
(apply 'call-process orgnv-grep-command nil t nil grep-args))
It seems to have different behavior between Linux (Ubuntu) and Windows. If you are on macOS, it may be even different. I can’t help you with macOS, but perhaps you can identify the issue with a bit of edebug.
Also… I’m using bash – no other shell; not sure if it is relevant. The code uses the external program, so the cross-platform support might be more difficult – I’m just guessing here. I didn’t fail between my Windows and Ubuntu (except for the case-sensitivity of TITLE).
I’m running on Ubuntu, and I’ve already verified that the grep invocation works… It may something to with orgnv–format? I’m not so sure how to work with edebug yet… I’ve tried evaluating
(with-temp-buffer (let ((grep-args (list "-A" "3" "-E" "-Z" "-m" "1" "--include=*.org" "^#\\+TITLE\\:" "/home/jonathan/test.org")))
(apply 'call-process orgnv-grep-command nil t nil grep-args)
(setq test-db (orgnv--format "word_in_buffer"))))
and I get test-db = nil, but when I eval the whole thing without the temp-buffer, I do get the test note as output…
I’ve also tried evaluating
(orgnv-completing-read (or orgnv--database
(orgnv-build-database)))
which should create the database and assign it to orgnv–database, but it’s still nil after that
I downloaded the source from this link in the website:
Download the source code here
I only changed the following – see the diff below.
Since I’ve changed defcustom, you might need to restart Emacs to get the new value effective (or use setq.
Since I put test files in org-directory, I added no customizing. And it just works…
diff --git a/orgnv.el b/orgnv.el
index 14d887a..43177ab 100644
--- a/orgnv.el
+++ b/orgnv.el
@@ -40,8 +40,8 @@
"Emacs OrgNV mode."
:group 'local)
-(defcustom orgnv-directories (or (expand-file-name org-directory)
- (expand-file-name "~/.orgnv/"))
+(defcustom orgnv-directories (list (or (expand-file-name org-directory)
+ (expand-file-name "~/.orgnv/")))
"List of directories where OrgNV looks for notes."
:type 'directory
:safe 'stringp
@@ -141,7 +141,7 @@ orgnv-directories."
"--directories=recurse"
"--directories=skip")
,(concat "^" pattern)
- ,@(mapcar (lambda (d) (expand-file-name "*" d))
+ ,@(mapcar (lambda (d) (expand-file-name d))
(or directories orgnv-directories)))))
(message "Invoking %s with %S" orgnv-grep-command grep-args)
(apply 'call-process orgnv-grep-command nil t nil grep-args)
@@ -177,7 +177,7 @@ that can compare pairs of records."
(setq lastfile file
description ""
record (cons string description)
- records (acons lastfile record records)))
+ records (cl-acons lastfile record records)))
((or (null string) (zerop (length string)))
;; Nothing in the description
)
@@ -435,6 +435,7 @@ other keys continue extending the database query
(file-relative-name file org-directory)
file))
-(bind-key (kbd "C-x g") 'orgnv-browse org-mode-map)
+;; `bind-key' function is missing from my Emacs
+;;(bind-key (kbd "C-x g") 'orgnv-browse org-mode-map)
(provide 'orgnv)
I think this argument should be a directory, not a file.
So… "/home/jonathan" – note the trailing “/” is not there, either.
So part of the problem was the regexp matching file names in orgnv--format, it didn’t allow for more than one hyphen in the name of the file.
Now I get the list of files, but I can’t filter them by their content (description) - I’ve printed out the record it looks at in orgnv--filter-database (as an %s) and looks like it contains the entire description, so something about string-match doesn’t work? I don’t understand why. I’m not sure what’s the actual type of a record, and what cadr does…
Another thing is that it rebuilds the database on every invocation - that takes a couple seconds, and from the code it seem like it should keep it built in that variable orgnv--database which is always nil.
Thanks again for helping out!
edit:
this is a printout of template and record:
matching “network” to record: (/home/jonathan/test/1638092580-2020-07-28-functional_connectome.org functional connectome . is the network of regions in the brain that are functionally connected * sources cite:turk19_funct_connec_fetal_brain )
This is as far as I can help you with orgNV.
I don’t know how grep really works.
I suggest you contact the author directly; you might work with him to improve it to your liking, perhaps.
Thanks.
I think at this point grep is working as the record contains what I expect it to, it’s just the elisp pattern matching which doesn’t work for me… can you maybe tell me how to find out what’s the type of record, when I try to break on it, it looks like it’s breaking, but just continues on with the loop.
And can you tell me if your orgnv--database is set at all?
Thanks again for your help
The “type” of record? It’s a cons cell. It looks like this (first one of the two records):
See the result in the minibuffer. I think car is the title. In this record, description is empty with value of "".
orgnv--database is set, and it looks like this when I see the list returned. There are two records in the database. Yes, the text content is part of the database.
Thanks!