Org attach with all attachments in one flat folder

How To
1

I would like to have all my attachments in one folder without sub-directories, unlike what org-attach does by default.

I know I could just use file: links to a single folder rather than attachment: links, but then I might not get some benefits of org-attach? Please correct me if I am wrong and it is actually the same.

Actually one benefit is then I could give just file name instead of also file directory.

Although maybe I do need the links because it seems an attachment is only attached to an individual Org file. When I want all attachments/files to be able to be used by every Org file.

2

Relevant variables are

  1. org-attach-id-dir and
  2. org-attach-id-to-path-function-list

The first is responsible for the root directory containing the attachments
The second creates subdirectories according to the functions in here

List of functions used to derive attachment path from an ID string.
The functions are called with a single ID argument until the return
value is an existing folder.  If no folder has been created yet for
the given ID, then the first non-nil value defines the attachment
dir to be created.

Usually, the ID format passed to the functions is defined by
ā€˜org-id-methodā€™.  It is advised that the first function in the list do
not generate all the attachment dirs inside the same parent dir.  Some
file systems may have performance issues in such scenario.

Care should be taken when customizing this variable.  Previously
created attachment folders might not be correctly mapped upon removing
functions from the list.  Then, Org will not be able to detect the
existing attachments.

If you do not care about attachments previously stored and want to start afresh you may as well create your own function and set it here exclusively.

To store all of them in the same directory, probably youā€™d hardcode the first variable and create a dummy function to override all the functions in the second variable.

Cannot test right now, so cannot give code output, but hopefully it makes sense to you.

3

Thank you, that is helpful.

Would there be a difference between using :file links vs attachments in this case?

Example, would the same file then be able to be attached to multiple Org files, rather than having separate attachments for each Org file?

Does org-attach offer any additional functionality over file links?

4

It all depends on your use case, I think once you understand your requirement you may choose one of the many possible paths available.

For me, org-attach makes sense because I take print out of my notes often, and file: links would show up in the printed text that I want to avoid, I generally attach pdfs and other documents, that are specific to that file only.

If you want to use a common resource pool, it might be better to create your own functionality, lets say a folder and a function that open dired in that folder only. That is a much simple solution.

5

My ideal use case is for as many filenames on machine as possible to be UUIDs, and to be able to link all files the same way I currently link Org-roam nodes. Because this model of organization work very well for me.

I will do this either by each non-Org file having a corresponding Org file holding metadata (in properties draw, as well as title and tags) and any relevant text content to the file or links. Or by having a central node where all metadata for all non-Org files is stored. But I believe the first is superior option.

It will also be relatively easy to do this with other text formats example Markdown. Harder with non text.

But I believe this require coding a lot of custom functionality. So before I do that I want create common resource pool yes.

I see Denote did this. It has many features I like. But it does not use UUID as filename, and I want to stay in Org and Org-roam ecosystem.

What do you mean with dired? I thought dired was a way of navigating files not attaching them to Org files.

I also want to be able to display images inline in Org documents, so any custom solution needs to do that.

6

See if this is useful for you.

7

I saw that, but I want to have all files with UUIDs and in one directory, not directories with UUIDs.

Maybe I can edit the code to get what I want.

8 
(require 'emacsql)
(require 'emacsql-sqlite)

;; SECTION ONE

;; org-attach configuration
(defun custom/org-attach-id--dummy-fn (id)
  ;; Dummy Function
  (file-truename org-attach-id-dir))

(setq org-attach-id-dir "resources")
(setq org-attach-id-to-path-function-list '(custom/org-attach-id--dummy-fn))
;; HHHH------------------------------

;; Section 2

;; org-attach-db definitions
(defvar org-attach-db-file (expand-file-name "org-attachments.db" user-emacs-directory)
  "Path to the SQLite database for tracking Org attachments.")

(defvar org-attach-db-connection nil
  "Database connection handle.")

(defun org-attach-db-init ()
  "Initialize the database and create the table if needed."
  (unless (and org-attach-db-connection
               (emacsql-live-p org-attach-db-connection))
    (setq org-attach-db-connection
          (emacsql-sqlite-open org-attach-db-file)))
   (emacsql org-attach-db-connection
           "CREATE TABLE IF NOT EXISTS attachments (
                id TEXT PRIMARY KEY,
                filename TEXT,
                extension TEXT,
                path TEXT,
                size INTEGER,
                modified TEXT DEFAULT CURRENT_TIMESTAMP) ;"))

;; Use this function to sync db with file attachments
(defun org-attach-track-in-db (&optional attach-dir)
  "Track the newly attached file in the SQLite database."
  (interactive)
  (when-let* ((id (org-id-get))
              (attach-dir (or attach-dir
			   (org-attach-dir)))
	      ;; Ignore hidden files
              (files (directory-files attach-dir t "^[^.].*"))) 
    (dolist (file files)
      (when (file-regular-p file)
        (let* ((filename (file-name-nondirectory file))
	       (extension (file-name-extension file))
               (size (file-attribute-size (file-attributes file)))
               (path (expand-file-name file)))
          (org-attach-db-init) 
	  (emacsql org-attach-db-connection
           (format "INSERT OR REPLACE INTO attachments 
                    (id, filename, extension, path, size, modified) 
                    VALUES ('%s', '%s', '%s', '%s', %d, '%s');"
                   id filename extension path size (current-time))))))))

;; Section 3

;; Hooks
(add-hook 'org-attach-after-change-hook #'org-attach-track-in-db)

;; End
;; (provide 'org-attach-db)

Preliminary code.

set (setq org-attach-id-dir "resources") as per convenience to full path and not relative.

Note: We use id from where the attachment was made and refrain from creating new ids for each file.

1 Like
9

Thank you, will try this out!

10

I think there is a basic contradiction of terms here. For example, org-attach determines a files attachment from a given known folder structure for the file. If we use a flat folder, then we cannot create a /unique/ correspondence between the file and its attachees. Meaning, all files will correspond to all attachees.

This effectively makes it a either or situation.

We can either have unique ids corresponding to certain files and not filles.

Or all files being in a flat folder.

This constraint is handed to us down by org-attach. It is also not clear to me a path if exists to circumvent this. But even the exploration of such will increase the complexity up from a simple fix to a more elaborate rewriting of org-attach.

My hope was that a way couldā€™ve been elaborated in future time, but I have not been successful.

So you can either have one or the other in org-attach.

If you want to use IDS to link to files. But even then org-open would not be able to handle file links. We would have to use a file standing as the metadata, whose attachees would be the relevant link(s). But we have to have a definite folder-structure for each file and not flat.

Or a flat folder structure, but then all files will share the same universe.

11

Iā€™ll explain the motivation behind this question more.

What I want is to be able to reference other files the same way I reference Org-roam files (links, tags, more). Outlined in this question I asked before.

My basic idea: for non-Org, non-text based files, have a separate Org metadata file for each, where the UUID in metadata file is the filename of non-Org file so they can be identified. And then you can create linking of all files the same way as Org-roam files, they can have tags, they can have text referring to them, and more.

At that point, in theory, it also wouldnā€™t be necessary to have the files in a filesystem, but something more efficient (eg object storage). Because filesystems mainly give hierarchy, but this would be a completely non-hierarchical system. But Iā€™m getting ahead of myself here.

I thought a good first step would be to have all non-Org files in the same folder my Org files are in, give them sane titles, and then at some future date I could extract the titles, change the filenames to UUIDs, and create corresponding Org metadata files for them.

I saw Org-attach, and thought it could be good fit.

But I misunderstood how Org-attach works. It is better to use file links for this first step. Which I am now doing (in a flat folder).

I donā€™t think Org-attach offers any additional functionality apart from having unique attachments per Org file (like you say) but I want to be able to share ā€œattachmentsā€ across different Org files so that is actually what I donā€™t want.

Yes the way Org-attach currently works is to handle IDs by using folder structure. But I do want to have them all in the same ā€œuniverseā€. This is what I misunderstood.

Thank you for explaining, your explanation has really cleared it up for me.

12

Would you like to try this simple built-in alternative?

(require 'ffap)
(setopt ffap-next-regexp (concat org-uuid-regexp "\\|" ffap-next-regexp))

If you

  • Use a UUID as a prefix to a file name, like c04d3778-2787-4ab1-b42c-19f702d181a6.jpg.
  • Have ā€œattachment filesā€ in the same directory as notes files (flat directory structure).

See this.

  1. Put the UUID as is in the buffer (no need for the Org link syntax but can be inside one).

  2. Call command ffap.

  3. You should see a mini-buffer completion with the single file having the UUID as prefix.

A drawback of this approach is that FFAP shows only one file that has the prefix.

I have improved this approach to let me have multiple files that share the same prefix (I use date/time stamp as prefix), and preview the file with using Consultā€¦

1 Like
13

Please elaborate on the functions that need adjusting to allow for the modification, Or post code snippet. Please. Really cool solution. Will implement. :prayer_beads: :vulcan_salute:

Found it. You have a lot of interesting packages in your garage, should be publicized more.

14

Could you access my git.sr.ht/~nobiot? Good, because I cannot (I have been getting 403 Forbidden errors for the past week or so). It should be mono-at-point, which is my take of [do-at-point](https://elpa.gnu.org/packages/do-at-point.html).

mono-at-point is one of simple ideas and tiny libraries I have been slowly building over time. The idea is:

  1. I have a list of all files in all directories I am interested in (I started with ~/, which surprisingly worked well with 60,000 files and workable performance but there was too much noise ā€“ now down to about 2000 files.
  2. Use a regexp and the built-in thingatpt (Thing-at-Point) library to get Emacs to recognise my date/time-stamp prefix (ren-id) at point
  3. Filter my files (#1) by the prefix at point (#2)
  4. Use Consult as an improved library for completing-read and let me preview minibuffer candidates
  5. I borrowed the idea from ffap to implement mono-at-point, especially the way mono-highlight-overlay is used.

I donā€™t think you need to follow me but some quick suggestions; please do what you want to do.

  1. For #1, use your own list of files/directories.
  2. You can easily implement your own thing-at-point (#2).
  3. For #3, you can use my so (So ē¤Ž, meaning ā€œfoundationā€) library or make your own. Itā€™s simple but have been debugged on Linux and Windows, which I have been using everyday since 2024 ā€“ I just wanted to avoid repeating the same code for my own use.
  4. Preview with Consult is optional (#4) ā€“ it took some time to understand how Consult works. You may save time if you look at what I do.
  5. #5 is optional.
1 Like