[[Offtopic]] How do you guys manage your external packages?

I have thought of migrating to use-package: but i dont really dig declarative style of doing configuration, it just feels wrong to me somehow, I created a small package that generates the autoloads of a package, a configuration file and automatically adds to 'load-path the package. All that is required is to add a line calling the configuration file in the emacs init file. I have been able to create seperated modules like this that are very easy to manage.

A list is maintained in plain text that takes cares of adding those packages to load-path, the directory ~/.emacs.d/custom_packages/ is hardcoded because I havent found the necessity to generalise it.

Please provide feedback - or is it really foolish ?

;; packagesv2.el

(defun generate-autoloads-and-create-init-file ()
  "Generate autoloads, create an init file for Elisp files in the current buffer's directory.
   and add package to 'load-path"
  (interactive)
  (let* ((buffer-file (or buffer-file-name (dired-get-file-for-visit) (buffer-file-name (other-buffer))))
	 (directory (file-name-directory buffer-file))
	 (base-name (file-name-sans-extension (file-name-nondirectory buffer-file)))
	 (autoloads-file (concat "autoloads_" base-name ".el"))
	 (placeholder "%s")
	 (additional-content "
;; pre-init

;; put pre-init contents here

;;; pre-init ends here

;; post-init
(with-eval-after-load '%s

;; put post-init contents here

) ;;; post-init ends here

(provide 'init_%s)")
	 (resolved-content (replace-regexp-in-string (regexp-quote placeholder) base-name additional-content))
	 (init-content (format ";; Package initialization for %s\n(require 'autoloads_%s)%s"
			       base-name (replace-regexp-in-string " " "-" base-name) resolved-content))
	 (init-file (concat "init_" (replace-regexp-in-string " " "-" base-name) ".el")))
    
    ;; Generate autoloads
    (make-directory-autoloads directory autoloads-file)
    
    ;; Check if init file exists
    (unless (file-exists-p init-file)
      ;; Create init file
      (with-temp-buffer
	(insert init-content)
	(write-region (point-min) (point-max) init-file)
	(message "Autoloads generated and saved to %s. Init file %s created with additional content."
		 autoloads-file init-file)))
    
    ;; Append package name to the packages-list file
    (append-to-packages-list base-name)
    
    )    ; let* scope ends here
  )      ; func defun ends here

(defun append-to-packages-list (base-name)
  "Appends the package name to the packages-list file if it doesn't exist."
  (let* ((packages-list "~/.emacs.d/custom_packages/packages-list")
         (packages-list-entries (if (file-exists-p packages-list)
                                   packages-list
                                 (make-temp-file "packages-list" nil "packagesv2/"))))
    (unless (member base-name (with-temp-buffer
                                 (insert-file-contents packages-list-entries)
                                 (split-string (buffer-string) "\n")))
      (with-temp-buffer
        (insert (mapconcat 'identity (append (with-temp-buffer
                                               (insert-file-contents packages-list-entries)
                                               (split-string (buffer-string) "\n"))
                                             (list base-name)) "\n"))
        (write-region (point-min) (point-max) packages-list-entries)))))


;; The following ensures that entries specified in packages-list is added to load-path
;; --------------------------------------------------

;; helper function to load package to load-path from custom_packages dir
(defun load-package (package)
  "Add a package to the load-path and load its directory."
  (let ((package-directory (concat "~/.emacs.d/custom_packages/" package)))
    (add-to-list 'load-path package-directory)))

;; Read the packages from the file packages_list and add them to load-path
(let* ((packages-file "~/.emacs.d/custom_packages/packages-list")
       (packages (with-temp-buffer
                    (insert-file-contents packages-file)
                    (split-string (buffer-string) "\n" t))))
  (mapc 'load-package packages))

(provide 'packagesv2.el)

I don’t find what you do foolish, but what are “external packages” in your definition? If they are from a repository like ELPA and MELPA, package-install (installing from package-list-packages) does it all for you.

For my own code or someone’s small script, I create an .el file, add provide so they become a library (or feature), and put the file in ~/.config/emacs/lisp.

For autoloading functions in these libraries, I just add ;;;###autoloads cookie and do the following. It also native compiles the el file.

loaddefs-generate scrapes the el files, find the autoloads cookies and generates my-autoload.el file…

;;;###autoload
(defun my/auto-native-compile-inits ()
  (interactive)
  (dolist (f (directory-files (expand-file-name "lisp" user-emacs-directory)
                              'FULL "^[ini*|my*].*.el$"))
    (byte-compile-file f)
    (when (native-comp-available-p) (native-compile f)))
  (loaddefs-generate (expand-file-name "lisp" user-emacs-directory)
                     (expand-file-name "my-autoloads.el"
                                       (expand-file-name "lisp" user-emacs-directory))
                     '("my-org-functions.el")))
1 Like

Holiday greetings and a happy New year, couldn’t reply earlier because I was bogged down by some personal tasks today.

By external packages I mean those packages that I do not download from Melpa/Elpa. In my case, I only maintain a core list of packages from the repos, and I do not like to use the package-install to install packages mainly because it generates some entries automatically in my init file, I delete those automatically generated entries that are appended to the init file - whether it is generated due to some customisations or package installs, I am very particular about keeping the init file clean (and manual) :smile:

But mainly the packages downloaded from the repos are those that either have a long list of dependencies or those that I downloaded while I was very new to using Emacs, maintaining packages manually in my case have provided me with greater control over what is included and not, sometimes a lot of packages have lots of files that are unnecessary and i prune those packages to only keep the core files.

For example in my init file - I only have these packages downloaded from the repos

(setq package-selected-packages '(
				  org-roam-ui             ; visualise relationships between nodes of org-roam inside a browser
				  org-roam-bibtex         ; integrate bibliography system with org-roam
				  org-ref                 ; implement citation style of the form 'cite:&' and more
				  helm-bibtex             ; bibtex backend and helm frontend for bibliography system
				  consult-org-roam        ; integrate org-roam and consult
				  org-roam                ; a database based interliked note keeping system to implement a zettlekasten inside Emacs leveraging org files
				  pdf-tools               ; substitute DocView pdf renderer with PdfView
				  consult                 ; tool set for search and navigation inside Emacs
				  orderless               ; regex completions style for case-insensitive, disordered minibuffer inputs
				  vertico                 ; vertico completions
				  which-key               ; a key-map helper
				  ))

while reinstalling Emacs - I only need to do packages-install-selected-packages

while I manually maintain the following packages

org-mind-map
org-remark
delve
org-noter
dired-hacks
org-download

using git - I can fork them locally and do many manual tweaking.
But most importantly it allows me to patch them easily with various patches I have devised for my need.

I also have many small packages that I have created that i have not listed here but they integrate very well with this system too.

I will put to use the code you provided, to byte compile some of my packages. I had not earlier byte compiled them because the autoloads generated from these packages already reduce the initialisation time by a lot, since these packages are not loaded fully but only the autoload files.

Have a very prosperous new year,
Best.