Roam-sync - Bash Script

Meta
1

Background

I run roam on three machines simultaneously. I faced difficulty in maintaining a common version. Cloud solutions did not work for me - resulting in frequent conflicts. One solution is to create one folder in network and mount it - but problem here is a line of sight is needed with the network drive all the time.

My solution

Step 0 : Generate timestamps for roam sync - establish recency
Step 1: use rsync over ssh to sync : depending on recency

I have provided a solution to generate timestamps in org roam db optimisation

the rest is simple - convert lisp timestamp to unix time stamp - compare - provide choice

#!/bin/bash

# Remote server details
remote_server="akash@192.168.0.109"

# Timestamp files
local_timestamp_file="/home/akash/.emacs.d/org-roam-db-last-update-time"
remote_timestamp_file="/home/akash/.emacs.d/org-roam-db-last-update-time"

# Sync folders
local_sync_folder="/home/akash/roam/"
remote_sync_folder="/home/akash/roam/"

# Check if the local timestamp file exists and is readable
if [ ! -f "$local_timestamp_file" ]; then
    echo "Local timestamp file not found or not readable: $local_timestamp_file"
    exit 1
fi

# Function to convert Lisp timestamp to Unix timestamp
convert_lisp_timestamp() {
    local lisp_timestamp="$1"
    local high low usec psec total_seconds

    # Remove parentheses
    lisp_timestamp=${lisp_timestamp#\(}
    lisp_timestamp=${lisp_timestamp%\)}

    # Split the timestamp into HIGH, LOW, USEC, and PSEC
    IFS=' ' read -r high low usec psec <<< "$lisp_timestamp"

    # Calculate total seconds from HIGH and LOW
    total_seconds=$(( high * 65536 + low ))

    # Convert microseconds and picoseconds to seconds and add to total seconds
    total_seconds=$(echo "scale=6; $total_seconds + $usec / 1000000.0 + $psec / 1000000000.0" | bc)

    # Round total seconds to nearest integer
    total_seconds=$(printf "%.0f" "$total_seconds")

    echo "$total_seconds"
}

# Function to convert Unix timestamp to human-readable format
unix_to_human_readable() {
    local unix_timestamp="$1"
    date -d "@$unix_timestamp" "+%Y-%m-%d %H:%M:%S"
}

# Get the modification time of the local timestamp file
local_lisp_timestamp=$(< "$local_timestamp_file")

# Convert local Lisp timestamp to Unix timestamp
local_unix_timestamp=$(convert_lisp_timestamp "$local_lisp_timestamp")

# Get the modification time of the remote timestamp file via SSH
remote_lisp_timestamp=$(ssh "$remote_server" "cat '$remote_timestamp_file'")
remote_unix_timestamp=$(convert_lisp_timestamp "$remote_lisp_timestamp")

# Convert timestamps to human-readable format
local_human_readable=$(unix_to_human_readable "$local_unix_timestamp")
remote_human_readable=$(unix_to_human_readable "$remote_unix_timestamp")

# Display modification times
echo ""
echo "Remote Server: $remote_server"
echo "Remote Modification Time: $remote_human_readable"
echo "Local Modification Time: $local_human_readable"
echo ""

# Indicate which version is newer
if [ "$local_unix_timestamp" -gt "$remote_unix_timestamp" ]; then
    echo -e "\033[1mLocal version is newer.\033[0m"
elif [ "$local_unix_timestamp" -lt "$remote_unix_timestamp" ]; then
    echo -e "\033[1mRemote version is newer.\033[0m"
else
    echo "Both files have the same modification time."
fi

# Prompt user for action
echo ""
echo "Options:"
echo "1. Proceed to update old to new."
echo "2. Revert change (revert new to old)."
echo "3. No action."
echo ""
read -p "Choose an option (1/2/3): " choice


case "$choice" in
    1)
        if [ "$local_unix_timestamp" -gt "$remote_unix_timestamp" ]; then
            # Local file is newer, perform rsync from local to remote
            echo "Local file is newer. Performing rsync from local to remote..."
            rsync -av --delete "$local_sync_folder" "$remote_server:$remote_sync_folder"
            # Update remote timestamp file after syncing
            ssh "$remote_server" "echo '$local_lisp_timestamp' > '$remote_timestamp_file'"
        elif [ "$local_unix_timestamp" -lt "$remote_unix_timestamp" ]; then
            # Remote file is newer, perform rsync from remote to local
            echo "Remote file is newer. Performing rsync from remote to local..."
            rsync -av --delete "$remote_server:$remote_sync_folder" "$local_sync_folder"
            # Update local timestamp file after syncing
            echo "$remote_lisp_timestamp" > "$local_timestamp_file"
        else
            echo "Both files have the same modification time. No action needed."
        fi
        ;;
    2)
        if [ "$local_unix_timestamp" -gt "$remote_unix_timestamp" ]; then
            # Local file is newer, perform rsync from remote to local (revert change)
            echo "Local file is newer. Performing rsync from remote to local (revert change)..."
            rsync -av --delete "$remote_server:$remote_sync_folder" "$local_sync_folder"
            # Update local timestamp file after syncing
            echo "$remote_lisp_timestamp" > "$local_timestamp_file"
        elif [ "$local_unix_timestamp" -lt "$remote_unix_timestamp" ]; then
            # Remote file is newer, perform rsync from local to remote (revert change)
            echo "Remote file is newer. Performing rsync from local to remote (revert change)..."
            rsync -av --delete "$local_sync_folder" "$remote_server:$remote_sync_folder"
            # Update remote timestamp file after syncing
            ssh "$remote_server" "echo '$local_lisp_timestamp' > '$remote_timestamp_file'"
        else
            echo "Both files have the same modification time. No action needed."
        fi
        ;;
    3)
        echo "No action taken."
        ;;
    *)
        echo "Invalid option. No action taken."
        ;;
esac

So the schematic looks like this

Machine 0 (192.168.0.109) is “server”
Clients sync with Machine 0 to maintain common version.

Current Limitation

1. Cannot be non trivially changed to sync between clients directly as size of clients increases - one "server" is needed for properly managing conflicts.
1 Like
2

Akash via Org-roam notifications@orgroam.discoursemail.com
writes:

test

3

My solution to this is using git with a bunch of hooks. Autocommit hook on buffer save:

(add-hook 'org-mode-hook 'git-auto-commit-mode)
(defun my-auto-commit-message (filename)
  "Specify that my commit is a work in progress"
  (concat "braindb connect from " (system-name) ". file: " (gac-relative-file-name filename)))

(with-eval-after-load 'git-auto-commit-mode
  (setq gac-default-message #'my-auto-commit-message
                                        ;gac-ask-for-summary-p t
        gac-automatically-push-p t
        gac-debounce-interval 30)
  )

Hook to push every commit:

#!/bin/sh
#
# An example hook script to verify what is about to be committed.
# Called by "git commit" with no arguments.  The hook should
# exit with non-zero status after issuing an appropriate message if
# it wants to stop the commit.
#
# To enable this hook, rename this file to "pre-commit".

git push >&- 2>&- &

Then systemd timers:

# sync.timer

[Unit]
Description=Sync BrainDB Timer

[Timer]
OnBootSec=1min
OnUnitActiveSec=1min
Unit=sync-brain.service

[Install]
WantedBy=timers.target

# sync.service
[Unit]
Description=Pull new notes from BrainDB Git Repo

[Service]
Type=oneshot
ExecStart=/usr/bin/git -C /home/iamkarlson/braindb pull

[Install]
WantedBy=default.target

It relies on the assumption that your org files would be in the same location across all systems but besides this trade-off, it works like a charm!

1 Like