[IDEA] Sharing org files on a p2p network

Meta
1

[IDEA] Sharing org files on a p2p network

Org version of this post

Community database of interlinked org documents on the Hypercore peer-to-peer network.

File structure

Here's a mock discussion to illustrate how shared files might be organized: 

/home/alice/shared-org-files
├── alice-files
│   └── combining-org-mode-and-p2p.org
└── bob-files
    └── online-deliberation.org

In the above example, we are looking at Alice's computer, which contains a shared-org-files directory inside the home directory. The shared-org-files directory contains one subdirectory per peer. Alice has added combining-org-mode-and-p2p.org inside of her designated writable directory alice-files. Files inside alice-files are shared on the network, while bob-files contains Bob's shared files that were synced from the network to Alice's machine.

Link syntax

For initial testing, we've been using org-id links with a special org-id-method that prefixes each ID with the author's public key. When you follow one of these links, the package should either jump to that file+heading or attempt to fetch that peer's dataset from the network.

It might make more sense to create a new org link type to handle hyper:// links, but for now org-id links allow us to seamlessly integrate with org-roam.

I would be very grateful for feedback on this: Does it make sense to stick with org-id links? If we create a new link type, how would we need to extend org-roam so that it considers backlinks which are not of org-id links?

In the following examples, note that the each :ID: is prefixed with the public key of the author, here abbreviated as "alice1234" or "bob1234":

  1. Alice shares combining-org-mode-and-p2p.org:
:PROPERTIES:
:ID:       hyper://alice1234/4edb7a91-46e9-402a-8a30-caee62b16f6b
:END:
#+title: Combining org mode and p2p

* Let's created a shared community network of org documents!
:PROPERTIES:
:ID:       hyper://alice1234/1e8ee2b7-43d1-44f3-9659-01d9d24774f3
:END:
  1. Bob shares online-deliberation.org, linking back to Alice's post:
:PROPERTIES:
:ID:       hyper://bob1234/cc398998-efd6-45c8-8c7b-a64c7accd977
:END:
#+title: Online deliberation

* I would use [[id:hyper://alice1234/1e8ee2b7-43d1-44f3-9659-01d9d24774f3][this tool]] to deliberate important issues.
:PROPERTIES:
:ID:       hyper://bob1234/3fdbf5f4-3e5d-4802-a5e6-9f34d0afac0b
:END:

Queries

Each peer creates a sqlite cache for just their own dataset. Caches are shared with peers along with the org files. To query databases from peers as well as your own, attach and join the dbs, allowing for backlinks between peer databases.

Trust network

In addition to the org files and sqlite cache, users also share a .json (or perhaps something like YAML) file containing a series of trust relationships between peers. Taking inspiration from TrustNet, we can use this data for community curation while retaining user control of their sources of information.

This trust network allows for queries like "show me all of the backlinks to the heading at point which were published by peers I trust for the topic of 'Emacs'".

Please let me know what you think of this idea! I am grateful for feedback.

2

Hello! My name is Joseph Turner. I’m part of Ushin, a small organization that’s currently developing the idea described in my last post. Our team met today, and we came up with a design for syncing org files between devices, using hyper-sdk, hyperdrive, and mirror-drive .

Once the network layer is functional, we’ll need some help on the Elisp side of development. We want to extend org-roam to be able to query multiple sqlite DBs in parallel with attach. We also want to add an interface for viewing and manipulating trust data.

We are looking for people who are familiar with org-roam and sqlite to help with the project. Familiarity with Hypercore or p2p networks is not necessary.

Please send an email to contact@ushin.org if you are interested!

Thank you!

Joseph

3

I updated the original post:

  • Add a link to the Hypercore website.
  • Update the examples to demonstrate linking to headings by ID.
  • Remove the custom REPLY_TO property in Carol’s response. Instead, just link back to Bob’s heading the #+TITLE.
  • Remove the custom “reply view”.
  • Describe querying multiple dbs.
  • Remove restriction on edits after publication.
  • Remove description of sparse replication, since it’s not part of the current design.
  • Describe trust network.
1 Like
4

I updated the original post:

  • Simplify example by removing Carol’s reply
  • Create new Link syntax subheading

Most importantly, I added a question about integrating a new link type with org-roam: