Building RAG and incorporating your notes

Has anyone build RAG-system incorporating their notes in it?

I’m thinking of two main use cases for this:

  1. Semantically searching through all your notes on specific topics.

  2. Taking a web page, pdf document, etc., analyzing the structure, building a local knowledge graph creating either a single document with interlinked nodes or a series of .org files, the process where nodes automatically get linked to existing Org-Roam nodes

Has anyone done/experimented with anything like that?

For the first case, Khoj project covers some of it, although, in my experience it doesn’t do it well, often either completely missing relevant documents or bringing stuff that’s absolutely irrelevant for the given query. I dunno, YMMV, maybe I just have not configured the tool properly.

There was also Luis Moneda’s post from a while ago how to home brew something like that, I tried to reproduce it, it took me some time (darn Python dependencies are nuts), but it also turned out to be not very usable in practice. Perhaps someone knows updated technique that uses modern LangChain classes and whatnot.

It would be very cool to have something like this. This could entirely change the game of “capturing” relevant data, right?

2 Likes

I love this idea. Does anyone have any idea how to do it well?