Private AI Bot

People use custom fine tuned AI agents because:

1. can use custom instructions / context / prompts
2. can use custom context history
3. can use specific files for data (Retrieval Augmented Generation or RAG)

However, we want to keep these private. Use cases include:

1. creating your own AI bot with private data and not wanting to remember your API access keys
2. creating custom AI bots for others but not wanting to share OpenAI API keys, thread IDs, agent IDs, etc.

We use LIT Protocol to achieve this by locking API keys to AI models so that it's processed within a LIT action. User doesn't have the API key, it doesn't get exposed to anyone, but instead it decrypts the API key based on LIT access restrictions and then uses the decrypted key to make the fetch from a LIT node to the AI agent provider. The response is then shuttled back to the user. The code to complete these actions are stored on IPFS via Fleek and Pinata, tho of course the API keys are variables and thus not exposed publicly ;)

Here we used GPT2 from HuggingFace, but we recommend using OpenAI ChatGPT 4 with RAG enabled and leveraging a custom AI agent and thread (to keep context of prior conversations). We just used HF GPT2 because it's free and unlimited ;)

This is still very prototype so hence doesn't have the agent ID, thread ID, etc or chatbot UI but those are solvable Web2 problems :)

Video: https://github.com/financegeek-org/hackfs/raw/main/assets/demo_video.mkv Slides: https://docs.google.com/presentation/d/1sck-nCUy8ZUs7bS0s9QOrnLe9BlM16ydXkPfgS0LLX8/edit?usp=sharing Codebase: https://github.com/financegeek-org/hackfs/

Bonus: PR for fixing a problem with LIT protocol's sample code: https://github.com/LIT-Protocol/developer-guides-code/pull/17

We started with LIT Protocol's guide. However, because the signing/decryption within LIT actions is new (and similar in nature to their older signing/decryption), it was confusing. Also the sample code didn't work, so had to dig into their SDK source code to figure out what the right inputs are. I started a PR with those fixes to sample code here https://github.com/LIT-Protocol/developer-guides-code/pull/17

That way, future people won't run into the same issues as me :)

At first we wanted to use OpenAI ChatGPT since that has the RAG and thread features to demo the power of Lit Actions (locking customized chatbot context behind wallets), but we ran out of OpenAI credits. So we used Hugging Face GPT2 instead, with the note that switching over is pretty easy (using OpenAI SDK or making custom fetches).

We ran into some issues where it was giving an error within the nodes, but that got solved after we dug deeper into the error messages. LIT Protocol can get a bit overwhelming at times, and their guides could use some work, but this was fun overall :)