[GH-ISSUE #23] [Feature request] ChromaDB vector database #20

New issue

Closed

opened 2026-10-15 18:03:28 +00:00 by bsakaguchi · 3 comments

bsakaguchi commented

2026-10-15 18:03:28 +00:00

Owner

Originally created by @maceter on GitHub (May 15, 2023).
Original GitHub issue: https://github.com/SillyTavern/SillyTavern-Extras/issues/23

Add support for chromadb to manage inlinity context sizes.

It's already implemented in oobabooga - https://github.com/oobabooga/text-generation-webui/tree/main/extensions/superbooga
I'll try to make a pull request for this in a week, but if you have some suggestions for api design, please tell.

For now I want to make these endpoints:

/api/chroma/add-message - add message to db
- Body:
  - id - some id of message, to sort them chronologically
  - chat_id - id of chat to store
  - content - text of message
  - role - user/system/bot
/api/chroma/query - query relevant messages
- Body:
  - query - for example, latest user's message
  - chat_id - id of chat to query
- Answer:
  - Array of id, chat_id, content and role

And of course this all should be later connected to SillyTavern. There is a very high demand of this feature in ai roleplay communities.

Originally created by @maceter on GitHub (May 15, 2023). Original GitHub issue: https://github.com/SillyTavern/SillyTavern-Extras/issues/23 Add support for `chromadb` to manage inlinity context sizes. It's already implemented in oobabooga - https://github.com/oobabooga/text-generation-webui/tree/main/extensions/superbooga I'll try to make a pull request for this in a week, but if you have some suggestions for api design, please tell. For now I want to make these endpoints: - `/api/chroma/add-message` - add message to db - Body: - `id` - some id of message, to sort them chronologically - `chat_id` - id of chat to store - `content` - text of message - `role` - user/system/bot - `/api/chroma/query` - query relevant messages - Body: - ` query` - for example, latest user's message - `chat_id` - id of chat to query - Answer: - Array of `id`, `chat_id`, `content` and `role` And of course this all should be later connected to SillyTavern. There is a very high demand of this feature in ai roleplay communities.

bsakaguchi closed this issue

2026-10-15 18:03:28 +00:00

bsakaguchi commented

2026-10-15 18:03:29 +00:00

Author

Owner

@Cohee1207 commented on GitHub (May 15, 2023):

Looks like an exceptionally good idea, I had this in mind for quite some time already. API structure looks good to me.

ChromaDB also has a node.js port, which is actually better for data persistence, because Extras do not have any way to save data across sessions, this might be a downside for those who use extras on colab. So I guess you could add these straight to SillyTavern's server.js.

@Cohee1207 commented on GitHub (May 15, 2023): Looks like an exceptionally good idea, I had this in mind for quite some time already. API structure looks good to me. ChromaDB also has a node.js port, which is actually better for data persistence, because Extras do not have any way to save data across sessions, this might be a downside for those who use extras on colab. So I guess you could add these straight to SillyTavern's server.js.

bsakaguchi commented

2026-10-15 18:03:29 +00:00

Author

Owner

@maceter commented on GitHub (May 15, 2023):

Node.JS version requires additional server written in python anyway.

About persistence - just send whole chat history when chat is loaded. It'll take a while to process all data, though.
But it's easier to sync actual chats with db this way.

@maceter commented on GitHub (May 15, 2023): Node.JS version requires additional server written in python anyway. About persistence - just send whole chat history when chat is loaded. It'll take a while to process all data, though. But it's easier to sync actual chats with db this way.

bsakaguchi commented

2026-10-15 18:03:29 +00:00

Author

Owner

@Cohee1207 commented on GitHub (May 15, 2023):

Node.JS version requires additional server written in python anyway.

Didn't know that. Ok, that makes sense.

@Cohee1207 commented on GitHub (May 15, 2023): > Node.JS version requires additional server written in python anyway. Didn't know that. Ok, that makes sense.