[GH-ISSUE #97 ] Chromadb fix #63

Author

Owner

@Cohee1207 commented on GitHub (Aug 1, 2023):

@RBNXI
Why do your Extras interfere with Oobabooga install? How did you install extras?

@Cohee1207 commented on GitHub (Aug 1, 2023): @RBNXI Why do your Extras interfere with Oobabooga install? How did you install extras?

bsakaguchi commented

Author

Owner

@valden80 commented on GitHub (Aug 1, 2023):

I too getting error form chromaDB startup after the last update:

2023-08-01 12:39:16.825799: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-08-01 12:39:17.426976: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
Using torch device: cpu
Initializing an image captioning model...
Initializing a text summarization model...
Initializing a sentiment classification pipeline...
Initializing ChromaDB
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/AI/SillyTavern-extras/server.py:268 in <module>                                            │
│                                                                                                  │
│   265 │   posthog.capture = lambda *args, **kwargs: None                                         │
│   266 │   if args.chroma_host is None:                                                           │
│   267 │   │   if args.chroma_persist:                                                            │
│ ❱ 268 │   │   │   chromadb_client = chromadb.PersistentClient(path=args.chroma_folder, setting   │
│   269 │   │   │   print(f"ChromaDB is running in-memory with persistence. Persistence is store   │
│   270 │   │   else:                                                                              │
│   271 │   │   │   chromadb_client = chromadb.EphemeralClient(Settings(anonymized_telemetry=Fal   │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: module 'chromadb' has no attribute 'PersistentClient'
ERROR conda.cli.main_run:execute(47): `conda run python server.py --cpu --enable-modules=caption,summarize,classify,chromadb --listen` failed. (See above for error)

@valden80 commented on GitHub (Aug 1, 2023): I too getting error form chromaDB startup after the last update: ``` 2023-08-01 12:39:16.825799: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. 2023-08-01 12:39:17.426976: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT Using torch device: cpu Initializing an image captioning model... Initializing a text summarization model... Initializing a sentiment classification pipeline... Initializing ChromaDB ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /home/AI/SillyTavern-extras/server.py:268 in <module> │ │ │ │ 265 │ posthog.capture = lambda *args, **kwargs: None │ │ 266 │ if args.chroma_host is None: │ │ 267 │ │ if args.chroma_persist: │ │ ❱ 268 │ │ │ chromadb_client = chromadb.PersistentClient(path=args.chroma_folder, setting │ │ 269 │ │ │ print(f"ChromaDB is running in-memory with persistence. Persistence is store │ │ 270 │ │ else: │ │ 271 │ │ │ chromadb_client = chromadb.EphemeralClient(Settings(anonymized_telemetry=Fal │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ AttributeError: module 'chromadb' has no attribute 'PersistentClient' ERROR conda.cli.main_run:execute(47): `conda run python server.py --cpu --enable-modules=caption,summarize,classify,chromadb --listen` failed. (See above for error) ```

bsakaguchi commented

Author

Owner

@xNissX233 commented on GitHub (Aug 1, 2023):

@RBNXI Why do your Extras interfere with Oobabooga install? How did you install extras?

I installed it in the same conda than oobabooga, is that wrong?

@xNissX233 commented on GitHub (Aug 1, 2023): > @RBNXI Why do your Extras interfere with Oobabooga install? How did you install extras? I installed it in the same conda than oobabooga, is that wrong?

bsakaguchi commented

Author

Owner

@Cohee1207 commented on GitHub (Aug 1, 2023):

@RBNXI There would likely be problems with that. I suggest installing into a separate environment.
@valden80 Chroma broke reverse compatibility in their Python API. You need to update Chroma in your env: pip install --upgrade chromadb

@Cohee1207 commented on GitHub (Aug 1, 2023): @RBNXI There would likely be problems with that. I suggest installing into a separate environment. @valden80 Chroma broke reverse compatibility in their Python API. You need to update Chroma in your env: `pip install --upgrade chromadb `

bsakaguchi commented

Author

Owner

@xNissX233 commented on GitHub (Aug 1, 2023):

@RBNXI There would likely be problems with that. I suggest installing into a separate environment. `

I tried and I don't get the error anymore, but the character doesn't remember at all... I activated the extension, and I get logs of it working and the database file contains memories, but when I ask about those memories the character doesn't know, do I need more max tokens or what? (I know I have to restart for memories to load, tried and same result)

@xNissX233 commented on GitHub (Aug 1, 2023): > @RBNXI There would likely be problems with that. I suggest installing into a separate environment. ` I tried and I don't get the error anymore, but the character doesn't remember at all... I activated the extension, and I get logs of it working and the database file contains memories, but when I ask about those memories the character doesn't know, do I need more max tokens or what? (I know I have to restart for memories to load, tried and same result)

bsakaguchi commented

Author

Owner

@Cohee1207 commented on GitHub (Aug 1, 2023):

What's your model and API?

@Cohee1207 commented on GitHub (Aug 1, 2023): What's your model and API?

bsakaguchi commented

Author

Owner

@xNissX233 commented on GitHub (Aug 1, 2023):

So it depends on the model?
I use saigaggml-model-q5_1, and Llama.cpp in Oobabooga

@xNissX233 commented on GitHub (Aug 1, 2023): So it depends on the model? I use saigaggml-model-q5_1, and Llama.cpp in Oobabooga

bsakaguchi commented

Author

Owner

@Cohee1207 commented on GitHub (Aug 1, 2023):

Exactly. The model can simply ignore what is being fed into it. Also, I don't know how good Chroma embeddings work with non-English languages (I see your model is not trained on English).

@Cohee1207 commented on GitHub (Aug 1, 2023): Exactly. The model can simply ignore what is being fed into it. Also, I don't know how good Chroma embeddings work with non-English languages (I see your model is not trained on English).

bsakaguchi commented

Author

Owner

@xNissX233 commented on GitHub (Aug 1, 2023):

Exactly. The model can simply ignore what is being fed into it. Also, I don't know how good Chroma embeddings work with non-English languages (I see your model is not trained on English).

what model do you recommend?

@xNissX233 commented on GitHub (Aug 1, 2023): > Exactly. The model can simply ignore what is being fed into it. Also, I don't know how good Chroma embeddings work with non-English languages (I see your model is not trained on English). what model do you recommend?

bsakaguchi commented

Author

Owner

@ghost commented on GitHub (Aug 1, 2023):

@RBNXI There would likely be problems with that. I suggest installing into a separate environment. @valden80 Chroma broke reverse compatibility in their Python API. You need to update Chroma in your env: pip install --upgrade chromadb

Thanks, this fixed chromadb's HttpClient-error I was having

@ghost commented on GitHub (Aug 1, 2023): > @RBNXI There would likely be problems with that. I suggest installing into a separate environment. @valden80 Chroma broke reverse compatibility in their Python API. You need to update Chroma in your env: `pip install --upgrade chromadb ` Thanks, this fixed chromadb's HttpClient-error I was having

bsakaguchi commented

Author

Owner

@Wolf1030 commented on GitHub (Aug 3, 2023):

@valden80 Chroma broke reverse compatibility in their Python API. You need to update Chroma in your env: pip install --upgrade chromadb

This resolved the issue for me. thanks

@Wolf1030 commented on GitHub (Aug 3, 2023): >@valden80 Chroma broke reverse compatibility in their Python API. You need to update Chroma in your env: `pip install --upgrade chromadb ` This resolved the issue for me. thanks

bsakaguchi commented

Author

Owner

@EriIaz commented on GitHub (Aug 4, 2023):

@RBNXI There would likely be problems with that. I suggest installing into a separate environment. @valden80 Chroma broke reverse compatibility in their Python API. You need to update Chroma in your env: pip install --upgrade chromadb

Still fails. Now it gives me this error

(extras) D:\NeuralNetworks\SillyTavern-extras>python server.py --listen --enable-modules=chromadb
Using torch device: cpu
Initializing ChromaDB
Traceback (most recent call last):
  File "D:\NeuralNetworks\SillyTavern-extras\server.py", line 268, in <module>
    chromadb_client = chromadb.PersistentClient(path=args.chroma_folder, settings=Settings(anonymized_telemetry=False))
  File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\__init__.py", line 73, in PersistentClient
    return Client(settings)
  File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\__init__.py", line 110, in Client
    api = system.instance(API)
  File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\config.py", line 195, in instance
    impl = type(self)
  File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\api\segment.py", line 82, in __init__
    self._manager = self.require(SegmentManager)
  File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\config.py", line 134, in require
    inst = self._system.instance(type)
  File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\config.py", line 195, in instance
    impl = type(self)
  File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\segment\impl\manager\local.py", line 73, in __init__
    // PersistentLocalHnswSegment.get_file_handle_count()
  File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\segment\impl\vector\local_persistent_hnsw.py", line 398, in get_file_handle_count
    hnswlib_count = hnswlib.Index.file_handle_count
AttributeError: type object 'hnswlib.Index' has no attribute 'file_handle_count'

(extras) D:\NeuralNetworks\SillyTavern-extras>

Used to work well before the initial faulty change.

@EriIaz commented on GitHub (Aug 4, 2023): > @RBNXI There would likely be problems with that. I suggest installing into a separate environment. @valden80 Chroma broke reverse compatibility in their Python API. You need to update Chroma in your env: `pip install --upgrade chromadb ` Still fails. Now it gives me this error ``` (extras) D:\NeuralNetworks\SillyTavern-extras>python server.py --listen --enable-modules=chromadb Using torch device: cpu Initializing ChromaDB Traceback (most recent call last): File "D:\NeuralNetworks\SillyTavern-extras\server.py", line 268, in <module> chromadb_client = chromadb.PersistentClient(path=args.chroma_folder, settings=Settings(anonymized_telemetry=False)) File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\__init__.py", line 73, in PersistentClient return Client(settings) File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\__init__.py", line 110, in Client api = system.instance(API) File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\config.py", line 195, in instance impl = type(self) File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\api\segment.py", line 82, in __init__ self._manager = self.require(SegmentManager) File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\config.py", line 134, in require inst = self._system.instance(type) File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\config.py", line 195, in instance impl = type(self) File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\segment\impl\manager\local.py", line 73, in __init__ // PersistentLocalHnswSegment.get_file_handle_count() File "C:\Users\renar\.conda\envs\extras\lib\site-packages\chromadb\segment\impl\vector\local_persistent_hnsw.py", line 398, in get_file_handle_count hnswlib_count = hnswlib.Index.file_handle_count AttributeError: type object 'hnswlib.Index' has no attribute 'file_handle_count' (extras) D:\NeuralNetworks\SillyTavern-extras> ``` Used to work well before the initial faulty change.

bsakaguchi commented

Author

Owner

@Cohee1207 commented on GitHub (Aug 4, 2023):

Try removing your conda env and reinstalling. There may be a conflict in hnswlib previously installed via conda

@Cohee1207 commented on GitHub (Aug 4, 2023): Try removing your conda env and reinstalling. There may be a conflict in hnswlib previously installed via conda

bsakaguchi commented

Author

Owner

@EriIaz commented on GitHub (Aug 4, 2023):

Try removing your conda env and reinstalling. There may be a conflict in hnswlib previously installed via conda

I've already did that before the original reply. Reinstalled the environment strictly following the guide, except for omitting cloning the SillyTavern-extras with git and I've also run pip install -r requirements-complete.txt at the end. Got the issue. Tried the same without the complete requirements, only chromadb, nltk and scikit-learn, run into the same issue.

@EriIaz commented on GitHub (Aug 4, 2023): > Try removing your conda env and reinstalling. There may be a conflict in hnswlib previously installed via conda I've already did that before the original reply. Reinstalled the environment strictly following the guide, except for omitting cloning the SillyTavern-extras with git and I've also run pip install -r requirements-complete.txt at the end. Got the issue. Tried the same without the complete requirements, only chromadb, nltk and scikit-learn, run into the same issue.

bsakaguchi commented

Author

Owner

@FitzWM commented on GitHub (Aug 6, 2023):

I'm getting the same bug when trying to run Extras with ChromaDB. It happens on both old env and on a brand new env, both with the version of chromadb installed by requirements-complete.txt and the version from upgrading it via pip. Given that parts of the install seem to be moving to Python 3.11 - and that the program itself now complains about wanting 3.11 when it is run - I'm not even 100% sure that the installation steps are actually correct anymore. But I don't know enough about Conda and Python to really have any idea.

@FitzWM commented on GitHub (Aug 6, 2023): I'm getting the same bug when trying to run Extras with ChromaDB. It happens on both old env and on a brand new env, both with the version of chromadb installed by requirements-complete.txt and the version from upgrading it via pip. Given that parts of the install seem to be moving to Python 3.11 - and that the program itself now complains about wanting 3.11 when it is run - I'm not even 100% sure that the installation steps are actually correct anymore. But I don't know enough about Conda and Python to really have any idea.

bsakaguchi commented

Author

Owner

@Cohee1207 commented on GitHub (Aug 6, 2023):

If you have troubles with local install - run Extras from colab

@Cohee1207 commented on GitHub (Aug 6, 2023): If you have troubles with local install - run Extras from colab

bsakaguchi commented

Author

Owner

@FitzWM commented on GitHub (Aug 6, 2023):

If you have troubles with local install - run Extras from colab

What if I...don't want to do that? I run everything - SillyTavern, various backends, StableDiffusion and various frontends, PrivateGPT, etc. - locally, without any issue. Until the recent changes to ChromaDB, I did the same with SillyTavern-extras. I get that it's a community project, but the response to "the local install seems to be broken after an update, even following the official instructions" shouldn't be "use the remote version because it actually works."

@FitzWM commented on GitHub (Aug 6, 2023): > If you have troubles with local install - run Extras from colab What if I...don't want to do that? I run everything - SillyTavern, various backends, StableDiffusion and various frontends, PrivateGPT, etc. - locally, without any issue. Until the recent changes to ChromaDB, I did the same with SillyTavern-extras. I get that it's a community project, but the response to "the local install seems to be broken after an update, even following the official instructions" shouldn't be "use the remote version because it actually works."

bsakaguchi commented

Author

Owner

@Cohee1207 commented on GitHub (Aug 6, 2023):

We didn't break chroma. They released an update that broke backward compatibility with existing installs. So we either do nothing at all and newer installs will not work or we update but those who pull and not do a reinstall will have it broken. It's a lose-lose situation with no resolution other than prompting everyone to reinstall their local chromadb.

@Cohee1207 commented on GitHub (Aug 6, 2023): We didn't break chroma. They released an update that broke backward compatibility with existing installs. So we either do nothing at all and newer installs will not work or we update but those who pull and not do a reinstall will have it broken. It's a lose-lose situation with no resolution other than prompting everyone to reinstall their local chromadb.

bsakaguchi commented

Author

Owner

@ghost commented on GitHub (Aug 6, 2023):

I'm getting the same bug when trying to run Extras with ChromaDB. It happens on both old env and on a brand new env, both with the version of chromadb installed by requirements-complete.txt and the version from upgrading it via pip. Given that parts of the install seem to be moving to Python 3.11 - and that the program itself now complains about wanting 3.11 when it is run - I'm not even 100% sure that the installation steps are actually correct anymore. But I don't know enough about Conda and Python to really have any idea.

It's not complaining, it's recommending to use python 3.11. Python 3.10 works still currently, but most likely there will be a need to upgrade to the 3.11 in future.
I made fresh conda install just today and it runs fine if we ignore the python upgrade recommendation.

@ghost commented on GitHub (Aug 6, 2023): > I'm getting the same bug when trying to run Extras with ChromaDB. It happens on both old env and on a brand new env, both with the version of chromadb installed by requirements-complete.txt and the version from upgrading it via pip. Given that parts of the install seem to be moving to Python 3.11 - and that the program itself now complains about wanting 3.11 when it is run - I'm not even 100% sure that the installation steps are actually correct anymore. But I don't know enough about Conda and Python to really have any idea. It's not complaining, it's recommending to use python 3.11. Python 3.10 works still currently, but most likely there will be a need to upgrade to the 3.11 in future. I made fresh conda install just today and it runs fine if we ignore the python upgrade recommendation.

bsakaguchi commented

Author

Owner

@ghost commented on GitHub (Aug 6, 2023):

We didn't break chroma. They released an update that broke backward compatibility with existing installs. So we either do nothing at all and newer installs will not work or we update but those who pull and not do a reinstall will have it broken. It's a lose-lose situation with no resolution other than prompting everyone to reinstall their local chromadb.

I think it would be better to make everyone to reinstall if there is no other solution to this. It does sound like only solution to this issue, and I'm not sure if chromadb encrypts data or not.. but I'd rather not use colab where anyone with malicious intent basically could have access to the weird chats I'm having

@ghost commented on GitHub (Aug 6, 2023): > We didn't break chroma. They released an update that broke backward compatibility with existing installs. So we either do nothing at all and newer installs will not work or we update but those who pull and not do a reinstall will have it broken. It's a lose-lose situation with no resolution other than prompting everyone to reinstall their local chromadb. I think it would be better to make everyone to reinstall if there is no other solution to this. It does sound like only solution to this issue, and I'm not sure if chromadb encrypts data or not.. but I'd rather not use colab where anyone with malicious intent basically could have access to the weird chats I'm having

bsakaguchi commented

Author

Owner

@FitzWM commented on GitHub (Aug 6, 2023):

I made fresh conda install just today and it runs fine if we ignore the python upgrade recommendation.

Can I ask how exactly you did this? I'm not doubting you, but I just followed the instructions on the main page a few minutes ago to make a completely new env, and I get the same error with that one. It's using Python 3.10 and the version of ChromaDB installed by requirements-complete.txt - although upgrading ChromaDB via pip doesn't help, either.

Edit: Or would I need to reinstall Conda altogether? My understanding was that Conda provided a virtual environment specifically so you didn't have to do that kind of thing, but my Python knowledge is...limited.

@FitzWM commented on GitHub (Aug 6, 2023): >I made fresh conda install just today and it runs fine if we ignore the python upgrade recommendation. Can I ask how exactly you did this? I'm not doubting you, but I just followed the instructions on the main page a few minutes ago to make a completely new env, and I get the same error with that one. It's using Python 3.10 and the version of ChromaDB installed by requirements-complete.txt - although upgrading ChromaDB via pip doesn't help, either. Edit: Or would I need to reinstall Conda altogether? My understanding was that Conda provided a virtual environment specifically so you didn't have to do that kind of thing, but my Python knowledge is...limited.

bsakaguchi commented

Author

Owner

@ghost commented on GitHub (Aug 6, 2023):

I made fresh conda install just today and it runs fine if we ignore the python upgrade recommendation.

Can I ask how exactly you did this? I'm not doubting you, but I just followed the instructions on the main page a few minutes ago to make a completely new env, and I get the same error with that one. It's using Python 3.10 and the version of ChromaDB installed by requirements-complete.txt - although upgrading ChromaDB via pip doesn't help, either.

Edit: Or would I need to reinstall Conda altogether?

I followed the guide too. But I installed miniconda for the first time today.

I would recommend clearing also your pips packages cache it has. Not sure how conda uses pip, but sometimes that helps when doing fresh install

@ghost commented on GitHub (Aug 6, 2023): > >I made fresh conda install just today and it runs fine if we ignore the python upgrade recommendation. > > Can I ask how exactly you did this? I'm not doubting you, but I just followed the instructions on the main page a few minutes ago to make a completely new env, and I get the same error with that one. It's using Python 3.10 and the version of ChromaDB installed by requirements-complete.txt - although upgrading ChromaDB via pip doesn't help, either. > > Edit: Or would I need to reinstall Conda altogether? I followed the guide too. But I installed miniconda for the first time today. I would recommend clearing also your pips packages cache it has. Not sure how conda uses pip, but sometimes that helps when doing fresh install

bsakaguchi commented

Author

Owner

@Cohee1207 commented on GitHub (Aug 6, 2023):

You tried this and it didn't help?
I tested on my older install in conda and it worked.
pip install --upgrade chromadb

@Cohee1207 commented on GitHub (Aug 6, 2023): You tried this and it didn't help? I tested on my older install in conda and it worked. pip install --upgrade chromadb

bsakaguchi commented

Author

Owner

@FitzWM commented on GitHub (Aug 7, 2023):

Sadly, no. I tried it both in my old env and in a brand-new one made using the instructions on the main page here. The only change I made was installing from requirements-complete.txt, rather than requirements.txt - since that's required for the chromadb module. It upgraded fine, but didn't stop the error messages.

@FitzWM commented on GitHub (Aug 7, 2023): Sadly, no. I tried it both in my old env and in a brand-new one made using the instructions on the main page here. The only change I made was installing from requirements-complete.txt, rather than requirements.txt - since that's required for the chromadb module. It upgraded fine, but didn't stop the error messages.

bsakaguchi commented

Author

Owner

@Cohee1207 commented on GitHub (Aug 7, 2023):

Try this if you installed hnswlib with conda previously:

conda remove hnswlib
pip install chroma-hnswlib

@Cohee1207 commented on GitHub (Aug 7, 2023): Try this if you installed hnswlib with conda previously: ``` conda remove hnswlib pip install chroma-hnswlib ```

bsakaguchi commented

Author

Owner

@FitzWM commented on GitHub (Aug 7, 2023):

No joy, unfortunately. The remove command told me that hnswlib couldn't be found in the target environment, and the install command reported that chroma-hnswlib was already installed. Same thing with both my old env and my fresh one.

I'm honestly really confused, since creating a fresh new env should give me a completely clean slate, right? Meaning that, if it works for other people here, it should work for me. But that doesn't seem to be the case...

Edit: I also tried purging my pip cache after this, as well as reinstalling chroma-hnswlib with pip, but got the same error after both.

@FitzWM commented on GitHub (Aug 7, 2023): No joy, unfortunately. The remove command told me that hnswlib couldn't be found in the target environment, and the install command reported that chroma-hnswlib was already installed. Same thing with both my old env and my fresh one. I'm honestly really confused, since creating a fresh new env should give me a completely clean slate, right? Meaning that, if it works for other people here, it should work for me. But that doesn't seem to be the case... Edit: I also tried purging my pip cache after this, as well as reinstalling chroma-hnswlib with pip, but got the same error after both.

bsakaguchi commented