[GH-ISSUE #194] Cannot initialize the text summarization model #118

New issue

Open

opened 2026-10-15 18:04:38 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-15 18:04:38 +00:00

Owner

Originally created by @highpepsi826 on GitHub (Dec 9, 2023).
Original GitHub issue: https://github.com/SillyTavern/SillyTavern-Extras/issues/194

Hi, I try to install the Extra locally in my new PC run with Win11, however, it keep pop up error with "Cannot find the specified model" as below. Other module such as chromaDB all work fine. My old PC which run with Win11 also no problem at all.

Initializing model error:

_____
(extras) PS C:\Entertainment\SillyTavern-extras> python server.py --enable-modules=summarize
Using torch device: cpu
Initializing a text summarization model...
Traceback (most recent call last):
  File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\utils\import_utils.py", line 1353, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\miniconda3\envs\extras\Lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\bart\modeling_bart.py", line 52, in <module>
    from flash_attn import flash_attn_func, flash_attn_varlen_func
  File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\flash_attn\__init__.py", line 3, in <module>
    from flash_attn.flash_attn_interface import (
  File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\flash_attn\flash_attn_interface.py", line 8, in <module>
    import flash_attn_2_cuda as flash_attn_cuda
ImportError: DLL load failed while importing flash_attn_2_cuda: 找不到指定的模組。

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Entertainment\SillyTavern-extras\server.py", line 220, in <module>
    summarization_transformer = AutoModelForSeq2SeqLM.from_pretrained(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\auto\auto_factory.py", line 565, in from_pretrained
    model_class = _get_model_class(config, cls._model_mapping)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\auto\auto_factory.py", line 387, in _get_model_class
    supported_models = model_mapping[type(config)]
                       ~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\auto\auto_factory.py", line 740, in __getitem__
    return self._load_attr_from_module(model_type, model_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\auto\auto_factory.py", line 754, in _load_attr_from_module
    return getattribute_from_module(self._modules[module_name], attr)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\auto\auto_factory.py", line 698, in getattribute_from_module
    if hasattr(module, attr):
       ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\utils\import_utils.py", line 1343, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\utils\import_utils.py", line 1355, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.bart.modeling_bart because of the following error (look up to see its traceback):
DLL load failed while importing flash_attn_2_cuda: 找不到指定的模組。
(extras) PS C:\Entertainment\SillyTavern-extras>
_______________

On the other hand, while I install the requirement.txt, it has showed below error, don't know if it related:

____
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lm-eval 0.3.0 requires zstandard, which is not installed.
lm-eval 0.3.0 requires sacrebleu==1.5.0, but you have sacrebleu 2.3.1 which is incompatible.
_______

Originally created by @highpepsi826 on GitHub (Dec 9, 2023). Original GitHub issue: https://github.com/SillyTavern/SillyTavern-Extras/issues/194 Hi, I try to install the Extra locally in my new PC run with Win11, however, it keep pop up error with "Cannot find the specified model" as below. Other module such as chromaDB all work fine. My old PC which run with Win11 also no problem at all. Initializing model error: ``` _____ (extras) PS C:\Entertainment\SillyTavern-extras> python server.py --enable-modules=summarize Using torch device: cpu Initializing a text summarization model... Traceback (most recent call last): File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\utils\import_utils.py", line 1353, in _get_module return importlib.import_module("." + module_name, self.__name__) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ProgramData\miniconda3\envs\extras\Lib\importlib\__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1204, in _gcd_import File "<frozen importlib._bootstrap>", line 1176, in _find_and_load File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 690, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 940, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\bart\modeling_bart.py", line 52, in <module> from flash_attn import flash_attn_func, flash_attn_varlen_func File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\flash_attn\__init__.py", line 3, in <module> from flash_attn.flash_attn_interface import ( File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\flash_attn\flash_attn_interface.py", line 8, in <module> import flash_attn_2_cuda as flash_attn_cuda ImportError: DLL load failed while importing flash_attn_2_cuda: 找不到指定的模組。 The above exception was the direct cause of the following exception: Traceback (most recent call last): File "C:\Entertainment\SillyTavern-extras\server.py", line 220, in <module> summarization_transformer = AutoModelForSeq2SeqLM.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\auto\auto_factory.py", line 565, in from_pretrained model_class = _get_model_class(config, cls._model_mapping) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\auto\auto_factory.py", line 387, in _get_model_class supported_models = model_mapping[type(config)] ~~~~~~~~~~~~~^^^^^^^^^^^^^^ File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\auto\auto_factory.py", line 740, in __getitem__ return self._load_attr_from_module(model_type, model_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\auto\auto_factory.py", line 754, in _load_attr_from_module return getattribute_from_module(self._modules[module_name], attr) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\models\auto\auto_factory.py", line 698, in getattribute_from_module if hasattr(module, attr): ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\utils\import_utils.py", line 1343, in __getattr__ module = self._get_module(self._class_to_module[name]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\high_\AppData\Roaming\Python\Python311\site-packages\transformers\utils\import_utils.py", line 1355, in _get_module raise RuntimeError( RuntimeError: Failed to import transformers.models.bart.modeling_bart because of the following error (look up to see its traceback): DLL load failed while importing flash_attn_2_cuda: 找不到指定的模組。 (extras) PS C:\Entertainment\SillyTavern-extras> _______________ ``` On the other hand, while I install the requirement.txt, it has showed below error, don't know if it related: ``` ____ ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. lm-eval 0.3.0 requires zstandard, which is not installed. lm-eval 0.3.0 requires sacrebleu==1.5.0, but you have sacrebleu 2.3.1 which is incompatible. _______ ```