1.1" 404 Not Found #785

New issue

Closed

opened 2026-10-29 14:44:30 +00:00 by bsakaguchi · 2 comments

bsakaguchi commented

2026-10-29 14:44:30 +00:00

Owner

Originally created by @IAMJOYBO on GitHub (Apr 12, 2025).
Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1127

检查清单

1. 我已经搜索过相关问题，但未能获得预期的帮助
2. 该问题在最新版本中尚未修复
3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性
4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭
5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭

问题描述

ktransformers  | loading output_norm.weight to cuda:0
ktransformers  | 2025-04-12 18:05:39,930 DEBUG /opt/conda/lib/python3.11/site-packages/ktransformers/server/backend/context_manager.py[23]: Creating Context Manager
ktransformers  | 2025-04-12 18:05:39,931 INFO /opt/conda/lib/python3.11/site-packages/ktransformers/server/main.py[25]: Creating SQL tables
ktransformers  | 2025-04-12 18:05:39,989 INFO /opt/conda/lib/python3.11/site-packages/ktransformers/server/api/openai/assistants/assistants.py[75]: Creating default assistant
ktransformers  | INFO:     Started server process [1]
ktransformers  | INFO:     Waiting for application startup.
ktransformers  | INFO:     Application startup complete.
ktransformers  | INFO:     Uvicorn running on http://0.0.0.0:10002 (Press CTRL+C to quit)
ktransformers  | INFO:     172.21.0.1:40422 - "GET / HTTP/1.1" 404 Not Found
ktransformers  | INFO:     172.21.0.1:40422 - "GET /favicon.ico HTTP/1.1" 404 Not Found
ktransformers  | INFO:     172.21.0.1:59982 - "GET / HTTP/1.1" 404 Not Found
ktransformers  | INFO:     172.21.0.1:59982 - "GET /favicon.ico HTTP/1.1" 404 Not Found

复现步骤

services:
  ktransformers:
    image: registry.cn-hangzhou.aliyuncs.com/joybo/ktransformers:v2025.04.12-action-web
    container_name: ktransformers
    hostname: ktransformers
    environment:
      - TZ=Asia/Shanghai
      # - NVIDIA_VISIBLE_DEVICES=0
    volumes:
      - ./DeepSeek-V2-Lite-Chat-GGUF:/model/DeepSeek-V2-Lite-Chat-GGUF
      - ./DeepSeek-V2-Lite-Chat:/model/DeepSeek-V2-Lite-Chat
    ports:
      - "10002:10002"
    runtime: nvidia
    deploy:
      resources:
        reservations:
          devices:
          - driver: nvidia
            # device_ids: ['0']
            capabilities: [gpu]
    stdin_open: true
    tty: true
    restart: no
    networks:
      - ktransformers
    entrypoint: ["ktransformers", "--model_path=/model/DeepSeek-V2-Lite-Chat", "--gguf_path=/model/DeepSeek-V2-Lite-Chat-GGUF", "--port=10002", "--web=True"]

networks:
  ktransformers:
    driver: bridge
    name: ktransformers

环境信息

Docker Build Workflow：https://github.com/IAMJOYBO/ktransformers/actions/runs/14421109622/job/40443784050

# FROM pytorch/pytorch:2.5.1-cuda12.1-cudnn9-devel as compile_server
FROM pytorch/pytorch:2.6.0-cuda12.6-cudnn9-devel as compile_server

ARG CPU_INSTRUCT=NATIVE

# 设置工作目录和 CUDA 路径
WORKDIR /workspace
ENV CUDA_HOME=/usr/local/cuda

# 安装依赖
RUN apt update -y && apt install -y apt-utils && apt upgrade -y
RUN apt install -y --no-install-recommends \
    libtbb-dev \
    libssl-dev \
    libcurl4-openssl-dev \
    libaio1 \
    libaio-dev \
    libfmt-dev \
    libgflags-dev \
    zlib1g-dev \
    patchelf \
    git \
    wget \
    vim \
    gcc \
    g++

# 安装CMake
RUN wget https://github.com/Kitware/CMake/releases/download/v4.0.1/cmake-4.0.1-linux-x86_64.sh && echo y | bash cmake-4.0.1-linux-x86_64.sh && rm -rf cmake-4.0.1-linux-x86_64.sh

# 克隆代码
RUN git clone https://github.com/kvcache-ai/ktransformers.git 
# 清理 apt 缓存
RUN rm -rf /var/lib/apt/lists/*

# 进入项目目录
WORKDIR /workspace/ktransformers
# 初始化子模块
RUN git submodule update --init --recursive

# 升级 pip
RUN pip install --upgrade pip && pip install -U wheel setuptools

# 安装构建依赖
RUN pip install ninja pyproject numpy cpufeature aiohttp zmq openai

# 安装 flash-attn（提前装可以避免后续某些编译依赖出错）
RUN pip install flash-attn

# 安装WEB所需环境
RUN apt-get update -y && apt-get install -y apt-transport-https ca-certificates curl gnupg
RUN curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /usr/share/keyrings/nodesource.gpg
RUN chmod 644 /usr/share/keyrings/nodesource.gpg
RUN echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_23.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list
RUN apt-get update -y
RUN apt-get install nodejs -y

# 安装 WEB
RUN cd ktransformers/website && npm install @vue/cli && npm run build

# 安装 ktransformers 本体（含编译）
RUN CPU_INSTRUCT=${CPU_INSTRUCT} \
    USE_BALANCE_SERVE=1 \
    KTRANSFORMERS_FORCE_BUILD=TRUE \
    TORCH_CUDA_ARCH_LIST="8.0;8.6;8.7;8.9;9.0+PTX" \
    pip install . --no-build-isolation --verbose

RUN pip install third_party/custom_flashinfer/
# 清理 pip 缓存
RUN pip cache purge

# 拷贝 C++ 运行时库
RUN cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /opt/conda/lib/

# 预下载的配置文件
RUN pip install huggingface_hub modelscope
RUN huggingface-cli download deepseek-ai/DeepSeek-R1 --exclude *.safetensors --local-dir /app/model/DeepSeek-R1
RUN huggingface-cli download deepseek-ai/DeepSeek-V3-0324 --exclude *.safetensors --local-dir /app/model/DeepSeek-V3-0324
RUN huggingface-cli download deepseek-ai/DeepSeek-V2-Lite-Chat --exclude *.safetensors --local-dir /app/model/DeepSeek-V2-Lite-Chat

# 保持容器运行（调试用）
ENTRYPOINT ["tail", "-f", "/dev/null"]

Originally created by @IAMJOYBO on GitHub (Apr 12, 2025). Original GitHub issue: https://github.com/kvcache-ai/ktransformers/issues/1127 ### 检查清单 - [x] 1. 我已经搜索过相关问题，但未能获得预期的帮助 - [x] 2. 该问题在最新版本中尚未修复 - [x] 3. 请注意，如果您提交的BUG相关 issue 缺少对应环境信息和最小可复现示例，我们将难以复现和定位问题，降低获得反馈的可能性 - [x] 4. 如果您提出的不是bug而是问题，请在讨论区发起讨论 https://github.com/kvcache-ai/ktransformers/discussions。否则该 issue 将被关闭 - [x] 5. 为方便社区交流，我将使用中文/英文或附上中文/英文翻译（如使用其他语言）。未附带翻译的非中文/英语内容可能会被关闭 ### 问题描述 ```shell ktransformers | loading output_norm.weight to cuda:0 ktransformers | 2025-04-12 18:05:39,930 DEBUG /opt/conda/lib/python3.11/site-packages/ktransformers/server/backend/context_manager.py[23]: Creating Context Manager ktransformers | 2025-04-12 18:05:39,931 INFO /opt/conda/lib/python3.11/site-packages/ktransformers/server/main.py[25]: Creating SQL tables ktransformers | 2025-04-12 18:05:39,989 INFO /opt/conda/lib/python3.11/site-packages/ktransformers/server/api/openai/assistants/assistants.py[75]: Creating default assistant ktransformers | INFO: Started server process [1] ktransformers | INFO: Waiting for application startup. ktransformers | INFO: Application startup complete. ktransformers | INFO: Uvicorn running on http://0.0.0.0:10002 (Press CTRL+C to quit) ktransformers | INFO: 172.21.0.1:40422 - "GET / HTTP/1.1" 404 Not Found ktransformers | INFO: 172.21.0.1:40422 - "GET /favicon.ico HTTP/1.1" 404 Not Found ktransformers | INFO: 172.21.0.1:59982 - "GET / HTTP/1.1" 404 Not Found ktransformers | INFO: 172.21.0.1:59982 - "GET /favicon.ico HTTP/1.1" 404 Not Found ``` ![Image](https://github.com/user-attachments/assets/fbd8a68a-88a3-430a-8eec-4f5ac06ed9fc) ![Image](https://github.com/user-attachments/assets/d2876620-ea78-4808-b9af-e5312d4f35f1) ![Image](https://github.com/user-attachments/assets/d0533f9d-59e6-443c-8220-e01e9cbc06e6) ### 复现步骤 ```yaml services: ktransformers: image: registry.cn-hangzhou.aliyuncs.com/joybo/ktransformers:v2025.04.12-action-web container_name: ktransformers hostname: ktransformers environment: - TZ=Asia/Shanghai # - NVIDIA_VISIBLE_DEVICES=0 volumes: - ./DeepSeek-V2-Lite-Chat-GGUF:/model/DeepSeek-V2-Lite-Chat-GGUF - ./DeepSeek-V2-Lite-Chat:/model/DeepSeek-V2-Lite-Chat ports: - "10002:10002" runtime: nvidia deploy: resources: reservations: devices: - driver: nvidia # device_ids: ['0'] capabilities: [gpu] stdin_open: true tty: true restart: no networks: - ktransformers entrypoint: ["ktransformers", "--model_path=/model/DeepSeek-V2-Lite-Chat", "--gguf_path=/model/DeepSeek-V2-Lite-Chat-GGUF", "--port=10002", "--web=True"] networks: ktransformers: driver: bridge name: ktransformers ``` ### 环境信息 Docker Build Workflow：[https://github.com/IAMJOYBO/ktransformers/actions/runs/14421109622/job/40443784050](https://github.com/IAMJOYBO/ktransformers/actions/runs/14421109622/job/40443784050) ```Dockerfile # FROM pytorch/pytorch:2.5.1-cuda12.1-cudnn9-devel as compile_server FROM pytorch/pytorch:2.6.0-cuda12.6-cudnn9-devel as compile_server ARG CPU_INSTRUCT=NATIVE # 设置工作目录和 CUDA 路径 WORKDIR /workspace ENV CUDA_HOME=/usr/local/cuda # 安装依赖 RUN apt update -y && apt install -y apt-utils && apt upgrade -y RUN apt install -y --no-install-recommends \ libtbb-dev \ libssl-dev \ libcurl4-openssl-dev \ libaio1 \ libaio-dev \ libfmt-dev \ libgflags-dev \ zlib1g-dev \ patchelf \ git \ wget \ vim \ gcc \ g++ # 安装CMake RUN wget https://github.com/Kitware/CMake/releases/download/v4.0.1/cmake-4.0.1-linux-x86_64.sh && echo y | bash cmake-4.0.1-linux-x86_64.sh && rm -rf cmake-4.0.1-linux-x86_64.sh # 克隆代码 RUN git clone https://github.com/kvcache-ai/ktransformers.git # 清理 apt 缓存 RUN rm -rf /var/lib/apt/lists/* # 进入项目目录 WORKDIR /workspace/ktransformers # 初始化子模块 RUN git submodule update --init --recursive # 升级 pip RUN pip install --upgrade pip && pip install -U wheel setuptools # 安装构建依赖 RUN pip install ninja pyproject numpy cpufeature aiohttp zmq openai # 安装 flash-attn（提前装可以避免后续某些编译依赖出错） RUN pip install flash-attn # 安装WEB所需环境 RUN apt-get update -y && apt-get install -y apt-transport-https ca-certificates curl gnupg RUN curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /usr/share/keyrings/nodesource.gpg RUN chmod 644 /usr/share/keyrings/nodesource.gpg RUN echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_23.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list RUN apt-get update -y RUN apt-get install nodejs -y # 安装 WEB RUN cd ktransformers/website && npm install @vue/cli && npm run build # 安装 ktransformers 本体（含编译） RUN CPU_INSTRUCT=${CPU_INSTRUCT} \ USE_BALANCE_SERVE=1 \ KTRANSFORMERS_FORCE_BUILD=TRUE \ TORCH_CUDA_ARCH_LIST="8.0;8.6;8.7;8.9;9.0+PTX" \ pip install . --no-build-isolation --verbose RUN pip install third_party/custom_flashinfer/ # 清理 pip 缓存 RUN pip cache purge # 拷贝 C++ 运行时库 RUN cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /opt/conda/lib/ # 预下载的配置文件 RUN pip install huggingface_hub modelscope RUN huggingface-cli download deepseek-ai/DeepSeek-R1 --exclude *.safetensors --local-dir /app/model/DeepSeek-R1 RUN huggingface-cli download deepseek-ai/DeepSeek-V3-0324 --exclude *.safetensors --local-dir /app/model/DeepSeek-V3-0324 RUN huggingface-cli download deepseek-ai/DeepSeek-V2-Lite-Chat --exclude *.safetensors --local-dir /app/model/DeepSeek-V2-Lite-Chat # 保持容器运行（调试用） ENTRYPOINT ["tail", "-f", "/dev/null"] ```

bsakaguchi closed this issue

2026-10-29 14:44:30 +00:00

bsakaguchi commented

2026-10-29 14:44:30 +00:00

Author

Owner

@ErvinXie commented on GitHub (Apr 13, 2025):

你好，这个不是一个问题，我们使用 fastapi，但是没有在主页放任何的内容。请直接访问 127.0.0.1:10002/docs 进入到 api 的目录。具体请参见 fastapi 官方文档 https://fastapi.tiangolo.com/zh/#api

@ErvinXie commented on GitHub (Apr 13, 2025): 你好，这个不是一个问题，我们使用 fastapi，但是没有在主页放任何的内容。请直接访问 127.0.0.1:10002/docs 进入到 api 的目录。具体请参见 fastapi 官方文档 https://fastapi.tiangolo.com/zh/#api

bsakaguchi commented

2026-10-29 14:44:31 +00:00

Author

Owner

@KathySkyer commented on GitHub (May 23, 2025):

楼主解决了吗？favicon.ico 需要放在哪里？

@KathySkyer commented on GitHub (May 23, 2025): 楼主解决了吗？favicon.ico 需要放在哪里？