open-webui/pipelines
1
0
Fork 0
mirror of https://github.com/open-webui/pipelines.git synced 2026-11-05 01:37:24 +00:00
Code Issues 203 Projects Packages Wiki Activity

[GH-ISSUE #386] pipeline always runs a second time #179

New issue
Closed
opened 2026-10-18 21:03:24 +00:00 by bsakaguchi · 1 comment
bsakaguchi commented 2026-10-18 21:03:24 +00:00
Owner
Copy link

Originally created by @thistleknot on GitHub (Jan 5, 2025).
Original GitHub issue: https://github.com/open-webui/pipelines/issues/386

"""
title: Llama Index Pipeline
author: open-webui
date: 2024-05-30
version: 1.0
license: MIT
description: A pipeline for retrieving relevant information from a knowledge base using the Llama Index library.
requirements: llama-index
"""

from pydantic import BaseModel, Field
from typing import List, Optional
from schemas import OpenAIChatMessage
from langchain.vectorstores.pgvector import DistanceStrategy
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import PGVector

class Pipeline:
    class Valves(BaseModel):
        # Add your custom parameters/configuration here
        fetch_k: int = Field(default=13, description="Number of documents to fetch initially")
        top_k: int = Field(default=8, description="Top K documents to return after filtering")
        pipelines: List[str] = []
        priority: int = 0
        pass
        
    def __init__(self):
        print("Initializing Pipeline...")
        self.documents = None
        self.index = None
        self.valves = self.Valves()
        
        try:
            print("Setting up embeddings...")
            model_kwargs = {'device': 'cuda:0'}
            self.embeddings = HuggingFaceEmbeddings(
                model_name="sentence-transformers/all-MiniLM-L12-v2", 
                model_kwargs=model_kwargs
            )
            print("Embeddings initialized successfully")
            
            print("Connecting to vector store...")
            self.arxiv_db = PGVector(
                embedding_function=self.embeddings,
                collection_name="arxiv", 
                connection_string="postgresql+psycopg2://postgres:abcde@192.168.3.123:5432/vectordb",
                distance_strategy=DistanceStrategy.COSINE,
                use_jsonb=True
            )
            
            print("Vector store connected")
            
        except Exception as e:
            print(f"Error in initialization: {str(e)}")
            raise
        pass

    def pipe(self, user_message: str, model_id: str, messages: List[dict], body: dict) -> str:
        print(f"Processing message: {user_message[:50]}...")
        
        try:
            # Access valve settings
            fetch_k = self.valves.fetch_k
            top_k = self.valves.top_k
            
            print("Getting retriever...")
            retriever = self.arxiv_db.as_retriever(
                search_type="mmr",
                search_kwargs={"k": fetch_k}
            )
            
            print("Retrieving documents...")
            docs = retriever.get_relevant_documents(user_message)
            print(f"Retrieved {len(docs)} documents")
            
            context = "\n".join([doc.page_content for doc in docs])
            print(f"Context length: {len(context)}")
            
            prompt = f"""RAG Context given query: {user_message}

{context}"""
            
            return prompt
            
        except Exception as e:
            print(f"Error in pipe: {str(e)}")
            raise
            
        pass

runs twice in pipeline cli

INFO:     192.168.3.122:38353 - "GET /favicon.png HTTP/1.1" 304 Not Modified
INFO:     192.168.3.122:38353 - "GET /static/favicon.png HTTP/1.1" 304 Not Modified
INFO:     192.168.3.122:38358 - "GET /api/v1/users/user/settings HTTP/1.1" 200 OK
INFO:     192.168.3.122:38353 - "GET /api/v1/users/user/settings HTTP/1.1" 200 OK
INFO:     192.168.3.122:38353 - "GET /static/favicon.png HTTP/1.1" 304 Not Modified
INFO:     192.168.3.122:38353 - "POST /api/v1/chats/new HTTP/1.1" 200 OK
INFO:     192.168.3.122:38353 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
INFO:     192.168.3.122:38353 - "GET /static/favicon.png HTTP/1.1" 304 Not Modified
INFO:     192.168.3.122:38358 - "POST /api/v1/chats/99a0ffce-3eae-4029-8b0d-d4dbbfe9d5fd HTTP/1.1" 200 OK
INFO:     192.168.3.122:38358 - "GET /static/favicon.png HTTP/1.1" 304 Not Modified
INFO:     192.168.3.122:38358 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
INFO:     192.168.3.122:38358 - "POST /api/v1/memories/query HTTP/1.1" 200 OK
INFO:     192.168.3.122:38358 - "POST /api/chat/completions HTTP/1.1" 200 OK
INFO:     192.168.3.122:38358 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
INFO:     192.168.3.122:38359 - "POST /api/chat/completed HTTP/1.1" 200 OK
INFO:     192.168.3.122:38359 - "POST /api/v1/chats/99a0ffce-3eae-4029-8b0d-d4dbbfe9d5fd HTTP/1.1" 200 OK
INFO:     192.168.3.122:38359 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
INFO:     192.168.3.122:38359 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
Error: Extra data: line 3 column 1 (char 38)

Error: Extra data: line 3 column 1 (char 38)

is the result from running a second time

INFO:     127.0.0.1:55958 - "POST /chat/completions HTTP/1.1" 200 OK
Processing message: test...
Getting retriever...
Retrieving documents...
Retrieved 13 documents
Context length: 12613
INFO:     127.0.0.1:55986 - "POST /fetch_arxiv_v2/filter/inlet HTTP/1.1" 200 OK
fetch_arxiv_v2
fetch_arxiv_v2
Processing message: Create a concise, 3-5 word title with an emoji as ...
Getting retriever...
Retrieving documents...
INFO:     127.0.0.1:55990 - "POST /fetch_arxiv_v2/filter/outlet HTTP/1.1" 200 OK
Retrieved 13 documents
Context length: 12057
INFO:     127.0.0.1:55988 - "POST /chat/completions HTTP/1.1" 200 OK
INFO:     127.0.0.1:56008 - "POST /fetch_arxiv_v2/filter/inlet HTTP/1.1" 200 OK
fetch_arxiv_v2
fetch_arxiv_v2
Processing message: ### Task:
Generate 1-3 broad tags categorizing the...
Getting retriever...
Retrieving documents...
Retrieved 13 documents
Context length: 32204
INFO:     127.0.0.1:56010 - "POST /chat/completions HTTP/1.1" 200 OK

I noticed some phrases in the cli that were related to this post

https://github.com/open-webui/pipelines/issues/343

So I set my model

export TASK_MODEL_EXTERNAL=mistral-large-2411

and then ran
open-webui serve

but got what you see above... so I'm not sure what I'm doing wrong.

I suspect the second call has something to do with

Notice that we have to set the Pipeline URL as an OpenAI API connection. This configuration causes the system to interpret pipeline calls as OpenAI API requests, defaulting to TASK_MODEL_EXTERNAL. If this variable isn’t set (which is the default), the pipeline itself will be used for task generation.

I only have one pipeline (removed all others from ui)

my model is handled through a connection that leverages mistral api endpoint, the name of that model is mistral-large-2411, but I'm not sure if I'm supposed to name any available submodel or the connection wrapper that is pointing to this model (i.e. the openwebui connection manager for the api endpoint).

Originally created by @thistleknot on GitHub (Jan 5, 2025). Original GitHub issue: https://github.com/open-webui/pipelines/issues/386 ``` """ title: Llama Index Pipeline author: open-webui date: 2024-05-30 version: 1.0 license: MIT description: A pipeline for retrieving relevant information from a knowledge base using the Llama Index library. requirements: llama-index """ from pydantic import BaseModel, Field from typing import List, Optional from schemas import OpenAIChatMessage from langchain.vectorstores.pgvector import DistanceStrategy from langchain_text_splitters import RecursiveCharacterTextSplitter from langchain_community.embeddings import HuggingFaceEmbeddings from langchain.vectorstores import PGVector class Pipeline: class Valves(BaseModel): # Add your custom parameters/configuration here fetch_k: int = Field(default=13, description="Number of documents to fetch initially") top_k: int = Field(default=8, description="Top K documents to return after filtering") pipelines: List[str] = [] priority: int = 0 pass def __init__(self): print("Initializing Pipeline...") self.documents = None self.index = None self.valves = self.Valves() try: print("Setting up embeddings...") model_kwargs = {'device': 'cuda:0'} self.embeddings = HuggingFaceEmbeddings( model_name="sentence-transformers/all-MiniLM-L12-v2", model_kwargs=model_kwargs ) print("Embeddings initialized successfully") print("Connecting to vector store...") self.arxiv_db = PGVector( embedding_function=self.embeddings, collection_name="arxiv", connection_string="postgresql+psycopg2://postgres:abcde@192.168.3.123:5432/vectordb", distance_strategy=DistanceStrategy.COSINE, use_jsonb=True ) print("Vector store connected") except Exception as e: print(f"Error in initialization: {str(e)}") raise pass def pipe(self, user_message: str, model_id: str, messages: List[dict], body: dict) -> str: print(f"Processing message: {user_message[:50]}...") try: # Access valve settings fetch_k = self.valves.fetch_k top_k = self.valves.top_k print("Getting retriever...") retriever = self.arxiv_db.as_retriever( search_type="mmr", search_kwargs={"k": fetch_k} ) print("Retrieving documents...") docs = retriever.get_relevant_documents(user_message) print(f"Retrieved {len(docs)} documents") context = "\n".join([doc.page_content for doc in docs]) print(f"Context length: {len(context)}") prompt = f"""RAG Context given query: {user_message} {context}""" return prompt except Exception as e: print(f"Error in pipe: {str(e)}") raise pass ``` runs twice in pipeline cli ``` INFO: 192.168.3.122:38353 - "GET /favicon.png HTTP/1.1" 304 Not Modified INFO: 192.168.3.122:38353 - "GET /static/favicon.png HTTP/1.1" 304 Not Modified INFO: 192.168.3.122:38358 - "GET /api/v1/users/user/settings HTTP/1.1" 200 OK INFO: 192.168.3.122:38353 - "GET /api/v1/users/user/settings HTTP/1.1" 200 OK INFO: 192.168.3.122:38353 - "GET /static/favicon.png HTTP/1.1" 304 Not Modified INFO: 192.168.3.122:38353 - "POST /api/v1/chats/new HTTP/1.1" 200 OK INFO: 192.168.3.122:38353 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK INFO: 192.168.3.122:38353 - "GET /static/favicon.png HTTP/1.1" 304 Not Modified INFO: 192.168.3.122:38358 - "POST /api/v1/chats/99a0ffce-3eae-4029-8b0d-d4dbbfe9d5fd HTTP/1.1" 200 OK INFO: 192.168.3.122:38358 - "GET /static/favicon.png HTTP/1.1" 304 Not Modified INFO: 192.168.3.122:38358 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK INFO: 192.168.3.122:38358 - "POST /api/v1/memories/query HTTP/1.1" 200 OK INFO: 192.168.3.122:38358 - "POST /api/chat/completions HTTP/1.1" 200 OK INFO: 192.168.3.122:38358 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK INFO: 192.168.3.122:38359 - "POST /api/chat/completed HTTP/1.1" 200 OK INFO: 192.168.3.122:38359 - "POST /api/v1/chats/99a0ffce-3eae-4029-8b0d-d4dbbfe9d5fd HTTP/1.1" 200 OK INFO: 192.168.3.122:38359 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK INFO: 192.168.3.122:38359 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK Error: Extra data: line 3 column 1 (char 38) ``` Error: Extra data: line 3 column 1 (char 38) is the result from running a second time ``` INFO: 127.0.0.1:55958 - "POST /chat/completions HTTP/1.1" 200 OK Processing message: test... Getting retriever... Retrieving documents... Retrieved 13 documents Context length: 12613 INFO: 127.0.0.1:55986 - "POST /fetch_arxiv_v2/filter/inlet HTTP/1.1" 200 OK fetch_arxiv_v2 fetch_arxiv_v2 Processing message: Create a concise, 3-5 word title with an emoji as ... Getting retriever... Retrieving documents... INFO: 127.0.0.1:55990 - "POST /fetch_arxiv_v2/filter/outlet HTTP/1.1" 200 OK Retrieved 13 documents Context length: 12057 INFO: 127.0.0.1:55988 - "POST /chat/completions HTTP/1.1" 200 OK INFO: 127.0.0.1:56008 - "POST /fetch_arxiv_v2/filter/inlet HTTP/1.1" 200 OK fetch_arxiv_v2 fetch_arxiv_v2 Processing message: ### Task: Generate 1-3 broad tags categorizing the... Getting retriever... Retrieving documents... Retrieved 13 documents Context length: 32204 INFO: 127.0.0.1:56010 - "POST /chat/completions HTTP/1.1" 200 OK ``` I noticed some phrases in the cli that were related to this post https://github.com/open-webui/pipelines/issues/343 So I set my model export TASK_MODEL_EXTERNAL=mistral-large-2411 and then ran open-webui serve but got what you see above... so I'm not sure what I'm doing wrong. I suspect the second call has something to do with `Notice that we have to set the Pipeline URL as an OpenAI API connection. This configuration causes the system to interpret pipeline calls as OpenAI API requests, defaulting to TASK_MODEL_EXTERNAL. If this variable isn’t set (which is the default), the pipeline itself will be used for task generation.` I only have one pipeline (removed all others from ui) my model is handled through a connection that leverages mistral api endpoint, the name of that model is mistral-large-2411, but I'm not sure if I'm supposed to name any available submodel or the connection wrapper that is pointing to this model (i.e. the openwebui connection manager for the api endpoint).
bsakaguchi commented 2026-10-18 21:03:25 +00:00
Author
Owner
Copy link

@thistleknot commented on GitHub (Jan 5, 2025):

image
I set the model parms there, and it at least stopped sending a second retrieval

<!-- gh-comment-id:2571770713 --> @thistleknot commented on GitHub (Jan 5, 2025): ![image](https://github.com/user-attachments/assets/53bd8757-168e-49d3-92cf-b8576ed580cf) I set the model parms there, and it at least stopped sending a second retrieval
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
gemini-permissive-safety
No results found.
Labels
Clear labels   
bug

   
bug

   
enhancement

   
enhancement

   
good first issue

   
help wanted

   
pull-request
Mirrored from GitHub Pull Request

No labels
bug
bug
enhancement
enhancement
good first issue
help wanted
pull-request
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
open-webui/pipelines#179
No description provided.