Concurrency: Remove release_semaphore method

At any point for any request cancellation, the semaphore will be
decremented. This is an issue since an arbitrary request can desync
the semaphore, causing multiple tasks to be processed at once and
break generation.

Remove this from the networking handlers and therefore, remove the
release_semaphore function itself.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri
2024-05-19 10:42:26 -04:00
parent b9fd8555fe
commit c474076b22
2 changed files with 0 additions and 7 deletions

View File

@@ -8,8 +8,6 @@ from loguru import logger
from pydantic import BaseModel
from typing import Optional
from common.concurrency import release_semaphore
class TabbyRequestErrorMessage(BaseModel):
"""Common request error type."""
@@ -53,7 +51,6 @@ def handle_request_error(message: str, exc_info: bool = True):
def handle_request_disconnect(message: str):
"""Wrapper for handling for request disconnection."""
release_semaphore()
logger.error(message)