GigaChat LLM provider + GigaAM (LiteLLM) dev ASR by SaintDemon25 · Pull Request #14 · Timik232/DialogScribe

SaintDemon25 · 2026-05-28T17:09:32Z

Contributing changes from the SaintDemon25 fork (2 commits).

What's included

Pluggable LLM provider: create_llm_client() in summarizer.py returns GigaChatLLMClient when GIGACHAT_API_KEY is set, else the existing OpenAI-compatible LLMClient. All call sites (summary, insights, chat, live-hints cascade, autoflow) route through the factory instead of instantiating LLMClient directly — fixes 401s where live/background paths picked up a stale key.
GigaChat SDK timeout (GIGACHAT_TIMEOUT, default 60s) so the initial TLS+OAuth handshake doesn't ConnectTimeout.
Dev ASR via GigaAM through a LiteLLM proxy (docker-compose.dev.yaml, LITELLM_URL/MODEL).
Secrets kept out of VCS: .env.dev gitignored; no credentials in the diff or commit history.

Notes for the maintainer

The dev compose references a private LiteLLM endpoint for the fork's setup — adjust/remove as appropriate for upstream.

🤖 Generated with Claude Code

- Replace ASR_URL/ASR_MODEL (Mistral) with LITELLM_URL/LITELLM_MODEL (GigaAM via LiteLLM proxy) - Add env_file (.env.dev) for secrets instead of inline ${VAR:-} which overrides with empty host values - Remove MISTRAL_API_KEY, LLM_API_KEY, JWT_SECRET, API_KEY from environment section (provided by .env.dev) Co-authored-by: Saint <saintdemon1337@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Switch summary/insights/chat/live-hints/autoflow from the OpenAI-compatible client to GigaChat via create_llm_client(). Replacing every direct LLMClient() instantiation fixes the live-mode 401 (stale OpenAI LLM_API_KEY) and adds an explicit GigaChat SDK timeout (GIGACHAT_TIMEOUT, default 60s) so the initial TLS+OAuth handshake no longer times out ("Failed to fetch" on summary). Also move GIGACHAT_API_KEY out of docker-compose.dev.yaml into the gitignored .env.dev to keep the credential out of version control. Co-authored-by: Saint <saintdemon1337@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

- LLMCascade._build_client and live_hints router use LLMClient (Mistral via LLM_BASE_URL/LLM_MODEL/LLM_API_KEY) for the Live Advisor, not GigaChat. Transcription stays on GigaAM; summary/insights/chat stay on GigaChat. - New GET /api/live-hints/info: introspects active providers (ASR / advisor / analysis) so the frontend can show a non-intrusive model badge. Co-authored-by: Saint <saintdemon1337@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

SaintDemon25 · 2026-05-29T13:17:58Z

Добавил ещё один коммит в эту ветку: Live Advisor back to Mistral (OpenAI-compatible) + /api/live-hints/info (#6). Live Advisor осознанно использует OpenAI-совместимый \LLMClient\ (Mistral) вместо GigaChat; транскрибация остаётся на GigaAM, а summary/insights/chat — на GigaChat. Также добавлен \GET /api/live-hints/info\ для интроспекции активных провайдеров (нужен фронту для бейджа модели).

…closed) (#7) The transcriber drives ASR via repeated asyncio.run() calls, each creating and closing a fresh event loop. LiteLLMASRClient created its httpx.AsyncClient in __init__, binding to the first loop -> 'Event loop is closed' on the next call, silently falling back to Mistral (whose intermittent SSL errors surfaced as 502 on /api/transcribe). Now each request opens its own client bound to the current loop; close() is a no-op. GigaAM (LiteLLM) works directly without fallback. Co-authored-by: Saint <saintdemon1337@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Pull request overview

This PR adds a pluggable LLM provider layer (OpenAI-compatible vs GigaChat) and updates dev ASR configuration to support GigaAM via a LiteLLM proxy, along with UI introspection for active ASR/LLM models.

Changes:

Introduces GigaChatLLMClient plus create_llm_client() factory and routes several API paths through it.
Adds /api/live-hints/info to surface currently active ASR + LLM models/providers for the UI.
Updates dev tooling/config: LiteLLM-based ASR wiring, gigachat dev dependency, and gitignore for .env.dev.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
`gigaam_transcriber/summarizer.py`	Adds GigaChat client implementation and factory selection logic.
`gigaam_transcriber/chat.py`	Switches default chat client creation path to the factory.
`gigaam_transcriber/llm_cascade.py`	Refactors Live Advisor client construction into a helper method.
`gigaam_transcriber/litellm_client.py`	Uses per-request `httpx.AsyncClient` to avoid cross-event-loop reuse issues.
`routers/analysis.py`	Uses the factory for the analysis router’s global `llm_client`.
`routers/saved_transcriptions.py`	Uses the factory for LLM client initialization.
`routers/meeting_prep.py`	Uses the factory for LLM client initialization.
`routers/autoflow.py`	Uses the factory and gates model override to OpenAI-compatible client.
`routers/live_hints.py`	Adds `/info` endpoint + provider labeling; clarifies Live Advisor still uses OpenAI-compatible client.
`docker-compose.dev.yaml`	Switches dev env to `.env.dev` + LiteLLM/GigaChat defaults.
`.gitignore`	Adds `.env.dev` to keep dev secrets out of VCS.
`requirements.dev.txt`	Adds `gigachat` dependency for dev installs.

Comments suppressed due to low confidence (3)

routers/saved_transcriptions.py:31

After introducing create_llm_client(), llm_client.config.api_key may be coming from GIGACHAT_API_KEY, but the error message still says LLM_API_KEY not configured, which is misleading for GigaChat users. Update the message to be provider-agnostic and mention both env vars.

def _ensure_llm() -> None:
    if not llm_client.config.api_key:
        raise HTTPException(status_code=503, detail="LLM_API_KEY not configured")

routers/meeting_prep.py:23

After introducing create_llm_client(), llm_client.config.api_key may be coming from GIGACHAT_API_KEY, but the error message still says LLM_API_KEY not configured, which is misleading for GigaChat users. Update the message to be provider-agnostic and mention both env vars.

def _ensure_llm() -> None:
    if not llm_client.config.api_key:
        raise HTTPException(status_code=503, detail="LLM_API_KEY not configured")

routers/analysis.py:24

LLMClient is no longer referenced in this module after switching to create_llm_client(), so the import is now unused.

from gigaam_transcriber.summarizer import (
    LLMClient,
    create_llm_client,
    generate_summary,
    get_available_models,
    summary_to_html,
)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+            self._client = GigaChatSDK(
+                credentials=self._credentials,
+                scope=self._scope,
+                model=self._model,
+                verify_ssl_certs=False,
+                timeout=self._timeout,
+            )


+        # Дефолтный таймаут httpx в gigachat SDK слишком мал для начального
+        # TLS handshake + OAuth — увеличиваем, иначе ConnectTimeout.
+        self._timeout = float(os.getenv("GIGACHAT_TIMEOUT", "60"))
+        self._client = None


+def create_llm_client() -> LLMClient | GigaChatLLMClient:
+    """Создать LLM-клиент на основе env-конфигурации."""
+    gigachat_key = os.getenv("GIGACHAT_API_KEY", "")
+    if gigachat_key and HAS_GIGACHAT:
+        logger.info("LLM provider: GigaChat")
+        return GigaChatLLMClient(credentials=gigachat_key)
+    logger.info("LLM provider: OpenAI-compatible (%s)", _default_base_url())
+    return LLMClient()


+class GigaChatLLMClient:
+    """LLM-клиент для GigaChat API (Sber)."""
+
+    def __init__(self, credentials: str = "", scope: str = "GIGACHAT_API_CORP",
+                 model: str = "GigaChat-Pro"):
+        self._credentials = credentials or os.getenv("GIGACHAT_API_KEY", "")
+        self._scope = scope or os.getenv("GIGACHAT_SCOPE", "GIGACHAT_API_CORP")
+        self._model = model or os.getenv("GIGACHAT_MODEL", "GigaChat-Pro")
+        # Дефолтный таймаут httpx в gigachat SDK слишком мал для начального
+        # TLS handshake + OAuth — увеличиваем, иначе ConnectTimeout.
+        self._timeout = float(os.getenv("GIGACHAT_TIMEOUT", "60"))
+        self._client = None
+
+    @property
+    def config(self) -> LLMClientConfig:
+        return LLMClientConfig(
+            base_url="gigachat",
+            api_key=self._credentials,
+            model=self._model,
+        )
+
+    def _get_client(self):
+        if self._client is None:
+            if not HAS_GIGACHAT:
+                raise ImportError("Пакет 'gigachat' не установлен: pip install gigachat")
+            if not self._credentials:
+                raise ValueError("GIGACHAT_API_KEY не задан")
+            self._client = GigaChatSDK(
+                credentials=self._credentials,
+                scope=self._scope,
+                model=self._model,
+                verify_ssl_certs=False,
+                timeout=self._timeout,
+            )
+        return self._client
+
+    def update_config(self, base_url: str, api_key: str, model: str) -> None:
+        self._credentials = api_key
+        self._model = model or "GigaChat-Pro"
+        self._client = None
+
+    def call(self, system_prompt: str, user_text: str, max_tokens: int = 4096) -> str:
+        client = self._get_client()
+        try:
+            payload = Chat(
+                messages=[
+                    Messages(role=MessagesRole.SYSTEM, content=system_prompt),
+                    Messages(role=MessagesRole.USER, content=user_text),
+                ],
+                max_tokens=max_tokens,
+                temperature=0.3,
+            )
+            response = client.chat(payload)
+            return response.choices[0].message.content or ""
+        except Exception as e:
+            raise RuntimeError(f"GigaChat error: {e}") from e
+
+    def test_connection(self) -> tuple[bool, str]:
+        if not self._credentials:
+            return False, "GIGACHAT_API_KEY не задан"
+        try:
+            client = self._get_client()
+            payload = Chat(
+                messages=[Messages(role=MessagesRole.USER, content="Hi")],
+                max_tokens=5,
+            )
+            response = client.chat(payload)
+            if response.choices:
+                return True, f"GigaChat подключён (модель: {self._model})"
+            return False, "Пустой ответ"
+        except Exception as e:
+            return False, f"Ошибка GigaChat: {e}"
+
+
+def create_llm_client() -> LLMClient | GigaChatLLMClient:
+    """Создать LLM-клиент на основе env-конфигурации."""
+    gigachat_key = os.getenv("GIGACHAT_API_KEY", "")
+    if gigachat_key and HAS_GIGACHAT:
+        logger.info("LLM provider: GigaChat")
+        return GigaChatLLMClient(credentials=gigachat_key)
+    logger.info("LLM provider: OpenAI-compatible (%s)", _default_base_url())
+    return LLMClient()


+        asr = get_asr_provider()
+        primary = getattr(asr, "_primary", asr)
+        asr_model = getattr(primary, "_model", "") or ""
+        info["asr"] = {
+            "model": asr_model,
+            "label": "GigaAM" if "gigaam" in asr_model.lower() else (asr_model or "ASR"),
+            "fallback": getattr(asr, "_secondary_name", None),


 from gigaam_transcriber.mindmap import generate_mindmap_markdown
 from gigaam_transcriber.models import SavedTranscription, User
-from gigaam_transcriber.summarizer import LLMClient, generate_summary
+from gigaam_transcriber.summarizer import LLMClient, create_llm_client, generate_summary


 from gigaam_transcriber.limits import check_limit
 from gigaam_transcriber.models import User
-from gigaam_transcriber.summarizer import LLMClient
+from gigaam_transcriber.summarizer import LLMClient, create_llm_client



 from gigaam_transcriber.autoflow import run_autoflow
-from gigaam_transcriber.summarizer import LLMClient, LLMClientConfig
+from gigaam_transcriber.summarizer import LLMClient, LLMClientConfig, create_llm_client


+    env_file:
+      - .env.dev
    environment:
-      # Transcription via Mistral Voxtral
-      - MISTRAL_API_KEY=${MISTRAL_API_KEY:-}
-      - ASR_URL=https://api.mistral.ai
-      - ASR_MODEL=voxtral-mini-latest
-      # LLM for summary / insights / chat (Mistral OpenAI-compat)
-      - LLM_API_KEY=${LLM_API_KEY:-}
-      - LLM_BASE_URL=https://api.mistral.ai/v1
-      - LLM_MODEL=mistral-small-latest
-      # Auth
-      - JWT_SECRET=${JWT_SECRET:-dev-secret-change-in-prod}
-      - API_KEY=${API_KEY:-}
+      # Transcription via GigaAM (LiteLLM proxy)
+      - LITELLM_URL=https://litellm.komolov.synology.me
+      - LITELLM_MODEL=gigaamv3-generation
+      # LLM for summary / insights / chat (GigaChat)
+      # GIGACHAT_API_KEY is injected from .env.dev (gitignored — keep secrets out of VCS)
+      - GIGACHAT_SCOPE=GIGACHAT_API_CORP
+      - GIGACHAT_MODEL=GigaChat-Pro


    """
    if llm_client is None:
-        llm_client = LLMClient()
+        llm_client = create_llm_client()



SaintDemon25 and others added 2 commits May 28, 2026 20:08

SaintDemon25 changed the title ~~feat(asr): switch dev transcription to GigaAM via LiteLLM~~ GigaChat LLM provider + GigaAM (LiteLLM) dev ASR May 28, 2026

Timik232 requested a review from Copilot June 1, 2026 08:16

Copilot started reviewing on behalf of Timik232 June 1, 2026 08:16 View session

Copilot AI reviewed Jun 1, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GigaChat LLM provider + GigaAM (LiteLLM) dev ASR#14

GigaChat LLM provider + GigaAM (LiteLLM) dev ASR#14
SaintDemon25 wants to merge 4 commits into
Timik232:masterfrom
SaintDemon25:master

SaintDemon25 commented May 28, 2026 •

edited

Loading

Uh oh!

SaintDemon25 commented May 29, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

SaintDemon25 commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What's included

Notes for the maintainer

Uh oh!

SaintDemon25 commented May 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SaintDemon25 commented May 28, 2026 •

edited

Loading