diff --git a/control-plane/docs/superpowers/plans/2026-07-01-tdlib-media-download-poc.md b/control-plane/docs/superpowers/plans/2026-07-01-tdlib-media-download-poc.md new file mode 100644 index 0000000..95581d6 --- /dev/null +++ b/control-plane/docs/superpowers/plans/2026-07-01-tdlib-media-download-poc.md @@ -0,0 +1,1115 @@ +# TDLib Media-Download POC Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build an isolated, ADR-compliant proof of concept that measures whether TDLib beats the current `telegram-mcp` (Telethon) path on media-download latency and resumability, using real large media from the `main` account. + +**Architecture:** A standalone Python project at `experiments/tdlib-media-poc/`, fully separate from `mcp/` and `plugin/`. It uses `pytdbot` (tdjson binding) with its own TDLib database/files directory, authenticated as a second, independent session on the `main` account. It never reads or writes the Telethon session tree. Benchmark inputs are real t.me links the operator supplies; outputs are two JSON result files plus a generated `RESULTS.md` comparing both backends against the same inputs. + +**Tech Stack:** Python ≥3.12, `uv` for env/deps, `pytdbot` 0.10.1 (confirmed installable and functional on macOS arm64 in this environment), `python-dotenv`, `pytest`. Baseline runs shell out to the existing `mcp/bin/tg download` CLI (the current production large-media path, added in commit `cc51cd8`). + +## Global Constraints + +- This is governed by [control-plane/docs/adr/2026-06-21-tdlib-is-not-default-runtime.md](../adr/2026-06-21-tdlib-is-not-default-runtime.md). The POC must stay: read-only; isolated from existing Telethon session files; limited to one account (`main`) and one scenario (media download latency/resumability); backed by a dedicated database and files directory; measured against the current `telegram-mcp` path with the same input data; excluded from default routing, LaunchAgents, release gates, and installed plugin docs. +- All POC code lives under `experiments/tdlib-media-poc/` at the repo root. Never modify `mcp/src/telegram_mcp/`, `plugin/`, or any LaunchAgent/release-gate config as part of this plan. +- TDLib's isolated state (`files_directory`) must live at `experiments/tdlib-media-poc/data/tdlib/`. It must never point at or nest inside `~/.telegram-mcp/` (the Telethon session dir per `mcp/.env.example:5`) — enforced in code via `assert_isolated_from_telethon`. +- `pytdbot.Client(...)` real constructor signature on this machine (verified live): accepts `api_id`, `api_hash`, `files_directory`, `database_encryption_key`, `use_test_dc`, `use_file_database`, `use_chat_info_database`, `use_message_database`, `workers`, `no_updates`, etc. There is **no** separate `database_directory` parameter — `files_directory` is the single isolation root. +- Confirmed method signatures on `pytdbot.Client` (verified live): + - `start(wait_login: bool = True) -> None` + - `getMe() -> Error | User` + - `downloadFile(*, file_id: int = 0, priority: int = 0, offset: int = 0, limit: int = 0, synchronous: bool = False) -> Error | File` + - `cancelDownloadFile(*, file_id: int = 0, only_if_pending: bool = False) -> Error | Ok` +- `pytdbot.types.File.to_dict()` and `pytdbot.types.LocalFile.__init__` fields (verified live): `File` has `id`, `size`, `expected_size`, `local`, `remote`; `LocalFile` has `path`, `can_be_downloaded`, `can_be_deleted`, `is_downloading_active`, `is_downloading_completed`, `download_offset`, `downloaded_prefix_size`, `downloaded_size`. +- The `tg` CLI baseline lives at `mcp/bin/tg` (already built in this worktree via `cd mcp && uv sync`). Relevant subcommands used here: `tg message --json --account main` and `tg download --dest --account main --json`. +- `tg message --json` envelope shape (verified from `mcp/src/telegram_mcp/tg_cli.py:578-585` and `mcp/src/telegram_mcp/types.py:220-226,40-59`): `{"ok": bool, "command": "message", "payload": {"chat": {...}, "message_id": int, "message": {"id": int, "file_size": int|null, "media_type": str|null, ...} | null, "data_source": str, "method": str}}`. +- API credentials: reuse the same `TELEGRAM_API_ID`/`TELEGRAM_API_HASH` values already used by `mcp/.env` (same my.telegram.org app — safe to reuse across sessions), stored in a separate `experiments/tdlib-media-poc/.env` (gitignored, never committed). +- Test account: `main`. +- All Telegram operations performed by this POC (both Telethon-side and TDLib-side) must be read-only (message lookup, media download). No send/edit/delete/react calls anywhere in this plan. + +--- + +### Task 1: Bootstrap isolated POC project + +**Files:** +- Create: `experiments/tdlib-media-poc/pyproject.toml` +- Create: `experiments/tdlib-media-poc/.gitignore` +- Create: `experiments/tdlib-media-poc/.env.example` +- Create: `experiments/tdlib-media-poc/README.md` +- Create: `experiments/tdlib-media-poc/benchmark/__init__.py` +- Create: `experiments/tdlib-media-poc/benchmark/tdlib_client.py` +- Test: `experiments/tdlib-media-poc/tests/test_tdlib_client.py` + +**Interfaces:** +- Produces: `benchmark.tdlib_client.assert_isolated_from_telethon(files_directory: str) -> None` (raises `ValueError` on overlap), `benchmark.tdlib_client.build_client(api_id: int, api_hash: str, files_directory: str, database_encryption_key: str = "tdlib-media-poc") -> pytdbot.Client`. Used by Task 3 and Task 5. + +- [ ] **Step 1: Create the project scaffold files** + +`experiments/tdlib-media-poc/pyproject.toml`: +```toml +[project] +name = "tdlib-media-poc" +version = "0.1.0" +requires-python = ">=3.12" +dependencies = [ + "Pytdbot>=0.10.1", + "python-dotenv>=1.0.1", +] + +[dependency-groups] +dev = [ + "pytest>=8.0.0", +] + +[tool.uv] +package = false +``` + +`experiments/tdlib-media-poc/.gitignore`: +``` +data/ +.env +__pycache__/ +*.pyc +.pytest_cache/ +.venv/ +``` + +`experiments/tdlib-media-poc/.env.example`: +``` +# Reuse the same values as mcp/.env (same my.telegram.org app, safe to share). +TELEGRAM_API_ID= +TELEGRAM_API_HASH= +``` + +`experiments/tdlib-media-poc/README.md`: +```markdown +# TDLib media-download POC + +Isolated lab proof of concept required by +[control-plane/docs/adr/2026-06-21-tdlib-is-not-default-runtime.md](../../control-plane/docs/adr/2026-06-21-tdlib-is-not-default-runtime.md). + +Scenario: media download latency/resumability, measured against the current +`telegram-mcp` (Telethon) path, on the `main` account only. + +## Isolation guarantees + +- TDLib state lives at `data/tdlib/` (gitignored). It is never pointed at the + Telethon session tree (`~/.telegram-mcp/`). +- Read-only Telegram operations only. +- Not wired into `mcp/`, `plugin/`, LaunchAgents, or release gates. + +## Setup + +```bash +cd experiments/tdlib-media-poc +uv sync +cp .env.example .env # fill in TELEGRAM_API_ID / TELEGRAM_API_HASH from mcp/.env +``` + +## Run order + +1. `uv run python benchmark/build_benchmark_set.py [ ...]` +2. `uv run python benchmark/login_tdlib.py` (one-time, interactive — needs a live login code) +3. `uv run python benchmark/run_telethon.py` +4. `uv run python benchmark/run_tdlib.py` +5. `uv run python benchmark/compare.py` → writes `data/RESULTS.md` + +## Tests + +```bash +uv run pytest tests/ -v +``` +``` + +`experiments/tdlib-media-poc/benchmark/__init__.py`: empty file. + +- [ ] **Step 2: Write the failing test for the isolation guard** + +```python +# experiments/tdlib-media-poc/tests/test_tdlib_client.py +import pytest + +from benchmark.tdlib_client import assert_isolated_from_telethon, build_client + + +def test_assert_isolated_from_telethon_passes_for_poc_dir(tmp_path): + assert_isolated_from_telethon(str(tmp_path / "tdlib-media-poc" / "data" / "tdlib")) + + +def test_assert_isolated_from_telethon_rejects_telethon_session_dir(): + with pytest.raises(ValueError, match="Telethon session tree"): + assert_isolated_from_telethon("/Users/sereja/.telegram-mcp/session") + + +def test_build_client_constructs_with_isolated_directory(tmp_path): + client = build_client( + api_id=1, + api_hash="0" * 32, + files_directory=str(tmp_path / "tdlib"), + ) + assert client is not None +``` + +- [ ] **Step 3: Run test to verify it fails** + +Run: `cd experiments/tdlib-media-poc && uv sync -q && uv run pytest tests/test_tdlib_client.py -v` +Expected: FAIL with `ModuleNotFoundError: No module named 'benchmark.tdlib_client'` + +- [ ] **Step 4: Write the minimal implementation** + +```python +# experiments/tdlib-media-poc/benchmark/tdlib_client.py +"""Isolated pytdbot client factory for the TDLib media-download POC. + +Per control-plane/docs/adr/2026-06-21-tdlib-is-not-default-runtime.md, this +POC must never share state with the Telethon session tree that telegram-mcp +owns. +""" + +import pytdbot + +TELETHON_SESSION_DIR_MARKERS = (".telegram-mcp",) + + +def assert_isolated_from_telethon(files_directory: str) -> None: + for marker in TELETHON_SESSION_DIR_MARKERS: + if marker in files_directory: + raise ValueError( + f"files_directory must not overlap the Telethon session tree (found {marker!r})" + ) + + +def build_client( + api_id: int, + api_hash: str, + files_directory: str, + database_encryption_key: str = "tdlib-media-poc", +) -> pytdbot.Client: + assert_isolated_from_telethon(files_directory) + return pytdbot.Client( + api_id=api_id, + api_hash=api_hash, + files_directory=files_directory, + database_encryption_key=database_encryption_key, + ) +``` + +- [ ] **Step 5: Run test to verify it passes** + +Run: `cd experiments/tdlib-media-poc && uv run pytest tests/test_tdlib_client.py -v` +Expected: PASS (3 tests) + +- [ ] **Step 6: Commit** + +```bash +git add experiments/tdlib-media-poc/pyproject.toml experiments/tdlib-media-poc/.gitignore \ + experiments/tdlib-media-poc/.env.example experiments/tdlib-media-poc/README.md \ + experiments/tdlib-media-poc/benchmark/__init__.py experiments/tdlib-media-poc/benchmark/tdlib_client.py \ + experiments/tdlib-media-poc/tests/test_tdlib_client.py +git commit -m "poc: bootstrap isolated tdlib media-download poc project" +``` + +--- + +### Task 2: Benchmark target model, link parsing, file-size extraction + +**Files:** +- Create: `experiments/tdlib-media-poc/benchmark/models.py` +- Create: `experiments/tdlib-media-poc/benchmark/select_targets.py` +- Create: `experiments/tdlib-media-poc/benchmark/build_benchmark_set.py` +- Test: `experiments/tdlib-media-poc/tests/test_models.py` +- Test: `experiments/tdlib-media-poc/tests/test_select_targets.py` + +**Interfaces:** +- Consumes: nothing from Task 1. +- Produces: `benchmark.models.BenchmarkTarget` (dataclass: `label: str, chat: str, message_id: int, link: str, expected_size_bytes: int | None`), `benchmark.models.DownloadResult` (dataclass: `label: str, backend: str, ok: bool, elapsed_seconds: float, bytes_downloaded: int | None, resumed: bool, error: str | None = None`), `benchmark.models.save_benchmark_set/load_benchmark_set/save_results/load_results`, `benchmark.select_targets.parse_link(link: str) -> tuple[str, int]`, `benchmark.select_targets.extract_file_size(tg_message_envelope: dict) -> int | None`. Used by Tasks 4, 5, 6. + +- [ ] **Step 1: Write the failing tests for models** + +```python +# experiments/tdlib-media-poc/tests/test_models.py +import json + +from benchmark.models import ( + BenchmarkTarget, + DownloadResult, + load_benchmark_set, + load_results, + save_benchmark_set, + save_results, +) + + +def test_save_and_load_benchmark_set_round_trip(tmp_path): + path = tmp_path / "data" / "benchmark_set.json" + targets = [ + BenchmarkTarget( + label="big-video-1", + chat="durov", + message_id=123, + link="https://t.me/durov/123", + expected_size_bytes=52_428_800, + ) + ] + + save_benchmark_set(path, targets) + loaded = load_benchmark_set(path) + + assert loaded == targets + assert json.loads(path.read_text())[0]["label"] == "big-video-1" + + +def test_save_and_load_results_round_trip(tmp_path): + path = tmp_path / "data" / "results_telethon.json" + results = [ + DownloadResult( + label="big-video-1", + backend="telethon", + ok=True, + elapsed_seconds=12.5, + bytes_downloaded=52_428_800, + resumed=False, + ) + ] + + save_results(path, results) + loaded = load_results(path) + + assert loaded == results +``` + +```python +# experiments/tdlib-media-poc/tests/test_select_targets.py +import pytest + +from benchmark.select_targets import extract_file_size, parse_link + + +def test_parse_link_public_username(): + chat, message_id = parse_link("https://t.me/durov/123") + assert chat == "durov" + assert message_id == 123 + + +def test_parse_link_private_channel_id(): + chat, message_id = parse_link("https://t.me/c/1234567890/456") + assert chat == "-1001234567890" + assert message_id == 456 + + +def test_parse_link_rejects_non_telegram_url(): + with pytest.raises(ValueError, match="not a recognized t.me post link"): + parse_link("https://example.com/foo/1") + + +def test_extract_file_size_from_tg_message_envelope(): + envelope = { + "ok": True, + "payload": { + "chat": {"id": 123}, + "message_id": 456, + "message": {"id": 456, "file_size": 52428800, "media_type": "video"}, + }, + } + assert extract_file_size(envelope) == 52428800 + + +def test_extract_file_size_returns_none_when_no_media(): + envelope = {"ok": True, "payload": {"message": {"id": 1, "file_size": None}}} + assert extract_file_size(envelope) is None + + +def test_extract_file_size_returns_none_when_message_missing(): + envelope = {"ok": True, "payload": {"message": None}} + assert extract_file_size(envelope) is None +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cd experiments/tdlib-media-poc && uv run pytest tests/test_models.py tests/test_select_targets.py -v` +Expected: FAIL with `ModuleNotFoundError: No module named 'benchmark.models'` (and `benchmark.select_targets`) + +- [ ] **Step 3: Write the minimal implementation** + +```python +# experiments/tdlib-media-poc/benchmark/models.py +"""Data models for the TDLib vs Telethon media-download benchmark.""" + +import json +from dataclasses import asdict, dataclass +from pathlib import Path + + +@dataclass(frozen=True) +class BenchmarkTarget: + label: str + chat: str + message_id: int + link: str + expected_size_bytes: int | None + + +@dataclass(frozen=True) +class DownloadResult: + label: str + backend: str # "telethon" or "tdlib" + ok: bool + elapsed_seconds: float + bytes_downloaded: int | None + resumed: bool + error: str | None = None + + +def load_benchmark_set(path: Path) -> list[BenchmarkTarget]: + raw = json.loads(path.read_text()) + return [BenchmarkTarget(**item) for item in raw] + + +def save_benchmark_set(path: Path, targets: list[BenchmarkTarget]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps([asdict(t) for t in targets], indent=2)) + + +def load_results(path: Path) -> list[DownloadResult]: + raw = json.loads(path.read_text()) + return [DownloadResult(**item) for item in raw] + + +def save_results(path: Path, results: list[DownloadResult]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps([asdict(r) for r in results], indent=2)) +``` + +```python +# experiments/tdlib-media-poc/benchmark/select_targets.py +"""Parse t.me links and pull file-size metadata from `tg message` output.""" + +import re + +LINK_PATTERN = re.compile( + r"^https?://t\.me/(?:c/(?P\d+)|(?P[A-Za-z0-9_]+))/(?P\d+)$" +) + + +def parse_link(link: str) -> tuple[str, int]: + match = LINK_PATTERN.match(link.strip()) + if not match: + raise ValueError(f"not a recognized t.me post link: {link!r}") + message_id = int(match.group("message_id")) + if match.group("internal_id"): + chat = f"-100{match.group('internal_id')}" + else: + chat = match.group("username") + return chat, message_id + + +def extract_file_size(tg_message_envelope: dict) -> int | None: + payload = tg_message_envelope.get("payload") or {} + message = payload.get("message") or {} + return message.get("file_size") +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cd experiments/tdlib-media-poc && uv run pytest tests/test_models.py tests/test_select_targets.py -v` +Expected: PASS (8 tests) + +- [ ] **Step 5: Write `build_benchmark_set.py` (live orchestration script, not unit tested)** + +```python +# experiments/tdlib-media-poc/benchmark/build_benchmark_set.py +"""Build data/benchmark_set.json from operator-supplied t.me links. + +Usage: + uv run python benchmark/build_benchmark_set.py [ ...] + +For each link this shells out to the existing `tg message` CLI (read-only) +to fetch file_size metadata, then writes the isolated benchmark set used by +both the Telethon and TDLib benchmark runners. +""" + +import json +import subprocess +import sys +from pathlib import Path + +from benchmark.models import BenchmarkTarget, save_benchmark_set +from benchmark.select_targets import extract_file_size, parse_link + +POC_ROOT = Path(__file__).resolve().parent.parent +TG_CLI = POC_ROOT.parent.parent / "mcp" / "bin" / "tg" +OUTPUT_PATH = POC_ROOT / "data" / "benchmark_set.json" + + +def fetch_metadata(chat: str, message_id: int) -> dict: + proc = subprocess.run( + [str(TG_CLI), "message", chat, str(message_id), "--json", "--account", "main"], + capture_output=True, + text=True, + check=True, + ) + return json.loads(proc.stdout) + + +def main(links: list[str]) -> None: + targets = [] + for index, link in enumerate(links, start=1): + chat, message_id = parse_link(link) + envelope = fetch_metadata(chat, message_id) + targets.append( + BenchmarkTarget( + label=f"target-{index}", + chat=chat, + message_id=message_id, + link=link, + expected_size_bytes=extract_file_size(envelope), + ) + ) + save_benchmark_set(OUTPUT_PATH, targets) + print(f"Wrote {len(targets)} targets to {OUTPUT_PATH}") + + +if __name__ == "__main__": + if len(sys.argv) < 2: + raise SystemExit("usage: build_benchmark_set.py [ ...]") + main(sys.argv[1:]) +``` + +- [ ] **Step 6: Commit** + +```bash +git add experiments/tdlib-media-poc/benchmark/models.py experiments/tdlib-media-poc/benchmark/select_targets.py \ + experiments/tdlib-media-poc/benchmark/build_benchmark_set.py experiments/tdlib-media-poc/tests/test_models.py \ + experiments/tdlib-media-poc/tests/test_select_targets.py +git commit -m "poc: add benchmark models, link parsing, and target-set builder" +``` + +--- + +### Task 3: Isolated TDLib login for `main` account (interactive, manual) + +**Files:** +- Create: `experiments/tdlib-media-poc/benchmark/login_tdlib.py` + +**Interfaces:** +- Consumes: `benchmark.tdlib_client.build_client` (Task 1). +- Produces: an authenticated TDLib session on disk at `experiments/tdlib-media-poc/data/tdlib/`, consumed by Task 5. + +This task has no automated test — it is a one-time interactive login and must be run live, with the operator present to receive and enter the Telegram login code for the `main` account (confirmed account choice). It does not touch the Telethon session in any way. + +- [ ] **Step 1: Write the login script** + +```python +# experiments/tdlib-media-poc/benchmark/login_tdlib.py +"""One-time interactive TDLib login for the isolated media-download POC. + +Run manually: + uv run python benchmark/login_tdlib.py + +This creates a TDLib session under data/tdlib/, fully separate from the +Telethon session telegram-mcp owns. It requires live phone-code entry for +the `main` account (confirmed with the operator before running this). +""" + +import asyncio +import os +from pathlib import Path + +from dotenv import load_dotenv + +from benchmark.tdlib_client import build_client + +POC_ROOT = Path(__file__).resolve().parent.parent +DATA_DIR = POC_ROOT / "data" / "tdlib" + + +async def main() -> None: + load_dotenv(POC_ROOT / ".env") + api_id = int(os.environ["TELEGRAM_API_ID"]) + api_hash = os.environ["TELEGRAM_API_HASH"] + + client = build_client(api_id=api_id, api_hash=api_hash, files_directory=str(DATA_DIR)) + await client.start() + me = await client.getMe() + print(f"Logged in: {me}") + await client.stop() + + +if __name__ == "__main__": + asyncio.run(main()) +``` + +- [ ] **Step 2: Run it live with the operator present** + +Run: `cd experiments/tdlib-media-poc && uv run python benchmark/login_tdlib.py` +Expected: TDLib prompts for phone number (or reuses saved `+`-prefixed number if asked), then a login code delivered to the `main` account's other active Telegram sessions. Operator enters the code interactively. Script prints `Logged in: ...` with the `main` account's user object and exits cleanly. + +- [ ] **Step 3: Verify isolation held** + +Run: `ls -la ~/.telegram-mcp/ 2>/dev/null | head -5` before and after — file listing and mtimes must be identical (no Telethon files touched). Then: `ls experiments/tdlib-media-poc/data/tdlib/` — must show TDLib's own database/file-cache directory tree that did not exist before this step. + +- [ ] **Step 4: Commit** + +```bash +git add experiments/tdlib-media-poc/benchmark/login_tdlib.py +git commit -m "poc: add isolated interactive tdlib login script" +``` + +(`data/` stays gitignored — the authenticated session itself is never committed.) + +--- + +### Task 4: Telethon baseline download benchmark + +**Files:** +- Create: `experiments/tdlib-media-poc/benchmark/run_telethon.py` +- Test: `experiments/tdlib-media-poc/tests/test_run_telethon.py` + +**Interfaces:** +- Consumes: `benchmark.models.BenchmarkTarget/DownloadResult/load_benchmark_set/save_results` (Task 2). +- Produces: `benchmark.run_telethon.build_telethon_result(target, elapsed_seconds, envelope, downloaded_size_bytes) -> DownloadResult`, and `data/results_telethon.json` when run live. Consumed by Task 6. + +- [ ] **Step 1: Write the failing test for the pure result-builder** + +```python +# experiments/tdlib-media-poc/tests/test_run_telethon.py +from benchmark.models import BenchmarkTarget +from benchmark.run_telethon import build_telethon_result + +TARGET = BenchmarkTarget( + label="big-video-1", + chat="durov", + message_id=123, + link="https://t.me/durov/123", + expected_size_bytes=52_428_800, +) + + +def test_build_telethon_result_success(): + envelope = {"ok": True, "payload": {"local_path": "/tmp/foo.mp4"}} + result = build_telethon_result(TARGET, 12.5, envelope, 52_428_800) + + assert result.ok is True + assert result.backend == "telethon" + assert result.elapsed_seconds == 12.5 + assert result.bytes_downloaded == 52_428_800 + assert result.resumed is False + assert result.error is None + + +def test_build_telethon_result_failure_from_bad_envelope(): + envelope = {"ok": False, "error": "flood_wait"} + result = build_telethon_result(TARGET, 3.0, envelope, None) + + assert result.ok is False + assert result.bytes_downloaded is None + assert result.error == "flood_wait" +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd experiments/tdlib-media-poc && uv run pytest tests/test_run_telethon.py -v` +Expected: FAIL with `ModuleNotFoundError: No module named 'benchmark.run_telethon'` + +- [ ] **Step 3: Write the implementation** + +```python +# experiments/tdlib-media-poc/benchmark/run_telethon.py +"""Benchmark media downloads through the current telegram-mcp path. + +Live usage: + uv run python benchmark/run_telethon.py + +Shells out to the existing `tg download` CLI (direct Telethon, no 120s MCP +cap — see mcp/src/telegram_mcp/tg_cli.py:cmd_download) for each target in +data/benchmark_set.json, times it, and records results. +""" + +import json +import subprocess +import time +from pathlib import Path + +from benchmark.models import BenchmarkTarget, DownloadResult, load_benchmark_set, save_results + +POC_ROOT = Path(__file__).resolve().parent.parent +TG_CLI = POC_ROOT.parent.parent / "mcp" / "bin" / "tg" +BENCHMARK_SET_PATH = POC_ROOT / "data" / "benchmark_set.json" +RESULTS_PATH = POC_ROOT / "data" / "results_telethon.json" +DOWNLOAD_DEST = POC_ROOT / "data" / "downloads" / "telethon" + + +def build_telethon_result( + target: BenchmarkTarget, + elapsed_seconds: float, + envelope: dict, + downloaded_size_bytes: int | None, +) -> DownloadResult: + ok = bool(envelope.get("ok")) and downloaded_size_bytes is not None + error = None + if not ok: + error = str(envelope.get("error") or envelope.get("payload") or "download failed") + return DownloadResult( + label=target.label, + backend="telethon", + ok=ok, + elapsed_seconds=elapsed_seconds, + bytes_downloaded=downloaded_size_bytes, + resumed=False, + error=error, + ) + + +def download_one(target: BenchmarkTarget) -> DownloadResult: + DOWNLOAD_DEST.mkdir(parents=True, exist_ok=True) + started = time.perf_counter() + proc = subprocess.run( + [str(TG_CLI), "download", target.link, "--dest", str(DOWNLOAD_DEST), "--account", "main", "--json"], + capture_output=True, + text=True, + ) + elapsed = time.perf_counter() - started + try: + envelope = json.loads(proc.stdout) + except json.JSONDecodeError: + envelope = {"ok": False, "error": proc.stderr.strip() or "no JSON output"} + + downloaded_size = None + payload = envelope.get("payload") if isinstance(envelope.get("payload"), dict) else {} + local_path = payload.get("local_path") or payload.get("path") + if local_path and Path(local_path).exists(): + downloaded_size = Path(local_path).stat().st_size + + return build_telethon_result(target, elapsed, envelope, downloaded_size) + + +def main() -> None: + targets = load_benchmark_set(BENCHMARK_SET_PATH) + results = [download_one(target) for target in targets] + save_results(RESULTS_PATH, results) + for result in results: + print(f"{result.label}: ok={result.ok} elapsed={result.elapsed_seconds:.2f}s bytes={result.bytes_downloaded}") + + +if __name__ == "__main__": + main() +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd experiments/tdlib-media-poc && uv run pytest tests/test_run_telethon.py -v` +Expected: PASS (2 tests) + +- [ ] **Step 5: Run live against real targets (manual verification)** + +Prerequisite: Task 2's `build_benchmark_set.py` has been run with real t.me links pointing at large media in the `main` account, producing `data/benchmark_set.json`. + +Run: `cd experiments/tdlib-media-poc && uv run python benchmark/run_telethon.py` +Expected: one line per target printed with `ok=True`, a non-zero `elapsed`, and `bytes` matching the target's `expected_size_bytes`; `data/results_telethon.json` created. + +- [ ] **Step 6: Commit** + +```bash +git add experiments/tdlib-media-poc/benchmark/run_telethon.py experiments/tdlib-media-poc/tests/test_run_telethon.py +git commit -m "poc: add telethon baseline download benchmark runner" +``` + +--- + +### Task 5: TDLib message resolution + download benchmark with resumability test + +**Files:** +- Create: `experiments/tdlib-media-poc/benchmark/tdlib_message.py` +- Create: `experiments/tdlib-media-poc/benchmark/run_tdlib.py` +- Test: `experiments/tdlib-media-poc/tests/test_tdlib_message.py` + +**Interfaces:** +- Consumes: `benchmark.models.BenchmarkTarget/DownloadResult/load_benchmark_set/save_results` (Task 2), `benchmark.tdlib_client.build_client` (Task 1), authenticated session from Task 3. +- Produces: `benchmark.tdlib_message.extract_file_id_from_message(message: dict) -> int`, `benchmark.run_tdlib.build_tdlib_result(target, elapsed_seconds, file_object, resumed) -> DownloadResult`, and `data/results_tdlib.json` when run live. Consumed by Task 6. + +- [ ] **Step 1: Write the failing tests for pure TDLib-schema logic** + +```python +# experiments/tdlib-media-poc/tests/test_tdlib_message.py +import pytest + +from benchmark.tdlib_message import extract_file_id_from_message +from benchmark.run_tdlib import build_tdlib_result +from benchmark.models import BenchmarkTarget + +TARGET = BenchmarkTarget( + label="big-video-1", + chat="durov", + message_id=123, + link="https://t.me/durov/123", + expected_size_bytes=52_428_800, +) + + +def test_extract_file_id_from_message_video(): + message = { + "content": { + "@type": "messageVideo", + "video": {"video": {"id": 555, "size": 52_428_800}}, + } + } + assert extract_file_id_from_message(message) == 555 + + +def test_extract_file_id_from_message_document(): + message = { + "content": { + "@type": "messageDocument", + "document": {"document": {"id": 777, "size": 10_000_000}}, + } + } + assert extract_file_id_from_message(message) == 777 + + +def test_extract_file_id_from_message_unsupported_type(): + message = {"content": {"@type": "messageText"}} + with pytest.raises(ValueError, match="unsupported message content type"): + extract_file_id_from_message(message) + + +def test_build_tdlib_result_completed_download(): + file_object = { + "id": 555, + "size": 52_428_800, + "local": {"downloaded_size": 52_428_800, "is_downloading_completed": True}, + } + result = build_tdlib_result(TARGET, 8.0, file_object, resumed=True) + + assert result.ok is True + assert result.backend == "tdlib" + assert result.bytes_downloaded == 52_428_800 + assert result.resumed is True + assert result.error is None + + +def test_build_tdlib_result_incomplete_download(): + file_object = { + "id": 555, + "size": 52_428_800, + "local": {"downloaded_size": 1_000_000, "is_downloading_completed": False}, + } + result = build_tdlib_result(TARGET, 3.0, file_object, resumed=False) + + assert result.ok is False + assert result.bytes_downloaded == 1_000_000 + assert result.error == "download did not complete" +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `cd experiments/tdlib-media-poc && uv run pytest tests/test_tdlib_message.py -v` +Expected: FAIL with `ModuleNotFoundError: No module named 'benchmark.tdlib_message'` + +- [ ] **Step 3: Write the implementation** + +```python +# experiments/tdlib-media-poc/benchmark/tdlib_message.py +"""Pure helpers for pulling downloadable file_ids out of TDLib Message objects. + +TDLib content schema reference: https://core.telegram.org/tdlib/docs/classtd_1_1td__api_1_1_message.html +""" + + +def extract_file_id_from_message(message: dict) -> int: + content = message.get("content") or {} + content_type = content.get("@type", "") + if content_type == "messageVideo": + return content["video"]["video"]["id"] + if content_type == "messageDocument": + return content["document"]["document"]["id"] + if content_type == "messagePhoto": + sizes = content["photo"]["sizes"] + return sizes[-1]["photo"]["id"] + if content_type == "messageAudio": + return content["audio"]["audio"]["id"] + raise ValueError(f"unsupported message content type for download: {content_type!r}") +``` + +```python +# experiments/tdlib-media-poc/benchmark/run_tdlib.py +"""Benchmark media downloads through TDLib, including a resume test. + +Live usage: + uv run python benchmark/run_tdlib.py + +Requires the isolated session created by benchmark/login_tdlib.py. For each +target: resolves the t.me link via getMessageLinkInfo, fetches the Message, +extracts the file_id, downloads it, then re-runs the same download once more +after a mid-flight cancel to prove resumability. +""" + +import asyncio +import os +import time +from pathlib import Path + +from dotenv import load_dotenv + +from benchmark.models import BenchmarkTarget, DownloadResult, load_benchmark_set, save_results +from benchmark.tdlib_client import build_client +from benchmark.tdlib_message import extract_file_id_from_message + +POC_ROOT = Path(__file__).resolve().parent.parent +DATA_DIR = POC_ROOT / "data" / "tdlib" +BENCHMARK_SET_PATH = POC_ROOT / "data" / "benchmark_set.json" +RESULTS_PATH = POC_ROOT / "data" / "results_tdlib.json" + + +def build_tdlib_result( + target: BenchmarkTarget, + elapsed_seconds: float, + file_object: dict, + resumed: bool, +) -> DownloadResult: + local = file_object.get("local") or {} + ok = bool(local.get("is_downloading_completed")) + return DownloadResult( + label=target.label, + backend="tdlib", + ok=ok, + elapsed_seconds=elapsed_seconds, + bytes_downloaded=local.get("downloaded_size"), + resumed=resumed, + error=None if ok else "download did not complete", + ) + + +async def resolve_file_id(client, target: BenchmarkTarget) -> int: + link_info = await client.getMessageLinkInfo(url=target.link) + message = await client.getMessage( + chat_id=link_info["chat_id"], message_id=link_info["message"]["id"] + ) + return extract_file_id_from_message(message.to_dict() if hasattr(message, "to_dict") else message) + + +async def measure_clean_download(client, file_id: int) -> tuple[float, dict]: + """Single-shot download latency, directly comparable to the Telethon baseline.""" + started = time.perf_counter() + result = await client.downloadFile(file_id=file_id, priority=1, synchronous=True, offset=0, limit=0) + elapsed = time.perf_counter() - started + file_dict = result.to_dict() if hasattr(result, "to_dict") else result + return elapsed, file_dict + + +async def measure_resumability(client, file_id: int) -> bool: + """Delete the local copy, start a fresh async download, cancel it mid-flight + while it is still partial, then confirm a second call resumes from the + partial bytes already on disk instead of restarting from zero.""" + await client.deleteFile(file_id=file_id) + await client.downloadFile(file_id=file_id, priority=1, synchronous=False, offset=0, limit=0) + await asyncio.sleep(2) + + partial = await client.getFile(file_id=file_id) + partial_dict = partial.to_dict() if hasattr(partial, "to_dict") else partial + partial_bytes = (partial_dict.get("local") or {}).get("downloaded_size", 0) + + await client.cancelDownloadFile(file_id=file_id, only_if_pending=False) + + resumed_result = await client.downloadFile(file_id=file_id, priority=1, synchronous=True, offset=0, limit=0) + resumed_dict = resumed_result.to_dict() if hasattr(resumed_result, "to_dict") else resumed_result + resumed_local = resumed_dict.get("local") or {} + completed = bool(resumed_local.get("is_downloading_completed")) + grew_from_partial = resumed_local.get("downloaded_size", 0) >= partial_bytes + + return completed and partial_bytes > 0 and grew_from_partial + + +async def run(targets: list[BenchmarkTarget]) -> list[DownloadResult]: + load_dotenv(POC_ROOT / ".env") + client = build_client( + api_id=int(os.environ["TELEGRAM_API_ID"]), + api_hash=os.environ["TELEGRAM_API_HASH"], + files_directory=str(DATA_DIR), + ) + await client.start() + + results = [] + for target in targets: + file_id = await resolve_file_id(client, target) + elapsed, file_object = await measure_clean_download(client, file_id) + resumed_ok = await measure_resumability(client, file_id) + results.append(build_tdlib_result(target, elapsed, file_object, resumed=resumed_ok)) + + await client.stop() + return results + + +def main() -> None: + targets = load_benchmark_set(BENCHMARK_SET_PATH) + results = asyncio.run(run(targets)) + save_results(RESULTS_PATH, results) + for result in results: + print(f"{result.label}: ok={result.ok} elapsed={result.elapsed_seconds:.2f}s resumed={result.resumed}") + + +if __name__ == "__main__": + main() +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `cd experiments/tdlib-media-poc && uv run pytest tests/test_tdlib_message.py -v` +Expected: PASS (5 tests) + +- [ ] **Step 5: Run live against real targets (manual verification)** + +Prerequisite: Task 3's login completed successfully. + +Run: `cd experiments/tdlib-media-poc && uv run python benchmark/run_tdlib.py` +Expected: one line per target with `ok=True` and `resumed=True`; `data/results_tdlib.json` created. `getMessageLinkInfo`, `getMessage`, `getFile`, and `deleteFile` are standard, stable TDLib API methods that pytdbot generates dynamically from TDLib's schema, but — unlike `downloadFile`/`cancelDownloadFile`/`getMe`/`start` — they were not directly invoked during plan verification. If any raises `AttributeError`, run `uv run python -c "import pytdbot; print([m for m in dir(pytdbot.Client) if not m.startswith('_')])"` to list the real method names on the installed pytdbot version and adjust `run_tdlib.py` accordingly. + +- [ ] **Step 6: Commit** + +```bash +git add experiments/tdlib-media-poc/benchmark/tdlib_message.py experiments/tdlib-media-poc/benchmark/run_tdlib.py \ + experiments/tdlib-media-poc/tests/test_tdlib_message.py +git commit -m "poc: add tdlib download benchmark runner with resume check" +``` + +--- + +### Task 6: Compare results and generate the POC report + +**Files:** +- Create: `experiments/tdlib-media-poc/benchmark/compare.py` +- Test: `experiments/tdlib-media-poc/tests/test_compare.py` + +**Interfaces:** +- Consumes: `benchmark.models.DownloadResult/load_results` (Task 2), `data/results_telethon.json` (Task 4), `data/results_tdlib.json` (Task 5). +- Produces: `data/RESULTS.md`. + +- [ ] **Step 1: Write the failing test** + +```python +# experiments/tdlib-media-poc/tests/test_compare.py +from benchmark.compare import build_report +from benchmark.models import DownloadResult + + +def test_build_report_includes_table_rows_and_averages(): + telethon = [DownloadResult("big-video-1", "telethon", True, 10.0, 52428800, False)] + tdlib = [DownloadResult("big-video-1", "tdlib", True, 8.0, 52428800, True)] + + report = build_report(telethon, tdlib) + + assert "big-video-1" in report + assert "telethon=10.00s, tdlib=8.00s" in report + assert "resume after interruption: True" in report + + +def test_build_report_handles_no_successful_runs(): + telethon = [DownloadResult("big-video-1", "telethon", False, 0.0, None, False, error="boom")] + tdlib = [DownloadResult("big-video-1", "tdlib", False, 0.0, None, False, error="boom")] + + report = build_report(telethon, tdlib) + + assert "Not enough successful runs" in report +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd experiments/tdlib-media-poc && uv run pytest tests/test_compare.py -v` +Expected: FAIL with `ModuleNotFoundError: No module named 'benchmark.compare'` + +- [ ] **Step 3: Write the implementation** + +```python +# experiments/tdlib-media-poc/benchmark/compare.py +"""Compare Telethon vs TDLib download benchmark results and write RESULTS.md. + +Live usage: + uv run python benchmark/compare.py +""" + +from pathlib import Path +from statistics import mean + +from benchmark.models import DownloadResult, load_results + +POC_ROOT = Path(__file__).resolve().parent.parent +TELETHON_RESULTS_PATH = POC_ROOT / "data" / "results_telethon.json" +TDLIB_RESULTS_PATH = POC_ROOT / "data" / "results_tdlib.json" +REPORT_PATH = POC_ROOT / "data" / "RESULTS.md" + + +def build_report(telethon_results: list[DownloadResult], tdlib_results: list[DownloadResult]) -> str: + lines = [ + "# TDLib vs Telethon: media download latency/resumability POC results", + "", + "| label | backend | ok | elapsed_seconds | bytes_downloaded | resumed |", + "|---|---|---|---|---|---|", + ] + for result in [*telethon_results, *tdlib_results]: + lines.append( + f"| {result.label} | {result.backend} | {result.ok} | {result.elapsed_seconds:.2f} " + f"| {result.bytes_downloaded} | {result.resumed} |" + ) + + telethon_ok = [r for r in telethon_results if r.ok] + tdlib_ok = [r for r in tdlib_results if r.ok] + lines.append("") + if telethon_ok and tdlib_ok: + telethon_avg = mean(r.elapsed_seconds for r in telethon_ok) + tdlib_avg = mean(r.elapsed_seconds for r in tdlib_ok) + delta_pct = ((telethon_avg - tdlib_avg) / telethon_avg) * 100 + lines.append( + f"Average elapsed: telethon={telethon_avg:.2f}s, tdlib={tdlib_avg:.2f}s " + f"({delta_pct:+.1f}% telethon vs tdlib)." + ) + else: + lines.append("Not enough successful runs on both backends to compare averages.") + + tdlib_resumed = any(r.resumed and r.ok for r in tdlib_results) + lines.append(f"TDLib demonstrated successful resume after interruption: {tdlib_resumed}.") + + lines.append("") + lines.append("## ADR kill-criteria checklist (manual assessment)") + lines.append("- [ ] Required sharing/converting Telethon session files? (must be No)") + lines.append("- [ ] Auth/DB/update-loop code became the main work? (must be No)") + lines.append("- [ ] No clear measured advantage over telegram-mcp? (see averages above)") + lines.append("- [ ] Read behavior diverged from telegram-mcp in a way agents would need to understand? (must be No)") + lines.append("- [ ] Required new persistent daemon management before proving value? (must be No)") + + return "\n".join(lines) + + +def main() -> None: + telethon_results = load_results(TELETHON_RESULTS_PATH) + tdlib_results = load_results(TDLIB_RESULTS_PATH) + report = build_report(telethon_results, tdlib_results) + REPORT_PATH.write_text(report) + print(f"Wrote {REPORT_PATH}") + + +if __name__ == "__main__": + main() +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd experiments/tdlib-media-poc && uv run pytest tests/test_compare.py -v` +Expected: PASS (2 tests) + +- [ ] **Step 5: Run live to generate the final report (manual verification)** + +Prerequisite: Tasks 4 and 5 have both been run live and produced their result JSON files. + +Run: `cd experiments/tdlib-media-poc && uv run python benchmark/compare.py` +Expected: `data/RESULTS.md` written with a filled-in comparison table and averages; manually fill in the kill-criteria checklist based on how Tasks 3–5 actually went before treating the POC as concluded. + +- [ ] **Step 6: Run the full test suite** + +Run: `cd experiments/tdlib-media-poc && uv run pytest tests/ -v` +Expected: PASS (all tests across all tasks, 20 tests total) + +- [ ] **Step 7: Commit** + +```bash +git add experiments/tdlib-media-poc/benchmark/compare.py experiments/tdlib-media-poc/tests/test_compare.py +git commit -m "poc: add results comparison and RESULTS.md generator" +``` diff --git a/docs/operator-workflows.md b/docs/operator-workflows.md index f379bcc..038c44c 100644 --- a/docs/operator-workflows.md +++ b/docs/operator-workflows.md @@ -116,3 +116,43 @@ Use subscriber export only with explicit user intent, private local output paths, and clear reporting of `visible_count`, `exported_count`, `missing`, and completeness caveats. Do not treat a single `get_participants` slice as a full export. + +## TDLib Large-Media Downloads (main account only) + +`tg download` (and `download_post()` under it) can optionally route large +media downloads on the `main` account through TDLib instead of Telethon, +based on a measured advantage confirmed in a live POC (+78.7% faster average +elapsed, resumability confirmed on 3 real files — see +`mcp/docs/superpowers/specs/2026-07-01-tdlib-large-media-download-design.md`). +This does **not** change any other Telegram operation or account: reads, +search, sends, and MCP tool downloads (`download_media_batch`, +`download_dialog_media`) stay on Telethon unchanged. + +The capability stays off until explicitly enabled. Rollout: + +1. Install the optional extra: `pip install -e ".[tdlib]"` (from `mcp/`). +2. Run the one-time interactive login for `main`: + ``` + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --phone + + # then, after Telegram sends a code to another active session: + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --code + # only if 2FA is enabled: + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --password + ``` +3. Set in `main`'s env (`~/.telegram-mcp/launchd.env` or `mcp/.env`): + ``` + TELEGRAM_TDLIB_ENABLED=true + ``` + Optional tuning: `TELEGRAM_TDLIB_SESSION_DIR` (default + `~/.telegram-mcp-tdlib/main`), `TELEGRAM_TDLIB_DOWNLOAD_THRESHOLD_MB` + (default `20`). +4. Watch telemetry (`download_post_backend` events: `backend`, + `route_attempted`, `fallback_reason`) for the backend-used distribution + and fallback rate before considering wider rollout (other accounts, lower + threshold) — each of those is a separate future decision, not part of + this change. + +Every TDLib failure mode (session not authorized, network error, unsupported +content type, lock not acquired within 5s) falls back to Telethon +automatically using the connection already open for routing — there is no +new failure mode a user can hit from this change. diff --git a/experiments/tdlib-media-poc/.env.example b/experiments/tdlib-media-poc/.env.example new file mode 100644 index 0000000..c010f21 --- /dev/null +++ b/experiments/tdlib-media-poc/.env.example @@ -0,0 +1,3 @@ +# Reuse the same values as mcp/.env (same my.telegram.org app, safe to share). +TELEGRAM_API_ID= +TELEGRAM_API_HASH= diff --git a/experiments/tdlib-media-poc/.gitignore b/experiments/tdlib-media-poc/.gitignore new file mode 100644 index 0000000..8d1838c --- /dev/null +++ b/experiments/tdlib-media-poc/.gitignore @@ -0,0 +1,6 @@ +data/ +.env +__pycache__/ +*.pyc +.pytest_cache/ +.venv/ diff --git a/experiments/tdlib-media-poc/README.md b/experiments/tdlib-media-poc/README.md new file mode 100644 index 0000000..f71dccc --- /dev/null +++ b/experiments/tdlib-media-poc/README.md @@ -0,0 +1,36 @@ +# TDLib media-download POC + +Isolated lab proof of concept required by +[control-plane/docs/adr/2026-06-21-tdlib-is-not-default-runtime.md](../../control-plane/docs/adr/2026-06-21-tdlib-is-not-default-runtime.md). + +Scenario: media download latency/resumability, measured against the current +`telegram-mcp` (Telethon) path, on the `main` account only. + +## Isolation guarantees + +- TDLib state lives at `data/tdlib/` (gitignored). It is never pointed at the + Telethon session tree (`~/.telegram-mcp/`). +- Read-only Telegram operations only. +- Not wired into `mcp/`, `plugin/`, LaunchAgents, or release gates. + +## Setup + +```bash +cd experiments/tdlib-media-poc +uv sync +cp .env.example .env # fill in TELEGRAM_API_ID / TELEGRAM_API_HASH from mcp/.env +``` + +## Run order + +1. `uv run python benchmark/build_benchmark_set.py [ ...]` +2. `uv run python benchmark/login_tdlib.py` (one-time, interactive — needs a live login code) +3. `uv run python benchmark/run_telethon.py` +4. `uv run python benchmark/run_tdlib.py` +5. `uv run python benchmark/compare.py` → writes `data/RESULTS.md` + +## Tests + +```bash +uv run pytest tests/ -v +``` diff --git a/experiments/tdlib-media-poc/benchmark/__init__.py b/experiments/tdlib-media-poc/benchmark/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/experiments/tdlib-media-poc/benchmark/build_benchmark_set.py b/experiments/tdlib-media-poc/benchmark/build_benchmark_set.py new file mode 100644 index 0000000..3cbeb94 --- /dev/null +++ b/experiments/tdlib-media-poc/benchmark/build_benchmark_set.py @@ -0,0 +1,55 @@ +"""Build data/benchmark_set.json from operator-supplied t.me links. + +Usage: + uv run python benchmark/build_benchmark_set.py [ ...] + +For each link this shells out to the existing `tg message` CLI (read-only) +to fetch file_size metadata, then writes the isolated benchmark set used by +both the Telethon and TDLib benchmark runners. +""" + +import json +import subprocess +import sys +from pathlib import Path + +from benchmark.models import BenchmarkTarget, save_benchmark_set +from benchmark.select_targets import extract_file_size, parse_link + +POC_ROOT = Path(__file__).resolve().parent.parent +TG_CLI = POC_ROOT.parent.parent / "mcp" / "bin" / "tg" +OUTPUT_PATH = POC_ROOT / "data" / "benchmark_set.json" + + +def fetch_metadata(chat: str, message_id: int) -> dict: + proc = subprocess.run( + [str(TG_CLI), "message", chat, str(message_id), "--json", "--account", "main"], + capture_output=True, + text=True, + check=True, + ) + return json.loads(proc.stdout) + + +def main(links: list[str]) -> None: + targets = [] + for index, link in enumerate(links, start=1): + chat, message_id = parse_link(link) + envelope = fetch_metadata(chat, message_id) + targets.append( + BenchmarkTarget( + label=f"target-{index}", + chat=chat, + message_id=message_id, + link=link, + expected_size_bytes=extract_file_size(envelope), + ) + ) + save_benchmark_set(OUTPUT_PATH, targets) + print(f"Wrote {len(targets)} targets to {OUTPUT_PATH}") + + +if __name__ == "__main__": + if len(sys.argv) < 2: + raise SystemExit("usage: build_benchmark_set.py [ ...]") + main(sys.argv[1:]) diff --git a/experiments/tdlib-media-poc/benchmark/compare.py b/experiments/tdlib-media-poc/benchmark/compare.py new file mode 100644 index 0000000..2b4e55a --- /dev/null +++ b/experiments/tdlib-media-poc/benchmark/compare.py @@ -0,0 +1,68 @@ +"""Compare Telethon vs TDLib download benchmark results and write RESULTS.md. + +Live usage: + uv run python benchmark/compare.py +""" + +from pathlib import Path +from statistics import mean + +from benchmark.models import DownloadResult, load_results + +POC_ROOT = Path(__file__).resolve().parent.parent +TELETHON_RESULTS_PATH = POC_ROOT / "data" / "results_telethon.json" +TDLIB_RESULTS_PATH = POC_ROOT / "data" / "results_tdlib.json" +REPORT_PATH = POC_ROOT / "data" / "RESULTS.md" + + +def build_report(telethon_results: list[DownloadResult], tdlib_results: list[DownloadResult]) -> str: + lines = [ + "# TDLib vs Telethon: media download latency/resumability POC results", + "", + "| label | backend | ok | elapsed_seconds | bytes_downloaded | resumed |", + "|---|---|---|---|---|---|", + ] + for result in [*telethon_results, *tdlib_results]: + lines.append( + f"| {result.label} | {result.backend} | {result.ok} | {result.elapsed_seconds:.2f} " + f"| {result.bytes_downloaded} | {result.resumed} |" + ) + + telethon_ok = [r for r in telethon_results if r.ok] + tdlib_ok = [r for r in tdlib_results if r.ok] + lines.append("") + if telethon_ok and tdlib_ok: + telethon_avg = mean(r.elapsed_seconds for r in telethon_ok) + tdlib_avg = mean(r.elapsed_seconds for r in tdlib_ok) + delta_pct = ((telethon_avg - tdlib_avg) / telethon_avg) * 100 + lines.append( + f"Average elapsed: telethon={telethon_avg:.2f}s, tdlib={tdlib_avg:.2f}s " + f"({delta_pct:+.1f}% telethon vs tdlib)." + ) + else: + lines.append("Not enough successful runs on both backends to compare averages.") + + tdlib_resumed = any(r.resumed and r.ok for r in tdlib_results) + lines.append(f"TDLib demonstrated successful resume after interruption: {tdlib_resumed}.") + + lines.append("") + lines.append("## ADR kill-criteria checklist (manual assessment)") + lines.append("- [ ] Required sharing/converting Telethon session files? (must be No)") + lines.append("- [ ] Auth/DB/update-loop code became the main work? (must be No)") + lines.append("- [ ] No clear measured advantage over telegram-mcp? (see averages above)") + lines.append("- [ ] Read behavior diverged from telegram-mcp in a way agents would need to understand? (must be No)") + lines.append("- [ ] Required new persistent daemon management before proving value? (must be No)") + + return "\n".join(lines) + + +def main() -> None: + telethon_results = load_results(TELETHON_RESULTS_PATH) + tdlib_results = load_results(TDLIB_RESULTS_PATH) + report = build_report(telethon_results, tdlib_results) + REPORT_PATH.write_text(report) + print(f"Wrote {REPORT_PATH}") + + +if __name__ == "__main__": + main() diff --git a/experiments/tdlib-media-poc/benchmark/login_tdlib.py b/experiments/tdlib-media-poc/benchmark/login_tdlib.py new file mode 100644 index 0000000..a45c3c2 --- /dev/null +++ b/experiments/tdlib-media-poc/benchmark/login_tdlib.py @@ -0,0 +1,105 @@ +"""Interactive-by-invocation TDLib login for the isolated media-download POC. + +pytdbot's Client.start() only auto-drives the authorization state machine +for bot-token logins (see pytdbot.Client.__handle_authorization_state_wait_phone_number, +which no-ops unless a bot token is set). For a real user account it does +nothing at authorizationStateWaitPhoneNumber, so this script drives the +state machine manually across multiple invocations, since a live phone/SMS +code can't be fed into a blocking input() call from outside a real TTY: + + uv run python benchmark/login_tdlib.py --phone +15551234567 + # (Telegram sends a login code to your other active sessions) + uv run python benchmark/login_tdlib.py --code 12345 + # (only if the account has 2FA enabled) + uv run python benchmark/login_tdlib.py --password ... + +Each invocation reconnects to the same persistent session under data/tdlib/, +fully separate from the Telethon session telegram-mcp owns. +""" + +import argparse +import asyncio +import os +from pathlib import Path + +import pytdbot +from dotenv import load_dotenv + +from benchmark.tdlib_client import build_client + +POC_ROOT = Path(__file__).resolve().parent.parent +DATA_DIR = POC_ROOT / "data" / "tdlib" + +_STATE_TO_REQUIRED_ARG = { + "authorizationStateWaitPhoneNumber": "phone", + "authorizationStateWaitCode": "code", + "authorizationStateWaitPassword": "password", +} + + +def required_arg_for_state(state: str) -> str | None: + return _STATE_TO_REQUIRED_ARG.get(state) + + +def raise_if_error(result): + if isinstance(result, pytdbot.types.Error): + raise RuntimeError(f"TDLib error {result['code']}: {result['message']}") + return result + + +async def wait_for_stable_state(client: pytdbot.Client) -> str: + for _ in range(20): + state = client.authorization_state + if state and state != "authorizationStateWaitTdlibParameters": + return state + await asyncio.sleep(0.5) + return client.authorization_state + + +async def main(phone: str | None, code: str | None, password: str | None) -> None: + load_dotenv(POC_ROOT / ".env") + api_id = int(os.environ["TELEGRAM_API_ID"]) + api_hash = os.environ["TELEGRAM_API_HASH"] + + client = build_client(api_id=api_id, api_hash=api_hash, files_directory=str(DATA_DIR)) + await client.start(wait_login=False) + + state = await wait_for_stable_state(client) + print(f"Current authorization state: {state}") + + required_arg = required_arg_for_state(state) + supplied = {"phone": phone, "code": code, "password": password}.get(required_arg) + + if required_arg and not supplied: + print( + f"Need --{required_arg}. Re-run: " + f"uv run python benchmark/login_tdlib.py --{required_arg} " + ) + await client.stop() + return + + if state == "authorizationStateWaitPhoneNumber": + raise_if_error(await client.setAuthenticationPhoneNumber(phone_number=phone)) + print("Code requested. Check Telegram on your other devices, then run with --code.") + elif state == "authorizationStateWaitCode": + raise_if_error(await client.checkAuthenticationCode(code=code)) + print("Code accepted.") + elif state == "authorizationStateWaitPassword": + raise_if_error(await client.checkAuthenticationPassword(password=password)) + print("Password accepted.") + elif state == "authorizationStateReady": + me = raise_if_error(await client.getMe()) + print(f"Already logged in: {me}") + + await asyncio.sleep(1) + print(f"Authorization state now: {client.authorization_state}") + await client.stop() + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--phone") + parser.add_argument("--code") + parser.add_argument("--password") + args = parser.parse_args() + asyncio.run(main(args.phone, args.code, args.password)) diff --git a/experiments/tdlib-media-poc/benchmark/models.py b/experiments/tdlib-media-poc/benchmark/models.py new file mode 100644 index 0000000..bcecf9e --- /dev/null +++ b/experiments/tdlib-media-poc/benchmark/models.py @@ -0,0 +1,45 @@ +"""Data models for the TDLib vs Telethon media-download benchmark.""" + +import json +from dataclasses import asdict, dataclass +from pathlib import Path + + +@dataclass(frozen=True) +class BenchmarkTarget: + label: str + chat: str + message_id: int + link: str + expected_size_bytes: int | None + + +@dataclass(frozen=True) +class DownloadResult: + label: str + backend: str # "telethon" or "tdlib" + ok: bool + elapsed_seconds: float + bytes_downloaded: int | None + resumed: bool + error: str | None = None + + +def load_benchmark_set(path: Path) -> list[BenchmarkTarget]: + raw = json.loads(path.read_text()) + return [BenchmarkTarget(**item) for item in raw] + + +def save_benchmark_set(path: Path, targets: list[BenchmarkTarget]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps([asdict(t) for t in targets], indent=2)) + + +def load_results(path: Path) -> list[DownloadResult]: + raw = json.loads(path.read_text()) + return [DownloadResult(**item) for item in raw] + + +def save_results(path: Path, results: list[DownloadResult]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps([asdict(r) for r in results], indent=2)) diff --git a/experiments/tdlib-media-poc/benchmark/run_tdlib.py b/experiments/tdlib-media-poc/benchmark/run_tdlib.py new file mode 100644 index 0000000..02cfb76 --- /dev/null +++ b/experiments/tdlib-media-poc/benchmark/run_tdlib.py @@ -0,0 +1,131 @@ +"""Benchmark media downloads through TDLib, including a resume test. + +Live usage: + uv run python benchmark/run_tdlib.py + +Requires the isolated session created by benchmark/login_tdlib.py. For each +target: resolves the t.me link via getMessageLinkInfo, fetches the Message, +extracts the file_id, downloads it, then re-runs the same download once more +after a mid-flight cancel to prove resumability. +""" + +import asyncio +import os +import time +from pathlib import Path + +import pytdbot +from dotenv import load_dotenv + +from benchmark.models import BenchmarkTarget, DownloadResult, load_benchmark_set, save_results +from benchmark.tdlib_client import build_client +from benchmark.tdlib_message import extract_file_id_from_message + +POC_ROOT = Path(__file__).resolve().parent.parent +DATA_DIR = POC_ROOT / "data" / "tdlib" +BENCHMARK_SET_PATH = POC_ROOT / "data" / "benchmark_set.json" +RESULTS_PATH = POC_ROOT / "data" / "results_tdlib.json" + + +def raise_if_error(result): + if isinstance(result, pytdbot.types.Error): + raise RuntimeError(f"TDLib error {result['code']}: {result['message']}") + return result + + +def build_tdlib_result( + target: BenchmarkTarget, + elapsed_seconds: float, + file_object, + resumed: bool, +) -> DownloadResult: + local = file_object["local"] + ok = bool(local["is_downloading_completed"]) if local else False + return DownloadResult( + label=target.label, + backend="tdlib", + ok=ok, + elapsed_seconds=elapsed_seconds, + bytes_downloaded=local["downloaded_size"] if local else None, + resumed=resumed, + error=None if ok else "download did not complete", + ) + + +async def resolve_file_id(client, target: BenchmarkTarget) -> int: + link_info = raise_if_error(await client.getMessageLinkInfo(url=target.link)) + message = raise_if_error( + await client.getMessage( + chat_id=link_info["chat_id"], message_id=link_info["message"]["id"] + ) + ) + return extract_file_id_from_message(message) + + +async def measure_clean_download(client, file_id: int): + """Single-shot download latency, directly comparable to the Telethon baseline.""" + started = time.perf_counter() + result = raise_if_error( + await client.downloadFile(file_id=file_id, priority=1, synchronous=True, offset=0, limit=0) + ) + elapsed = time.perf_counter() - started + return elapsed, result + + +async def measure_resumability(client, file_id: int) -> bool: + """Delete the local copy, start a fresh async download, cancel it mid-flight + while it is still partial, then confirm a second call resumes from the + partial bytes already on disk instead of restarting from zero.""" + raise_if_error(await client.deleteFile(file_id=file_id)) + raise_if_error( + await client.downloadFile(file_id=file_id, priority=1, synchronous=False, offset=0, limit=0) + ) + await asyncio.sleep(2) + + partial = raise_if_error(await client.getFile(file_id=file_id)) + partial_local = partial["local"] + partial_bytes = partial_local["downloaded_size"] if partial_local else 0 + + raise_if_error(await client.cancelDownloadFile(file_id=file_id, only_if_pending=False)) + + resumed_result = raise_if_error( + await client.downloadFile(file_id=file_id, priority=1, synchronous=True, offset=0, limit=0) + ) + resumed_local = resumed_result["local"] + completed = bool(resumed_local["is_downloading_completed"]) if resumed_local else False + downloaded = resumed_local["downloaded_size"] if resumed_local else 0 + grew_from_partial = downloaded >= partial_bytes + + return completed and partial_bytes > 0 and grew_from_partial + + +async def run(targets: list[BenchmarkTarget]) -> list[DownloadResult]: + load_dotenv(POC_ROOT / ".env") + client = build_client( + api_id=int(os.environ["TELEGRAM_API_ID"]), + api_hash=os.environ["TELEGRAM_API_HASH"], + files_directory=str(DATA_DIR), + ) + await client.start() + + results = [] + for target in targets: + file_id = await resolve_file_id(client, target) + elapsed, file_object = await measure_clean_download(client, file_id) + resumed_ok = await measure_resumability(client, file_id) + results.append(build_tdlib_result(target, elapsed, file_object, resumed=resumed_ok)) + + await client.stop() + return results + + +def main() -> None: + targets = load_benchmark_set(BENCHMARK_SET_PATH) + results = asyncio.run(run(targets)) + save_results(RESULTS_PATH, results) + for result in results: + print(f"{result.label}: ok={result.ok} elapsed={result.elapsed_seconds:.2f}s resumed={result.resumed}") + + +if __name__ == "__main__": + main() diff --git a/experiments/tdlib-media-poc/benchmark/run_telethon.py b/experiments/tdlib-media-poc/benchmark/run_telethon.py new file mode 100644 index 0000000..643270e --- /dev/null +++ b/experiments/tdlib-media-poc/benchmark/run_telethon.py @@ -0,0 +1,89 @@ +"""Benchmark media downloads through the current telegram-mcp path. + +Live usage: + uv run python benchmark/run_telethon.py + +Shells out to the existing `tg download` CLI (direct Telethon, no 120s MCP +cap — see mcp/src/telegram_mcp/tg_cli.py:cmd_download) for each target in +data/benchmark_set.json, times it, and records results. +""" + +import json +import subprocess +import time +from pathlib import Path + +from benchmark.models import BenchmarkTarget, DownloadResult, load_benchmark_set, save_results + +POC_ROOT = Path(__file__).resolve().parent.parent +TG_CLI = POC_ROOT.parent.parent / "mcp" / "bin" / "tg" +BENCHMARK_SET_PATH = POC_ROOT / "data" / "benchmark_set.json" +RESULTS_PATH = POC_ROOT / "data" / "results_telethon.json" +DOWNLOAD_DEST = POC_ROOT / "data" / "downloads" / "telethon" + + +def build_telethon_result( + target: BenchmarkTarget, + elapsed_seconds: float, + envelope: dict, + downloaded_size_bytes: int | None, +) -> DownloadResult: + ok = bool(envelope.get("ok")) and downloaded_size_bytes is not None + error = None + if not ok: + error = str(envelope.get("error") or envelope.get("payload") or "download failed") + return DownloadResult( + label=target.label, + backend="telethon", + ok=ok, + elapsed_seconds=elapsed_seconds, + bytes_downloaded=downloaded_size_bytes, + resumed=False, + error=error, + ) + + +def extract_json_envelope(stdout: str) -> dict: + """`tg download --json` interleaves PROGRESS lines with its JSON envelope + on stdout even with --json set; the envelope is the trailing `{...}` + block, so parse from the first brace rather than the whole stream.""" + json_start = stdout.find("{") + if json_start == -1: + raise json.JSONDecodeError("no JSON object found in stdout", stdout, 0) + return json.loads(stdout[json_start:]) + + +def download_one(target: BenchmarkTarget) -> DownloadResult: + DOWNLOAD_DEST.mkdir(parents=True, exist_ok=True) + started = time.perf_counter() + proc = subprocess.run( + [str(TG_CLI), "download", target.link, "--dest", str(DOWNLOAD_DEST), "--account", "main", "--json"], + capture_output=True, + text=True, + ) + elapsed = time.perf_counter() - started + try: + envelope = extract_json_envelope(proc.stdout) + except json.JSONDecodeError: + envelope = {"ok": False, "error": proc.stderr.strip() or "no JSON output"} + + downloaded_size = None + payload = envelope.get("payload") if isinstance(envelope.get("payload"), dict) else {} + downloaded_size = payload.get("size_bytes") + local_path = payload.get("path") + if downloaded_size is None and local_path and Path(local_path).exists(): + downloaded_size = Path(local_path).stat().st_size + + return build_telethon_result(target, elapsed, envelope, downloaded_size) + + +def main() -> None: + targets = load_benchmark_set(BENCHMARK_SET_PATH) + results = [download_one(target) for target in targets] + save_results(RESULTS_PATH, results) + for result in results: + print(f"{result.label}: ok={result.ok} elapsed={result.elapsed_seconds:.2f}s bytes={result.bytes_downloaded}") + + +if __name__ == "__main__": + main() diff --git a/experiments/tdlib-media-poc/benchmark/select_targets.py b/experiments/tdlib-media-poc/benchmark/select_targets.py new file mode 100644 index 0000000..ff0c986 --- /dev/null +++ b/experiments/tdlib-media-poc/benchmark/select_targets.py @@ -0,0 +1,25 @@ +"""Parse t.me links and pull file-size metadata from `tg message` output.""" + +import re + +LINK_PATTERN = re.compile( + r"^https?://t\.me/(?:c/(?P\d+)|(?P[A-Za-z0-9_]+))/(?P\d+)$" +) + + +def parse_link(link: str) -> tuple[str, int]: + match = LINK_PATTERN.match(link.strip()) + if not match: + raise ValueError(f"not a recognized t.me post link: {link!r}") + message_id = int(match.group("message_id")) + if match.group("internal_id"): + chat = f"-100{match.group('internal_id')}" + else: + chat = match.group("username") + return chat, message_id + + +def extract_file_size(tg_message_envelope: dict) -> int | None: + payload = tg_message_envelope.get("payload") or {} + message = payload.get("message") or {} + return message.get("file_size") diff --git a/experiments/tdlib-media-poc/benchmark/tdlib_client.py b/experiments/tdlib-media-poc/benchmark/tdlib_client.py new file mode 100644 index 0000000..95cca70 --- /dev/null +++ b/experiments/tdlib-media-poc/benchmark/tdlib_client.py @@ -0,0 +1,33 @@ +"""Isolated pytdbot client factory for the TDLib media-download POC. + +Per control-plane/docs/adr/2026-06-21-tdlib-is-not-default-runtime.md, this +POC must never share state with the Telethon session tree that telegram-mcp +owns. +""" + +import pytdbot + +TELETHON_SESSION_DIR_MARKERS = (".telegram-mcp",) + + +def assert_isolated_from_telethon(files_directory: str) -> None: + for marker in TELETHON_SESSION_DIR_MARKERS: + if marker in files_directory: + raise ValueError( + f"files_directory must not overlap the Telethon session tree (found {marker!r})" + ) + + +def build_client( + api_id: int, + api_hash: str, + files_directory: str, + database_encryption_key: str = "tdlib-media-poc", +) -> pytdbot.Client: + assert_isolated_from_telethon(files_directory) + return pytdbot.Client( + api_id=api_id, + api_hash=api_hash, + files_directory=files_directory, + database_encryption_key=database_encryption_key, + ) diff --git a/experiments/tdlib-media-poc/benchmark/tdlib_message.py b/experiments/tdlib-media-poc/benchmark/tdlib_message.py new file mode 100644 index 0000000..37f1f03 --- /dev/null +++ b/experiments/tdlib-media-poc/benchmark/tdlib_message.py @@ -0,0 +1,25 @@ +"""Pure helpers for pulling downloadable file_ids out of TDLib Message objects. + +Works directly on pytdbot's native typed objects (pytdbot.types.Message and +friends), not on dicts: pytdbot's to_dict() only converts the top level of an +object, leaving nested fields (e.g. Message.content) as native typed objects, +so dict-shaped fixtures don't reflect what a live call actually returns. +Native pytdbot types support subscript access (obj["field"]) directly. + +TDLib content schema reference: https://core.telegram.org/tdlib/docs/classtd_1_1td__api_1_1_message.html +""" + + +def extract_file_id_from_message(message) -> int: + content = message["content"] + content_type = content.getType() + if content_type == "messageVideo": + return content["video"]["video"]["id"] + if content_type == "messageDocument": + return content["document"]["document"]["id"] + if content_type == "messagePhoto": + sizes = content["photo"]["sizes"] + return sizes[-1]["photo"]["id"] + if content_type == "messageAudio": + return content["audio"]["audio"]["id"] + raise ValueError(f"unsupported message content type for download: {content_type!r}") diff --git a/experiments/tdlib-media-poc/pyproject.toml b/experiments/tdlib-media-poc/pyproject.toml new file mode 100644 index 0000000..2b71d96 --- /dev/null +++ b/experiments/tdlib-media-poc/pyproject.toml @@ -0,0 +1,17 @@ +[project] +name = "tdlib-media-poc" +version = "0.1.0" +requires-python = ">=3.12" +dependencies = [ + "Pytdbot>=0.10.1", + "python-dotenv>=1.0.1", + "tdjson>=1.8.65", +] + +[dependency-groups] +dev = [ + "pytest>=8.0.0", +] + +[tool.uv] +package = false diff --git a/experiments/tdlib-media-poc/tests/__init__.py b/experiments/tdlib-media-poc/tests/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/experiments/tdlib-media-poc/tests/test_compare.py b/experiments/tdlib-media-poc/tests/test_compare.py new file mode 100644 index 0000000..e0e4973 --- /dev/null +++ b/experiments/tdlib-media-poc/tests/test_compare.py @@ -0,0 +1,22 @@ +from benchmark.compare import build_report +from benchmark.models import DownloadResult + + +def test_build_report_includes_table_rows_and_averages(): + telethon = [DownloadResult("big-video-1", "telethon", True, 10.0, 52428800, False)] + tdlib = [DownloadResult("big-video-1", "tdlib", True, 8.0, 52428800, True)] + + report = build_report(telethon, tdlib) + + assert "big-video-1" in report + assert "telethon=10.00s, tdlib=8.00s" in report + assert "resume after interruption: True" in report + + +def test_build_report_handles_no_successful_runs(): + telethon = [DownloadResult("big-video-1", "telethon", False, 0.0, None, False, error="boom")] + tdlib = [DownloadResult("big-video-1", "tdlib", False, 0.0, None, False, error="boom")] + + report = build_report(telethon, tdlib) + + assert "Not enough successful runs" in report diff --git a/experiments/tdlib-media-poc/tests/test_login_tdlib.py b/experiments/tdlib-media-poc/tests/test_login_tdlib.py new file mode 100644 index 0000000..1f971c5 --- /dev/null +++ b/experiments/tdlib-media-poc/tests/test_login_tdlib.py @@ -0,0 +1,21 @@ +from benchmark.login_tdlib import required_arg_for_state + + +def test_required_arg_for_wait_phone_number(): + assert required_arg_for_state("authorizationStateWaitPhoneNumber") == "phone" + + +def test_required_arg_for_wait_code(): + assert required_arg_for_state("authorizationStateWaitCode") == "code" + + +def test_required_arg_for_wait_password(): + assert required_arg_for_state("authorizationStateWaitPassword") == "password" + + +def test_required_arg_for_ready_state_is_none(): + assert required_arg_for_state("authorizationStateReady") is None + + +def test_required_arg_for_unknown_state_is_none(): + assert required_arg_for_state("authorizationStateWaitTdlibParameters") is None diff --git a/experiments/tdlib-media-poc/tests/test_models.py b/experiments/tdlib-media-poc/tests/test_models.py new file mode 100644 index 0000000..cbfaf12 --- /dev/null +++ b/experiments/tdlib-media-poc/tests/test_models.py @@ -0,0 +1,48 @@ +import json + +from benchmark.models import ( + BenchmarkTarget, + DownloadResult, + load_benchmark_set, + load_results, + save_benchmark_set, + save_results, +) + + +def test_save_and_load_benchmark_set_round_trip(tmp_path): + path = tmp_path / "data" / "benchmark_set.json" + targets = [ + BenchmarkTarget( + label="big-video-1", + chat="durov", + message_id=123, + link="https://t.me/durov/123", + expected_size_bytes=52_428_800, + ) + ] + + save_benchmark_set(path, targets) + loaded = load_benchmark_set(path) + + assert loaded == targets + assert json.loads(path.read_text())[0]["label"] == "big-video-1" + + +def test_save_and_load_results_round_trip(tmp_path): + path = tmp_path / "data" / "results_telethon.json" + results = [ + DownloadResult( + label="big-video-1", + backend="telethon", + ok=True, + elapsed_seconds=12.5, + bytes_downloaded=52_428_800, + resumed=False, + ) + ] + + save_results(path, results) + loaded = load_results(path) + + assert loaded == results diff --git a/experiments/tdlib-media-poc/tests/test_run_tdlib_error_handling.py b/experiments/tdlib-media-poc/tests/test_run_tdlib_error_handling.py new file mode 100644 index 0000000..8e73188 --- /dev/null +++ b/experiments/tdlib-media-poc/tests/test_run_tdlib_error_handling.py @@ -0,0 +1,15 @@ +import pytdbot +import pytest + +from benchmark.run_tdlib import raise_if_error + + +def test_raise_if_error_passes_through_non_error_result(): + result = raise_if_error("some value") + assert result == "some value" + + +def test_raise_if_error_raises_on_pytdbot_error(): + error = pytdbot.types.Error(code=400, message="MESSAGE_ID_INVALID") + with pytest.raises(RuntimeError, match="MESSAGE_ID_INVALID"): + raise_if_error(error) diff --git a/experiments/tdlib-media-poc/tests/test_run_telethon.py b/experiments/tdlib-media-poc/tests/test_run_telethon.py new file mode 100644 index 0000000..bd642b8 --- /dev/null +++ b/experiments/tdlib-media-poc/tests/test_run_telethon.py @@ -0,0 +1,31 @@ +from benchmark.models import BenchmarkTarget +from benchmark.run_telethon import build_telethon_result + +TARGET = BenchmarkTarget( + label="big-video-1", + chat="durov", + message_id=123, + link="https://t.me/durov/123", + expected_size_bytes=52_428_800, +) + + +def test_build_telethon_result_success(): + envelope = {"ok": True, "payload": {"local_path": "/tmp/foo.mp4"}} + result = build_telethon_result(TARGET, 12.5, envelope, 52_428_800) + + assert result.ok is True + assert result.backend == "telethon" + assert result.elapsed_seconds == 12.5 + assert result.bytes_downloaded == 52_428_800 + assert result.resumed is False + assert result.error is None + + +def test_build_telethon_result_failure_from_bad_envelope(): + envelope = {"ok": False, "error": "flood_wait"} + result = build_telethon_result(TARGET, 3.0, envelope, None) + + assert result.ok is False + assert result.bytes_downloaded is None + assert result.error == "flood_wait" diff --git a/experiments/tdlib-media-poc/tests/test_run_telethon_stdout_parsing.py b/experiments/tdlib-media-poc/tests/test_run_telethon_stdout_parsing.py new file mode 100644 index 0000000..f883561 --- /dev/null +++ b/experiments/tdlib-media-poc/tests/test_run_telethon_stdout_parsing.py @@ -0,0 +1,25 @@ +from benchmark.run_telethon import extract_json_envelope + + +def test_extract_json_envelope_strips_leading_progress_lines(): + stdout = ( + "PROGRESS 0/46MB 0.3%\n" + "PROGRESS 46/46MB 100.0%\n" + '{\n "ok": true,\n "payload": {"path": "/tmp/x.mp4", "size_bytes": 123}\n}\n' + ) + envelope = extract_json_envelope(stdout) + assert envelope == {"ok": True, "payload": {"path": "/tmp/x.mp4", "size_bytes": 123}} + + +def test_extract_json_envelope_handles_pure_json_with_no_progress_lines(): + stdout = '{"ok": false, "error": "boom"}' + envelope = extract_json_envelope(stdout) + assert envelope == {"ok": False, "error": "boom"} + + +def test_extract_json_envelope_raises_on_no_json_present(): + import json + import pytest + + with pytest.raises(json.JSONDecodeError): + extract_json_envelope("PROGRESS 0/1MB 0.0%\nsome error text, no braces") diff --git a/experiments/tdlib-media-poc/tests/test_select_targets.py b/experiments/tdlib-media-poc/tests/test_select_targets.py new file mode 100644 index 0000000..56f384d --- /dev/null +++ b/experiments/tdlib-media-poc/tests/test_select_targets.py @@ -0,0 +1,42 @@ +import pytest + +from benchmark.select_targets import extract_file_size, parse_link + + +def test_parse_link_public_username(): + chat, message_id = parse_link("https://t.me/durov/123") + assert chat == "durov" + assert message_id == 123 + + +def test_parse_link_private_channel_id(): + chat, message_id = parse_link("https://t.me/c/1234567890/456") + assert chat == "-1001234567890" + assert message_id == 456 + + +def test_parse_link_rejects_non_telegram_url(): + with pytest.raises(ValueError, match="not a recognized t.me post link"): + parse_link("https://example.com/foo/1") + + +def test_extract_file_size_from_tg_message_envelope(): + envelope = { + "ok": True, + "payload": { + "chat": {"id": 123}, + "message_id": 456, + "message": {"id": 456, "file_size": 52428800, "media_type": "video"}, + }, + } + assert extract_file_size(envelope) == 52428800 + + +def test_extract_file_size_returns_none_when_no_media(): + envelope = {"ok": True, "payload": {"message": {"id": 1, "file_size": None}}} + assert extract_file_size(envelope) is None + + +def test_extract_file_size_returns_none_when_message_missing(): + envelope = {"ok": True, "payload": {"message": None}} + assert extract_file_size(envelope) is None diff --git a/experiments/tdlib-media-poc/tests/test_tdlib_client.py b/experiments/tdlib-media-poc/tests/test_tdlib_client.py new file mode 100644 index 0000000..4589034 --- /dev/null +++ b/experiments/tdlib-media-poc/tests/test_tdlib_client.py @@ -0,0 +1,21 @@ +import pytest + +from benchmark.tdlib_client import assert_isolated_from_telethon, build_client + + +def test_assert_isolated_from_telethon_passes_for_poc_dir(tmp_path): + assert_isolated_from_telethon(str(tmp_path / "tdlib-media-poc" / "data" / "tdlib")) + + +def test_assert_isolated_from_telethon_rejects_telethon_session_dir(): + with pytest.raises(ValueError, match="Telethon session tree"): + assert_isolated_from_telethon("/Users/sereja/.telegram-mcp/session") + + +def test_build_client_constructs_with_isolated_directory(tmp_path): + client = build_client( + api_id=1, + api_hash="0" * 32, + files_directory=str(tmp_path / "tdlib"), + ) + assert client is not None diff --git a/experiments/tdlib-media-poc/tests/test_tdlib_message.py b/experiments/tdlib-media-poc/tests/test_tdlib_message.py new file mode 100644 index 0000000..137c3c1 --- /dev/null +++ b/experiments/tdlib-media-poc/tests/test_tdlib_message.py @@ -0,0 +1,66 @@ +import pytdbot +import pytest + +from benchmark.tdlib_message import extract_file_id_from_message +from benchmark.run_tdlib import build_tdlib_result +from benchmark.models import BenchmarkTarget + +TARGET = BenchmarkTarget( + label="big-video-1", + chat="durov", + message_id=123, + link="https://t.me/durov/123", + expected_size_bytes=52_428_800, +) + + +def test_extract_file_id_from_message_video(): + message = pytdbot.types.Message( + content=pytdbot.types.MessageVideo( + video=pytdbot.types.Video(video=pytdbot.types.File(id=555, size=52_428_800)) + ) + ) + assert extract_file_id_from_message(message) == 555 + + +def test_extract_file_id_from_message_document(): + message = pytdbot.types.Message( + content=pytdbot.types.MessageDocument( + document=pytdbot.types.Document(document=pytdbot.types.File(id=777, size=10_000_000)) + ) + ) + assert extract_file_id_from_message(message) == 777 + + +def test_extract_file_id_from_message_unsupported_type(): + message = pytdbot.types.Message(content=pytdbot.types.MessageText()) + with pytest.raises(ValueError, match="unsupported message content type"): + extract_file_id_from_message(message) + + +def test_build_tdlib_result_completed_download(): + file_object = pytdbot.types.File( + id=555, + size=52_428_800, + local=pytdbot.types.LocalFile(downloaded_size=52_428_800, is_downloading_completed=True), + ) + result = build_tdlib_result(TARGET, 8.0, file_object, resumed=True) + + assert result.ok is True + assert result.backend == "tdlib" + assert result.bytes_downloaded == 52_428_800 + assert result.resumed is True + assert result.error is None + + +def test_build_tdlib_result_incomplete_download(): + file_object = pytdbot.types.File( + id=555, + size=52_428_800, + local=pytdbot.types.LocalFile(downloaded_size=1_000_000, is_downloading_completed=False), + ) + result = build_tdlib_result(TARGET, 3.0, file_object, resumed=False) + + assert result.ok is False + assert result.bytes_downloaded == 1_000_000 + assert result.error == "download did not complete" diff --git a/experiments/tdlib-media-poc/uv.lock b/experiments/tdlib-media-poc/uv.lock new file mode 100644 index 0000000..af9b33a --- /dev/null +++ b/experiments/tdlib-media-poc/uv.lock @@ -0,0 +1,233 @@ +version = 1 +revision = 3 +requires-python = ">=3.12" + +[[package]] +name = "cachebox" +version = "5.2.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/36/f6/85f176d2518cf1d1be5f981fc2dadf6b131e33fefd721f36b330e3434d6c/cachebox-5.2.3.tar.gz", hash = "sha256:b1f68246685aa739bbbd2734befb1465363a1e1042407c154feadb065f17a099", size = 63686, upload-time = "2026-04-10T12:21:35.028Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e4/e7/6fa6abfc9c4c07b88f09a88466fa93c7081fd679d8e06f8f558bb4ac845c/cachebox-5.2.3-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:09c0340e9daa7b4530801e5a570cb0c1a1ad941a85d245d360020d3986d0e787", size = 377791, upload-time = "2026-04-10T12:20:23.87Z" }, + { url = "https://files.pythonhosted.org/packages/3a/79/89e4423352d0ca33bbf80fc1b4b665e654a93de8b16cf41e96fcac81801a/cachebox-5.2.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:f3162758792626685ec34950eedd565d015b115d0ff0d751d2716031fc32d51b", size = 359562, upload-time = "2026-04-10T12:20:10.626Z" }, + { url = "https://files.pythonhosted.org/packages/d2/ab/e533c2751e6a3411ebe369277aaed03199b9e4586a48f0a3712a1f4b418b/cachebox-5.2.3-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a189a780c3ccd7b9d157074ba6bf3e191e522b39abbdb590075111851f02d50d", size = 397910, upload-time = "2026-04-10T12:18:53.336Z" }, + { url = "https://files.pythonhosted.org/packages/7a/0d/b8492d6ca53278499a37c9f9d51afd4ad77bfbe813d6281944d45b97a1e7/cachebox-5.2.3-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:410b67baa99d433644199b11289627f7ebba4ee5786f95ca9858f238afcee157", size = 353699, upload-time = "2026-04-10T12:19:06.248Z" }, + { url = "https://files.pythonhosted.org/packages/78/d4/fd20b3a5362651303fa12d3ee62f56af2bd396e4a7303d7014a1a1e5b392/cachebox-5.2.3-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f81474dc19d3865fa5e57263f834bc6bbc00e471a594fb9d934ed552732c02fd", size = 372510, upload-time = "2026-04-10T12:19:18.997Z" }, + { url = "https://files.pythonhosted.org/packages/71/94/3ec55c946d300cc4eaed3a0f79740051ac6e11ef4032421332c6ca15f5d5/cachebox-5.2.3-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:85ccd827193b3e3e887a88a16b88ef7ed174e7e65be515b5253322aa75e665c3", size = 392802, upload-time = "2026-04-10T12:19:31.196Z" }, + { url = "https://files.pythonhosted.org/packages/01/b1/1a3c4e436ad8a4c4ba3e70f4c62e1f927cbbb3c943a9bba5813b8b815bde/cachebox-5.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2a1e7d3cb8a5e7e68996a8619e3ef8771a124d14568c251f9e586eba88d759c1", size = 398223, upload-time = "2026-04-10T12:19:57.583Z" }, + { url = "https://files.pythonhosted.org/packages/0a/ea/d36ad3976c4396b350b96a1582411b7a00e56c144eec0bb5ba5f36ce7d86/cachebox-5.2.3-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:adcedfcfcb933b21e7fdcfe560c79887bc8287abceab0586aa3730417dd0277d", size = 427696, upload-time = "2026-04-10T12:19:44.361Z" }, + { url = "https://files.pythonhosted.org/packages/a8/36/71845b5c7a9ffbd85e6fdb470c11a174f499bd5238fa37b1214157c2454d/cachebox-5.2.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:c7f0c72c51a3a9e7049ea6ff2a43cd3877ab7fee966eb65771a59621563b75e3", size = 567854, upload-time = "2026-04-10T12:20:38.357Z" }, + { url = "https://files.pythonhosted.org/packages/e8/a2/baf0e5a8392e64e352b137ccd7356b3d98068c842fd19f510a7790c05d34/cachebox-5.2.3-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:c48c10e498d573511aafbd545570e7f43b40a7428dc282183bf5adc334d9e1a8", size = 670306, upload-time = "2026-04-10T12:20:52.903Z" }, + { url = "https://files.pythonhosted.org/packages/a5/22/cd4e4c1d624b8ef9fb4b8bebf0bf5d2d74a399cf1ac46b667bb79d15359a/cachebox-5.2.3-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:2f1e086ab5ffd082a68bb63699d517655a59b06414927bfc84e01df91b81e34d", size = 645943, upload-time = "2026-04-10T12:21:08.238Z" }, + { url = "https://files.pythonhosted.org/packages/0a/d6/55859981f5ec6a9e412baaa4db6aa5973a00008750b3f054cdefcb6491fc/cachebox-5.2.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:649d18399f13735bb82daa33800196f815529c49e967767c40ca221723e68afa", size = 612309, upload-time = "2026-04-10T12:21:23.404Z" }, + { url = "https://files.pythonhosted.org/packages/d7/1e/313f650467ac85824c4199188f8f1ee3386cd12eb665dbf7c88d372e4956/cachebox-5.2.3-cp312-cp312-win32.whl", hash = "sha256:0a17aeb4e5b1c6ef1c3db8fc5186f9986e215ba5ea5a5d08baa45bcf55f261b2", size = 279789, upload-time = "2026-04-10T12:21:53.215Z" }, + { url = "https://files.pythonhosted.org/packages/c5/50/3b334f887accfa811cf5c7533b8ce22c523eb009363a86401198899dadd2/cachebox-5.2.3-cp312-cp312-win_amd64.whl", hash = "sha256:cfd69114141ab362acaa2099e425a1b965cf7b021a539a4e953143d593930b74", size = 290917, upload-time = "2026-04-10T12:21:39.696Z" }, + { url = "https://files.pythonhosted.org/packages/31/3b/16d5c295f6ec2913ef595b39986dc7b7cc179fdd2e73f5ebd1814c38fd51/cachebox-5.2.3-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:9527c5c70f8735f2d696331d8bcf77254f03b4dc8542046807823bd36ed4e8ba", size = 377408, upload-time = "2026-04-10T12:20:25.444Z" }, + { url = "https://files.pythonhosted.org/packages/cd/87/45f834154f79721e5b64a80ffab4f9710834c4f9c01fa977f94a9116c32a/cachebox-5.2.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:40ac878af00d5969862c1f6bc076de1e34ca248662fce6aecca1761f52e33e32", size = 359274, upload-time = "2026-04-10T12:20:12.127Z" }, + { url = "https://files.pythonhosted.org/packages/46/17/794e5f93e0a172aa14ecd692f6d89bdf094f71eb35fa923d0a0af25cef1c/cachebox-5.2.3-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b5ff26bfd8f7e95b3becf6d5f65c25edaca50fa68078868648b70d79bcccc260", size = 397520, upload-time = "2026-04-10T12:18:54.807Z" }, + { url = "https://files.pythonhosted.org/packages/23/19/9470b1a96de6e480192b1a92b2fafa72aa052efc2509a5418a5652205b33/cachebox-5.2.3-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:82e7002dd343afeeba2fcf0e483131b342a27ec3bc34b2214dc617691bda40d6", size = 353183, upload-time = "2026-04-10T12:19:07.797Z" }, + { url = "https://files.pythonhosted.org/packages/6c/2b/72813f80397ed4640e337cbd1a14ab7eaafe33e479291d3623b6a6a55fec/cachebox-5.2.3-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ccbdc54a6c4b5758408c1083bdfa217bd382894a8331c7d0a54b84ba0cf51e5b", size = 372239, upload-time = "2026-04-10T12:19:20.44Z" }, + { url = "https://files.pythonhosted.org/packages/05/17/47dc9687288fa55486573627089ecd9aae124de5924a4bce008af96d80b6/cachebox-5.2.3-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:df5135a168f143d186b1cc3be0ca16b66446897ab5cedc03bd80bcc926fcd403", size = 392568, upload-time = "2026-04-10T12:19:32.73Z" }, + { url = "https://files.pythonhosted.org/packages/13/95/450765b971a3bed9d7cf003c3833c1976482eb83b0241b6dbb840a25b43b/cachebox-5.2.3-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:10bedf96db8f9766cc956f9adcc623e604264e5d6fa2e255432f8c2ed7519143", size = 397920, upload-time = "2026-04-10T12:19:59.314Z" }, + { url = "https://files.pythonhosted.org/packages/5f/3e/dd8f4c1f92e58d479913ce9cbaa3227c911128e6046c82f4fd44309f685a/cachebox-5.2.3-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:f22732d0d69bb84ad2dca7480bffdfd0430c647152d488936e152ecbbfee52fb", size = 427332, upload-time = "2026-04-10T12:19:45.888Z" }, + { url = "https://files.pythonhosted.org/packages/7e/20/80d8c26ce63e78da3874a5bb07a3a78de53a2b0356ba80583a4927f0a074/cachebox-5.2.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:26ae0b68979204d360327f4c0725cfdc95cfc34ab73ab1a8f528e3bd2f6d023c", size = 567494, upload-time = "2026-04-10T12:20:40.373Z" }, + { url = "https://files.pythonhosted.org/packages/10/35/7249885dfed3602b3b48c1e67781197dcdc536c50f72caeabe3944348af8/cachebox-5.2.3-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:f3d628b816e28a6e7661d460e02dd5b421247cc2cd275814f80ea79621245fc4", size = 669968, upload-time = "2026-04-10T12:20:55.155Z" }, + { url = "https://files.pythonhosted.org/packages/2d/8a/e5b58f0bbd6fef74da5d8e5ab49e67898ce7e6df28c16280a0f2b78461f7/cachebox-5.2.3-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:64057caa6b741320655cd3c5997fe642dae5dbff571eb530e6f53e58272bb43b", size = 645547, upload-time = "2026-04-10T12:21:09.948Z" }, + { url = "https://files.pythonhosted.org/packages/d8/25/51783a4c6f25ca87ef1b4b762ff0364bd98053a02d597b30d26ff4cf13c5/cachebox-5.2.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:fa325306084aa2dc0b21e07723d7700f4d43dece3732c7fdaf7a269dc5e35aa7", size = 611844, upload-time = "2026-04-10T12:21:25.286Z" }, + { url = "https://files.pythonhosted.org/packages/c5/c5/b26c4b046e296d0e249448fe297626b3caca2e851837712f03c358662cb7/cachebox-5.2.3-cp313-cp313-win32.whl", hash = "sha256:55003089d21c2f5515089c307be063b45558e884a4a1cc9593944374c89975c4", size = 279421, upload-time = "2026-04-10T12:21:54.921Z" }, + { url = "https://files.pythonhosted.org/packages/e0/7f/a49420670393bfea618de7a893d45cae9294cf3293d7b158e7af20e8f39e/cachebox-5.2.3-cp313-cp313-win_amd64.whl", hash = "sha256:dcc5edb6ecf2b516e90b773d232360c5e4ed8fdcda038b19441da2ed9cf208ab", size = 290702, upload-time = "2026-04-10T12:21:41.458Z" }, + { url = "https://files.pythonhosted.org/packages/c9/0b/bf83bda13ef6fc490d208a1d4dd712034624526a88f61713cca0edc9884f/cachebox-5.2.3-cp313-cp313t-macosx_10_12_x86_64.whl", hash = "sha256:a4b7559fa4994c4032dd07466c2041d57e055feb814762e1f73f4e8beef188d0", size = 371704, upload-time = "2026-04-10T12:20:27.253Z" }, + { url = "https://files.pythonhosted.org/packages/8e/ea/aa5162273238e84f9e41b33600c69299572dc1c8f0f768d07660b71be07d/cachebox-5.2.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:f57afada3d9327adf87f3b5cf0094348c6fd49354ab2e9bd20b044648eb094ae", size = 353385, upload-time = "2026-04-10T12:20:13.668Z" }, + { url = "https://files.pythonhosted.org/packages/47/96/3ca013e2e48df5c1d7855669b208f4bf8014ccb842ccf7a3a0eaac07bee0/cachebox-5.2.3-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8342ff350ce86f062492752d612e9f056ac5dc56375713d75c3bf6e83b4d18db", size = 392181, upload-time = "2026-04-10T12:18:56.385Z" }, + { url = "https://files.pythonhosted.org/packages/63/ca/1bacb4efa0b0ce8065d1fb7c8dc7c382ec4e1cc3f007eb08417732be2725/cachebox-5.2.3-cp313-cp313t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:405f9cc8492fc9d953b5a6b9e2b661e99583755c6639ab8d09a287fdf336503c", size = 349494, upload-time = "2026-04-10T12:19:09.505Z" }, + { url = "https://files.pythonhosted.org/packages/d7/2e/75db4bda3768658f5baa5a54f6a4f643bc2de1a16788e40581a080e803c7/cachebox-5.2.3-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:94aae393ec1d9b26565d346445bb6afa3963d2a0d3eb5e4188d0e510fab871a0", size = 369216, upload-time = "2026-04-10T12:19:22.224Z" }, + { url = "https://files.pythonhosted.org/packages/f5/82/e1f833be0d57e29a8c5eb0a0275cd34b962f3c7f5b9e0517ec4bf75e7cc3/cachebox-5.2.3-cp313-cp313t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a8b0b575066fc09f6fae0d4bd30d6ff56584a6870cbe7d202916c5e0d725cfd4", size = 385922, upload-time = "2026-04-10T12:19:34.198Z" }, + { url = "https://files.pythonhosted.org/packages/53/d6/615a3c16c1d63839f2c67644eb414c4dc9769ab2e169d935110fd8e268d5/cachebox-5.2.3-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:41e99c1240106d39b63ce7868a6cd8c9da9243fef08848b85d428164e0769fd2", size = 393276, upload-time = "2026-04-10T12:20:00.925Z" }, + { url = "https://files.pythonhosted.org/packages/2f/a6/7844c9c84b170dae1005b22da174639968e64c8055d66a209a1598663771/cachebox-5.2.3-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:432ca62b99f7eafc21af669d76c88c1b7377db179b89fb6fca3ea93b8f9fff19", size = 421355, upload-time = "2026-04-10T12:19:47.691Z" }, + { url = "https://files.pythonhosted.org/packages/c9/0f/43f62355846cae3dc41cb4daccac0a4bb2b7b8b3c7d77d1b6a220bae6d54/cachebox-5.2.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:e51d9c59006b53447f806145406eb37a7fc3c25553d4fd24c3887f3b268d214e", size = 561656, upload-time = "2026-04-10T12:20:42.161Z" }, + { url = "https://files.pythonhosted.org/packages/9b/fc/a453813c6d000d69a41a06c6a3143a6c4d0d0e41f23c155db2f82ea0edfa/cachebox-5.2.3-cp313-cp313t-musllinux_1_2_armv7l.whl", hash = "sha256:5e48a405f699fb001b8af120a6e0b4a981277f84eb5dd66a1faa21e4b6fe9485", size = 665791, upload-time = "2026-04-10T12:20:56.842Z" }, + { url = "https://files.pythonhosted.org/packages/aa/a3/f6a9e75f1e602b67b6d67088a9a766adfc4e0a740a9c4b68e4e6207c1006/cachebox-5.2.3-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:8cbfc007ea78af61d75d7d26e5854df53dc5da6877d074afd4b4696c074f4ee7", size = 640975, upload-time = "2026-04-10T12:21:11.641Z" }, + { url = "https://files.pythonhosted.org/packages/a3/15/4ac98277f7fd9d855c8ed337e8e2a3386d17997cce2dd3eadb23dedc08e3/cachebox-5.2.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:6a94d0da8133b3a0707ae11c9ea321f8fc37e3b5a14517019a05d632218b0f56", size = 607242, upload-time = "2026-04-10T12:21:27.27Z" }, + { url = "https://files.pythonhosted.org/packages/9c/0b/ce61907a803f75854e0cc91b84c16e14dce0e4e939efbda26293eb4c8784/cachebox-5.2.3-cp313-cp313t-win32.whl", hash = "sha256:5fee33549877c03c2494ec5359a57a7667f872fe8e296a7f39d3dfe08dd3914c", size = 271619, upload-time = "2026-04-10T12:21:56.768Z" }, + { url = "https://files.pythonhosted.org/packages/b0/06/fece190ad5173d06b2779494aaad5528907f2e55c809618e5b67c2e3dbb5/cachebox-5.2.3-cp313-cp313t-win_amd64.whl", hash = "sha256:67548a05cd41fcc4f7af80a2f97f742fef3d436537ac2e1a1dce0fcba5d41190", size = 283133, upload-time = "2026-04-10T12:21:43.037Z" }, + { url = "https://files.pythonhosted.org/packages/b8/8b/72c0e80aad08e09867ce14a621bce689a733552f20cdf2ef96d4b052da10/cachebox-5.2.3-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:37fa0891f0defee053c09f5f43f802f731e36e6e6ca055d7d174af07f77232ca", size = 380523, upload-time = "2026-04-10T12:20:29.345Z" }, + { url = "https://files.pythonhosted.org/packages/fc/62/33aaade81b181d5191cc39c867c297aa7c65f3191aa9749bf99b77496b88/cachebox-5.2.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:dc6315902f2ef4afbf10bc8e08c54ff34de5ce124546b8e0016c9b0d327be21e", size = 362424, upload-time = "2026-04-10T12:20:15.215Z" }, + { url = "https://files.pythonhosted.org/packages/9e/0b/3eedaf9ea4b41c931f4340bfa42056efe2bb5fe3a79649d6c8a1dce585a5/cachebox-5.2.3-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7df1735ca778480d51b8232fed397ffe3935158f20d34fb1c5ed171b53d5a6e2", size = 399572, upload-time = "2026-04-10T12:18:58.331Z" }, + { url = "https://files.pythonhosted.org/packages/be/69/c79b8a6a5b889ac4a60800bacea3553cb3b86f6fd13b2262bade1cb962c6/cachebox-5.2.3-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e22451cde8f884051e941b21870e4fc91fcf58d0d8c285bb8964107e1f02445c", size = 353803, upload-time = "2026-04-10T12:19:11.21Z" }, + { url = "https://files.pythonhosted.org/packages/d4/c3/bc7838de51039f8c50506d8dc82f22ff9a652794339a223b12af595e1d2f/cachebox-5.2.3-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:dcbccf3015d9a42bcf41260fa5cc048a5bdb75aa10997d514d6c976117f30ee2", size = 374474, upload-time = "2026-04-10T12:19:23.658Z" }, + { url = "https://files.pythonhosted.org/packages/65/61/e5231ad2ae952ca482f9b9df55df4b96add1a80de28de537c5f574605987/cachebox-5.2.3-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:311eae5079e256cbbfafdc3dcff1714b6598a767f9c1ef8c3709e74ea0cc12b0", size = 393045, upload-time = "2026-04-10T12:19:35.651Z" }, + { url = "https://files.pythonhosted.org/packages/78/c4/c9b3fa764ac5420a9e079ad53fa8840d4a26b74c4ccda56acbef49cf76ff/cachebox-5.2.3-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1f4d2a80a5cd3380739c67f7d89e596634f5897b8d5a4a3dc1598312cb077535", size = 398700, upload-time = "2026-04-10T12:20:02.513Z" }, + { url = "https://files.pythonhosted.org/packages/9b/3e/c4e3acd4cb04e01c5fb7cc7a4de16059b9594d90672fff85af8670275267/cachebox-5.2.3-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:3977515b727a5203f494c44c4566fb936c4b940351c01d3d8e7b5d104dff4f53", size = 426725, upload-time = "2026-04-10T12:19:49.385Z" }, + { url = "https://files.pythonhosted.org/packages/25/5d/610b79479719951581109d985244d34c97f86a308c3d7c83443e2b1dac46/cachebox-5.2.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c5be17dd5c4fabcfecd5bcf6d54f9c6fb719daed3ef01ac1c03a14af0e2b26c1", size = 570042, upload-time = "2026-04-10T12:20:43.793Z" }, + { url = "https://files.pythonhosted.org/packages/8c/63/cad8a05db4d0c0f5ba6bccb32e57d15c472276de9476f56004445b40711f/cachebox-5.2.3-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:6d37334fc218fdaee31db8a4f938938716e7c3b1b4059e25de27c8447fc95fde", size = 670974, upload-time = "2026-04-10T12:20:58.528Z" }, + { url = "https://files.pythonhosted.org/packages/54/d1/9cff7c2b9048d1c38b7ad8199ce856596d09720b3bea74043f3bad71970b/cachebox-5.2.3-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:1e5f1b7e23411b748d919348c3b65db1f9f8927ab8f6f3acae19bd617543df2d", size = 646213, upload-time = "2026-04-10T12:21:13.619Z" }, + { url = "https://files.pythonhosted.org/packages/27/ae/2e1ad162ec13903e84469c8a753baf385f1bc324279d6c7cb6365e7099df/cachebox-5.2.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:e7b06a75a898b31fd73c4d8bf727a9b9f8b5b7738cccd0ab5e6fd2a9cf659d3c", size = 612787, upload-time = "2026-04-10T12:21:29.271Z" }, + { url = "https://files.pythonhosted.org/packages/c8/8a/07b5ffd841e1ff534bb6e8721c39fdfe0d7cdaac1398e1783b2a0c37bd22/cachebox-5.2.3-cp314-cp314-win32.whl", hash = "sha256:3b798052719f09a2ce7bf9fa9452dc0a7d4dc53b50a2d3aba6ce6ebc12d39df7", size = 278559, upload-time = "2026-04-10T12:21:58.482Z" }, + { url = "https://files.pythonhosted.org/packages/c4/f8/b88a82ce9ec7a2fa0f09ed1cdd031692c8664c41f9ab71831e177c7ce2df/cachebox-5.2.3-cp314-cp314-win_amd64.whl", hash = "sha256:4afc8b8575e3228a42ad8d819de5fbbecc6bd0b521295966b00244be37ae3b9b", size = 291928, upload-time = "2026-04-10T12:21:44.621Z" }, + { url = "https://files.pythonhosted.org/packages/4a/01/8c79c07c8c6517fb2fe7d479dd87044e38aac5b9af0245b33fcd695eae37/cachebox-5.2.3-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:0e8a34b82be30d3d9fb7dfaf9a86ec2b3ab9bc264715909ef27fc3d3587324d2", size = 374325, upload-time = "2026-04-10T12:20:30.923Z" }, + { url = "https://files.pythonhosted.org/packages/7f/51/0fc26b923e80ab857ac99d5f7f3784dc941e7b4de361c204835233176ddf/cachebox-5.2.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:4d4e336aebf866463878ccd28a4d0ef4003ea216708cf4a02a7f198481b3af81", size = 355444, upload-time = "2026-04-10T12:20:16.879Z" }, + { url = "https://files.pythonhosted.org/packages/c1/6d/a6b399221f8dc4b3e01b37d3240ef5b8a7eb78cd9bfbb99b0e655dd01649/cachebox-5.2.3-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8b102fcdd97b0602bf5d6ba1a571bba3e3d6fa912b89fd768b0da5427408eab8", size = 393978, upload-time = "2026-04-10T12:18:59.753Z" }, + { url = "https://files.pythonhosted.org/packages/bd/f1/4c8f998c117c1941a82bd824d6687280c50167f21fea6392e41531d641e2/cachebox-5.2.3-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:245a79fb2c5d3bff252f4263f76210ef3ad7c2ff9b0234859b26974830a80491", size = 349298, upload-time = "2026-04-10T12:19:12.843Z" }, + { url = "https://files.pythonhosted.org/packages/d1/dd/683bc5a32a0da660d02fa248b880b71a2b834e9b54b8d272b5801282f402/cachebox-5.2.3-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:dd0e8dbd8fd4cf664c645c08f9e10508e133353756705c4a738e90a5406224b5", size = 370619, upload-time = "2026-04-10T12:19:25.298Z" }, + { url = "https://files.pythonhosted.org/packages/81/49/d6c47c78a7769b355076c5b635c2b538c8b88e8ceeb408e104d0f269b515/cachebox-5.2.3-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:fdb74294bdc33e39e26606919a9b2229038d5fac0edb80c9056683c08584d4a9", size = 385988, upload-time = "2026-04-10T12:19:37.638Z" }, + { url = "https://files.pythonhosted.org/packages/70/e2/b669555ada7fa1392e4cdb8a19f3367db5c6abef0fde8ab034a9747760df/cachebox-5.2.3-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bba3e9a7f52fa196b434522f39675f3b32a076976ef2373ded6f1065e99f4d20", size = 394090, upload-time = "2026-04-10T12:20:03.978Z" }, + { url = "https://files.pythonhosted.org/packages/8f/01/42916249e53fe4fcbdf0419fb55dbc09b9f377475376e1d7f4ae9c9bd6cd/cachebox-5.2.3-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:abb21f0f937fb66528f1b9f1a04874d6aa503e78bbb26f4cf33bf67faddbdd68", size = 421632, upload-time = "2026-04-10T12:19:51.048Z" }, + { url = "https://files.pythonhosted.org/packages/a1/54/34eebe18c6ed8ba27b1331b5e3d08bd8bb62f03ba81fbf47a2db0fa646f7/cachebox-5.2.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:dab6fd3189b0c746fb03e1915fd947aaca9112cedf26ef3a0c39383acf87d2e5", size = 563871, upload-time = "2026-04-10T12:20:45.417Z" }, + { url = "https://files.pythonhosted.org/packages/ba/b2/f92da0d54e4f18609588709090de8c81dd7c8b20ed6ac30f9b91bedbedf5/cachebox-5.2.3-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:b4e7d2935b9df11d3717f99c7237b6780f1f8c70e6a99b69b8430d89929ec825", size = 665677, upload-time = "2026-04-10T12:21:00.512Z" }, + { url = "https://files.pythonhosted.org/packages/43/9d/bf2d3dc949afe4d21fc7eb15b7524255e834b9252df6bba111e6686d1c6f/cachebox-5.2.3-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:611aa260fe1b2506330ff72f415e2cb4053c9c4e3776ac68fe2eedee0e1b91b1", size = 642067, upload-time = "2026-04-10T12:21:15.727Z" }, + { url = "https://files.pythonhosted.org/packages/6e/4f/a789eda189550d239fbaf165b9810f148e733e97a2a4eda7c4192295c7f8/cachebox-5.2.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:a424ffb8514a9cb49bacff7995b7c767625cb2239692bd6524245e8579e375cc", size = 608048, upload-time = "2026-04-10T12:21:31.156Z" }, + { url = "https://files.pythonhosted.org/packages/41/c3/590e161c04ffbd36e33933e6dcca5ffa40b5548e3121a21d77aad42af138/cachebox-5.2.3-cp314-cp314t-win32.whl", hash = "sha256:83988dd8e9075ee837e8407e26db49a9944ae74924d5db57b477444d7d98622c", size = 271694, upload-time = "2026-04-10T12:22:00.589Z" }, + { url = "https://files.pythonhosted.org/packages/66/f4/f60b8506df467261178afe918801df37c02c46ec2b8ce019760a14e2abe7/cachebox-5.2.3-cp314-cp314t-win_amd64.whl", hash = "sha256:dbda6390fa5070a19157ae35ab8066d3fe468634e0e9e21452c68ce7999c7d0c", size = 284212, upload-time = "2026-04-10T12:21:46.241Z" }, +] + +[[package]] +name = "colorama" +version = "0.4.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, +] + +[[package]] +name = "deepdiff" +version = "9.1.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "cachebox" }, + { name = "orderly-set" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/f9/6b/6a4a5aaf38535eb332c2856aa08e73ed7c549d0851b1215401af0a2db1a7/deepdiff-9.1.0.tar.gz", hash = "sha256:07e9e366fab4297755153c4eab795ad4ef3cbd0d51660e847f5751c6bd727687", size = 382149, upload-time = "2026-05-15T20:18:05.751Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c7/26/4a2bad8eb430d8d805a4642c4bff25103a37548d74ab346f8b1e024abcc5/deepdiff-9.1.0-py3-none-any.whl", hash = "sha256:80c0460e1993b04f6f0ca79abf25548b129fd218478c4ebb08f80560f5d10610", size = 184662, upload-time = "2026-05-15T20:18:03.956Z" }, +] + +[[package]] +name = "iniconfig" +version = "2.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" }, +] + +[[package]] +name = "orderly-set" +version = "5.5.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/4a/88/39c83c35d5e97cc203e9e77a4f93bf87ec89cf6a22ac4818fdcc65d66584/orderly_set-5.5.0.tar.gz", hash = "sha256:e87185c8e4d8afa64e7f8160ee2c542a475b738bc891dc3f58102e654125e6ce", size = 27414, upload-time = "2025-07-10T20:10:55.885Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/12/27/fb8d7338b4d551900fa3e580acbe7a0cf655d940e164cb5c00ec31961094/orderly_set-5.5.0-py3-none-any.whl", hash = "sha256:46f0b801948e98f427b412fcabb831677194c05c3b699b80de260374baa0b1e7", size = 13068, upload-time = "2025-07-10T20:10:54.377Z" }, +] + +[[package]] +name = "packaging" +version = "26.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" }, +] + +[[package]] +name = "pluggy" +version = "1.6.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" }, +] + +[[package]] +name = "pygments" +version = "2.20.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" }, +] + +[[package]] +name = "pytdbot" +version = "0.10.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "deepdiff" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ad/b7/3dcfac2045106ebfdeb514eaf15987d2df720ec174ead890b694fa606901/pytdbot-0.10.1.tar.gz", hash = "sha256:121e2456da6b112da63436ee267ae35df7e6e34086ac618f912503f3df7fab36", size = 558177, upload-time = "2026-06-29T13:26:57.847Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c1/6a/9a37f566855910daf039701d7739f71bba08ad57b0a47836cdf9e359014e/pytdbot-0.10.1-py3-none-any.whl", hash = "sha256:e7190cbfdb0c86e9ecc81aec09402ad6d5e08ba0f5bca93e792ed5fd644d6f46", size = 571466, upload-time = "2026-06-29T13:26:56.46Z" }, +] + +[[package]] +name = "pytest" +version = "9.1.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "iniconfig" }, + { name = "packaging" }, + { name = "pluggy" }, + { name = "pygments" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" }, +] + +[[package]] +name = "python-dotenv" +version = "1.2.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/82/ed/0301aeeac3e5353ef3d94b6ec08bbcabd04a72018415dcb29e588514bba8/python_dotenv-1.2.2.tar.gz", hash = "sha256:2c371a91fbd7ba082c2c1dc1f8bf89ca22564a087c2c287cd9b662adde799cf3", size = 50135, upload-time = "2026-03-01T16:00:26.196Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0b/d7/1959b9648791274998a9c3526f6d0ec8fd2233e4d4acce81bbae76b44b2a/python_dotenv-1.2.2-py3-none-any.whl", hash = "sha256:1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a", size = 22101, upload-time = "2026-03-01T16:00:25.09Z" }, +] + +[[package]] +name = "tdjson" +version = "1.8.65" +source = { registry = "https://pypi.org/simple" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/be/43/6d0b308fad89594cb57cb76308557a506b551ae52767a01108866ec8ab9a/tdjson-1.8.65-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:e09ecfc6482a26bb5941ad99cfa38d3dba8841d11d174adc541d588f9209f873", size = 14260753, upload-time = "2026-06-11T18:32:39.394Z" }, + { url = "https://files.pythonhosted.org/packages/b4/7b/746cb7b42898630193e6e69493d137733d126fb4b1c8208af1a9aa2b9499/tdjson-1.8.65-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:be1a116d4845f98d07b1cabfad9c6c92adc5ff1cb8b78a96f094320ae5259083", size = 16952615, upload-time = "2026-06-11T18:32:41.86Z" }, + { url = "https://files.pythonhosted.org/packages/e6/9d/8ab29f89706e6837ee84c084ef1142dc9e256b051b5e6c8792069e6857d6/tdjson-1.8.65-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:eb8679e8a92fb3fd434f84ec32feda4cd5ce77dee3c142f6840607cb5341a583", size = 17161981, upload-time = "2026-06-11T18:32:44.449Z" }, + { url = "https://files.pythonhosted.org/packages/6e/ca/16d5af6d9d5f500871b777095c60ea4be1e2bf97e6f694d3b97b0826f234/tdjson-1.8.65-cp312-cp312-win_amd64.whl", hash = "sha256:17b7e69e73ef69aa9708dbd421c1e5ca637d83e45d6d2d5b5b4185e3e035f5b3", size = 12333381, upload-time = "2026-06-11T18:32:46.544Z" }, + { url = "https://files.pythonhosted.org/packages/09/d2/8cbd95d7312b3d44df012129aad0455329b64fed20ecbcdd89a581a312e4/tdjson-1.8.65-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:e72d979689d6a0a0683f29448a61295f68db78bf77b3cb04921ad5ca2ff391dc", size = 14260758, upload-time = "2026-06-11T18:32:48.851Z" }, + { url = "https://files.pythonhosted.org/packages/95/9b/97df847fb0fc9d37e373207a01bc2dd8e9eb7fccd226536673c2335ac268/tdjson-1.8.65-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:692fbeb2689ab2fb90e8c8acf30017018484c9b4049e94b4056df46b175f4081", size = 16952611, upload-time = "2026-06-11T18:32:50.934Z" }, + { url = "https://files.pythonhosted.org/packages/32/1b/d31cb772fba909d3496071cbbfa855db1cbac780bfbc369b1949a117f5fb/tdjson-1.8.65-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:49d126af4411d3a0ea60a6de0b9915349ad2e312f58acd77eb22c2ac0fc096b0", size = 17161975, upload-time = "2026-06-11T18:32:53.13Z" }, + { url = "https://files.pythonhosted.org/packages/d5/52/b1437039c308447bb115a6db45a1fc06995b3cbb22c06764d558c7b068d4/tdjson-1.8.65-cp313-cp313-win_amd64.whl", hash = "sha256:2e3f1918e0f72e0d94102b34e2962fd2358d2966dc6a49ebff4570261a682d3e", size = 12333384, upload-time = "2026-06-11T18:32:56.038Z" }, + { url = "https://files.pythonhosted.org/packages/52/b1/3dd45c1ed4c5d41ecfe9396b46ddc6049914fff910a9cfa6769062e125e7/tdjson-1.8.65-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:3e372eac06e1dbf9257ba3ab8a79c1c1f30fec800723862c5a0ca27773a8a78d", size = 14260764, upload-time = "2026-06-11T18:32:58.144Z" }, + { url = "https://files.pythonhosted.org/packages/bc/d0/bcd9c621390ac15d82a6241256c128428eef02137fc2d68efc76d05cd5cd/tdjson-1.8.65-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:12726af9debcf0516a14dcbc743d764e548fcff2141a6cc1c5d0a61d2d0372c9", size = 16952682, upload-time = "2026-06-11T18:33:00.338Z" }, + { url = "https://files.pythonhosted.org/packages/62/ac/f1d7bc8ebc48b49c17689a8b53880403c6f429c4440d12c0ab4e1bbd6643/tdjson-1.8.65-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:acfbba1039cfff5e1c2f56ec1cf5b8e30c6145fea22b85aca490d0dc4890357e", size = 17161979, upload-time = "2026-06-11T18:33:02.615Z" }, + { url = "https://files.pythonhosted.org/packages/6f/d5/6ae331cb0e802efa093facdf1c9361841547a48e2f612806933eea688015/tdjson-1.8.65-cp314-cp314-win_amd64.whl", hash = "sha256:efa5ef131be782e37df10a92fae81c14dce1a5e0cf01b5da0fbf2274b01044cc", size = 12706734, upload-time = "2026-06-11T18:33:04.963Z" }, + { url = "https://files.pythonhosted.org/packages/1f/10/79dae2a3f11c2dfa1d9256f1388395eff40e35486c797d8b2f39612f8a23/tdjson-1.8.65-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:f689ab06b76925ef2086729954c7be0620c1129dba88ab3e8e33a53890a1f3c4", size = 14260991, upload-time = "2026-06-11T18:33:07.233Z" }, + { url = "https://files.pythonhosted.org/packages/fa/7b/da77b36eebfb5336882b082cc58e018a49c26fa0426ed3cc6425a3c66dba/tdjson-1.8.65-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:aa51fb29e5c155111cfd78c3ba157919bc17f91e8f69b19e01e37fedb43fa1c9", size = 16954902, upload-time = "2026-06-11T18:33:09.38Z" }, + { url = "https://files.pythonhosted.org/packages/8a/8e/e9ff905dba0da399c09539e40866aae2f7e9f17532639d478d5e32f83d0b/tdjson-1.8.65-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:05d14e02df544f1f45bc59d2ebcbe26dd31fff850cf2c1ae6fb8b7855bbdf179", size = 17163905, upload-time = "2026-06-11T18:33:11.962Z" }, + { url = "https://files.pythonhosted.org/packages/9f/24/25c5112fb191788cc8c070ce103cab41d58c9fc8aff2851eb84bc099e214/tdjson-1.8.65-cp314-cp314t-win_amd64.whl", hash = "sha256:62ba43c93f3543a31b84346151822cc232eae9841e4516a1a5ecddad2de1aeb2", size = 12707830, upload-time = "2026-06-11T18:33:14.17Z" }, +] + +[[package]] +name = "tdlib-media-poc" +version = "0.1.0" +source = { virtual = "." } +dependencies = [ + { name = "pytdbot" }, + { name = "python-dotenv" }, + { name = "tdjson" }, +] + +[package.dev-dependencies] +dev = [ + { name = "pytest" }, +] + +[package.metadata] +requires-dist = [ + { name = "pytdbot", specifier = ">=0.10.1" }, + { name = "python-dotenv", specifier = ">=1.0.1" }, + { name = "tdjson", specifier = ">=1.8.65" }, +] + +[package.metadata.requires-dev] +dev = [{ name = "pytest", specifier = ">=8.0.0" }] diff --git a/mcp/.env.example b/mcp/.env.example index 62ed2c3..834426c 100644 --- a/mcp/.env.example +++ b/mcp/.env.example @@ -17,6 +17,12 @@ TELEGRAM_API_HASH= # Optional: minimum seconds between retention cleanup scans # TELEGRAM_DOWNLOAD_CLEANUP_INTERVAL_SECONDS=3600 +# Optional: route large media downloads on the main account through TDLib. +# Requires `pip install -e ".[tdlib]"` and a one-time `mcp/scripts/tdlib_login.py` run. +# TELEGRAM_TDLIB_ENABLED=false +# TELEGRAM_TDLIB_SESSION_DIR=~/.telegram-mcp-tdlib/main +# TELEGRAM_TDLIB_DOWNLOAD_THRESHOLD_MB=20 + # Optional: HTTP transport settings for launchd/local daemon mode # TELEGRAM_MCP_TRANSPORT=streamable-http # TELEGRAM_MCP_HOST=127.0.0.1 diff --git a/mcp/docs/superpowers/plans/2026-07-01-tdlib-large-media-download-implementation.md b/mcp/docs/superpowers/plans/2026-07-01-tdlib-large-media-download-implementation.md new file mode 100644 index 0000000..a74edda --- /dev/null +++ b/mcp/docs/superpowers/plans/2026-07-01-tdlib-large-media-download-implementation.md @@ -0,0 +1,893 @@ +# TDLib Large-Media-Download Graduation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Graduate the TDLib media-download POC into a narrowly-scoped production capability — TDLib as an auto-routed backend for large media downloads on the `main` account only, with every failure mode falling back to the existing Telethon path. + +**Architecture:** `download_post()` (`mcp/src/telegram_mcp/download_post.py`) keeps its existing Telethon connect/resolve-entity/resolve-message flow unchanged. After the message is resolved, a pure routing-decision function decides (account, enabled flag, content type, size vs. threshold) whether to attempt TDLib. On a route decision, a new `tdlib_download.py` module opens a fresh in-process TDLib client against an already-authorized on-disk session, downloads the file, and hands the path back; any failure — including a session-lock timeout — falls back to the Telethon `client`/`msg` the function already has open, so no extra Telegram round-trip is ever spent deciding or falling back. + +**Tech Stack:** Python ≥3.12, Telethon (existing), `pytdbot`/`tdjson` (new, optional extra), `fcntl`-based advisory locking (existing pattern, extended), `unittest`-based test suite (existing convention — this repo runs tests with `unittest discover`, not `pytest`). + +## Global Constraints + +- TDLib is **not** a general runtime: reads, search, sends, and all other Telegram operations stay on Telethon, unchanged. (spec Non-Goals) +- Only the `main` account is eligible for TDLib routing. No other account gets TDLib in this iteration. (spec Non-Goals) +- No persistent TDLib daemon — every download gets a fresh, in-process TDLib client reconnecting to an already-authorized on-disk session, then tears down. (spec Non-Goals) +- This does not touch the MCP tool surface (`download_media_batch`, `download_dialog_media`). TDLib routing applies only to `download_post()` / `tg download`. (spec Non-Goals) +- `pytdbot`/`tdjson` are an **optional** extras group (`telegram-mcp[tdlib]`), never a hard dependency. The `pytdbot` import is lazy, inside functions, guarded by `TELEGRAM_TDLIB_ENABLED` — nothing in the rest of the package imports `tdlib_download.py` at module load time (i.e. `download_post.py`'s top-level imports must not include `tdlib_download`; import it locally inside the function that needs it). (spec Dependencies) +- `TELEGRAM_TDLIB_ENABLED` defaults to unset/`false` — the capability stays off until an operator turns it on explicitly. (spec Configuration) +- Isolation marker for the TDLib files directory stays `.telegram-mcp` (not `.telegram-mcp-tdlib`) — same guard as the POC, ported verbatim. (spec Components) +- Every failure mode on the TDLib path falls back to Telethon using the connection/message already fetched during routing — never a hard failure introduced by this change. (spec Error Handling / Fallback) +- Test baseline: `cd mcp && TELEGRAM_API_ID=1 TELEGRAM_API_HASH=hash PYTHONPATH=src .venv/bin/python -m unittest discover -s tests -p 'test_*.py'` currently passes 379/379. Every task must keep this command green. + +--- + +## File map + +**Create** + +- `mcp/docs/superpowers/plans/2026-07-01-tdlib-large-media-download-implementation.md` (this file) +- `mcp/src/telegram_mcp/tdlib_download.py` +- `mcp/tests/test_tdlib_download.py` +- `mcp/scripts/tdlib_login.py` + +**Modify** + +- `mcp/src/telegram_mcp/locking.py` +- `mcp/tests/test_locking.py` +- `mcp/src/telegram_mcp/download_post.py` +- `mcp/tests/test_download_post.py` +- `mcp/pyproject.toml` +- `mcp/.env.example` +- `docs/operator-workflows.md` + +**Keep untouched** + +- `mcp/src/telegram_mcp/tools/media_tools.py` and the MCP `download_media_batch`/`download_dialog_media` tools +- `experiments/tdlib-media-poc/` (POC stays as historical reference, not imported from production code) +- Everything about non-`main` accounts + +--- + +## Task 1: Timeout-bounded lock acquisition in `locking.py` + +**Files:** +- Modify: `mcp/src/telegram_mcp/locking.py` +- Test: `mcp/tests/test_locking.py` + +**Interfaces:** +- Consumes: existing `FileSessionLock` class (`acquire()` raises `SessionLockError`, `release()`). +- Produces: `try_acquire_with_timeout(lock: FileSessionLock, *, timeout_seconds: float = 5.0, poll_interval_seconds: float = 0.2) -> bool` — `True` if acquired (caller must `release()` later), `False` if the timeout elapsed without acquiring (caller falls back, does **not** call `release()`). Task 2 imports this from `telegram_mcp.locking`. + +- [ ] **Step 1: Write the failing test** + +Add to `mcp/tests/test_locking.py` (append to the existing `LockingTests` class): + +```python + def test_try_acquire_with_timeout_returns_false_when_held(self): + with tempfile.TemporaryDirectory() as tmp: + lock_path = Path(tmp) / "telegram.lock" + + holder = FileSessionLock(lock_path) + holder.acquire() + try: + waiter = FileSessionLock(lock_path) + acquired = try_acquire_with_timeout( + waiter, timeout_seconds=0.3, poll_interval_seconds=0.1 + ) + self.assertFalse(acquired) + finally: + holder.release() + + def test_try_acquire_with_timeout_succeeds_once_free(self): + with tempfile.TemporaryDirectory() as tmp: + lock_path = Path(tmp) / "telegram.lock" + + lock = FileSessionLock(lock_path) + acquired = try_acquire_with_timeout(lock, timeout_seconds=0.3) + try: + self.assertTrue(acquired) + finally: + lock.release() +``` + +Update the import line at the top of `mcp/tests/test_locking.py`: + +```python +from telegram_mcp.locking import FileSessionLock, SessionLockError, try_acquire_with_timeout +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd mcp && PYTHONPATH=src .venv/bin/python -m unittest tests.test_locking -v` +Expected: FAIL with `ImportError: cannot import name 'try_acquire_with_timeout'` + +- [ ] **Step 3: Implement `try_acquire_with_timeout`** + +In `mcp/src/telegram_mcp/locking.py`, add `import time` to the top-level imports (alongside the existing `import fcntl` / `import os`), and append this function after the `FileSessionLock` class: + +```python +def try_acquire_with_timeout( + lock: FileSessionLock, + *, + timeout_seconds: float = 5.0, + poll_interval_seconds: float = 0.2, +) -> bool: + """Retry ``lock.acquire()`` until it succeeds or ``timeout_seconds`` elapse. + + Returns True if acquired (caller owns the lock and must call release()). + Returns False if the timeout elapsed without acquiring — the caller + should fall back rather than treat this as an error. + """ + deadline = time.monotonic() + timeout_seconds + while True: + try: + lock.acquire() + return True + except SessionLockError: + if time.monotonic() >= deadline: + return False + time.sleep(poll_interval_seconds) +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd mcp && PYTHONPATH=src .venv/bin/python -m unittest tests.test_locking -v` +Expected: PASS (3 tests: the existing one plus the 2 new ones) + +- [ ] **Step 5: Commit** + +```bash +git add mcp/src/telegram_mcp/locking.py mcp/tests/test_locking.py +git commit -m "feat: add timeout-bounded lock acquisition for TDLib session locking" +``` + +--- + +## Task 2: `tdlib_download.py` — routing decision, isolation guard, TDLib client wiring + +**Files:** +- Create: `mcp/src/telegram_mcp/tdlib_download.py` +- Test: `mcp/tests/test_tdlib_download.py` +- Modify: `mcp/pyproject.toml` (optional extras group) +- Modify: `mcp/.env.example` (new env vars) + +**Interfaces:** +- Consumes: `telegram_mcp.locking.FileSessionLock`, `telegram_mcp.locking.try_acquire_with_timeout` (Task 1). +- Produces (consumed by Task 3): + - `SUPPORTED_CONTENT_KINDS: frozenset[str]` = `{"video", "document", "photo", "audio"}` + - `should_route_to_tdlib(*, account: str, tdlib_enabled: bool, content_kind: str | None, media_size_bytes: int | None, threshold_mb: float) -> bool` + - `async def download_via_tdlib(*, link: str, session_dir: Path) -> Path` — raises `RuntimeError` (or a subclass) on any failure; returns the local downloaded file path on success. + +- [ ] **Step 1: Write the failing tests for the pure, pytdbot-free logic** + +Create `mcp/tests/test_tdlib_download.py`: + +```python +import importlib.util +import unittest + +from telegram_mcp.tdlib_download import assert_isolated_from_telethon, should_route_to_tdlib + +PYTDBOT_AVAILABLE = importlib.util.find_spec("pytdbot") is not None + + +class ShouldRouteToTdlibTests(unittest.TestCase): + def test_routes_when_all_conditions_met(self): + self.assertTrue( + should_route_to_tdlib( + account="main", + tdlib_enabled=True, + content_kind="video", + media_size_bytes=30 * 1024 * 1024, + threshold_mb=20, + ) + ) + + def test_rejects_non_main_account(self): + self.assertFalse( + should_route_to_tdlib( + account="pl", + tdlib_enabled=True, + content_kind="video", + media_size_bytes=30 * 1024 * 1024, + threshold_mb=20, + ) + ) + + def test_rejects_when_disabled(self): + self.assertFalse( + should_route_to_tdlib( + account="main", + tdlib_enabled=False, + content_kind="video", + media_size_bytes=30 * 1024 * 1024, + threshold_mb=20, + ) + ) + + def test_rejects_unsupported_content_kind(self): + self.assertFalse( + should_route_to_tdlib( + account="main", + tdlib_enabled=True, + content_kind=None, + media_size_bytes=30 * 1024 * 1024, + threshold_mb=20, + ) + ) + + def test_rejects_below_threshold(self): + self.assertFalse( + should_route_to_tdlib( + account="main", + tdlib_enabled=True, + content_kind="video", + media_size_bytes=1 * 1024 * 1024, + threshold_mb=20, + ) + ) + + def test_rejects_unknown_size(self): + self.assertFalse( + should_route_to_tdlib( + account="main", + tdlib_enabled=True, + content_kind="video", + media_size_bytes=None, + threshold_mb=20, + ) + ) + + +class AssertIsolatedFromTelethonTests(unittest.TestCase): + def test_rejects_telethon_session_tree(self): + with self.assertRaises(ValueError): + assert_isolated_from_telethon("/Users/x/.telegram-mcp/main") + + def test_accepts_isolated_directory(self): + assert_isolated_from_telethon("/Users/x/.telegram-mcp-tdlib/main") + + +@unittest.skipUnless(PYTDBOT_AVAILABLE, "pytdbot not installed (optional [tdlib] extra)") +class PytdbotDependentTests(unittest.TestCase): + def test_raise_if_error_passes_through_non_error_result(self): + from telegram_mcp.tdlib_download import raise_if_error + + self.assertEqual(raise_if_error("some value"), "some value") + + def test_raise_if_error_raises_on_pytdbot_error(self): + import pytdbot + + from telegram_mcp.tdlib_download import raise_if_error + + error = pytdbot.types.Error(code=400, message="MESSAGE_ID_INVALID") + with self.assertRaisesRegex(RuntimeError, "MESSAGE_ID_INVALID"): + raise_if_error(error) + + def test_extract_file_id_from_message_video(self): + import pytdbot + + from telegram_mcp.tdlib_download import extract_file_id_from_message + + message = pytdbot.types.Message( + content=pytdbot.types.MessageVideo( + video=pytdbot.types.Video(video=pytdbot.types.File(id=555, size=52_428_800)) + ) + ) + self.assertEqual(extract_file_id_from_message(message), 555) + + def test_extract_file_id_from_message_unsupported_type(self): + import pytdbot + + from telegram_mcp.tdlib_download import extract_file_id_from_message + + message = pytdbot.types.Message(content=pytdbot.types.MessageText()) + with self.assertRaisesRegex(ValueError, "unsupported message content type"): + extract_file_id_from_message(message) + + def test_build_client_rejects_telethon_session_tree(self): + from telegram_mcp.tdlib_download import build_client + + with self.assertRaises(ValueError): + build_client( + api_id=1, + api_hash="hash", + files_directory="/Users/x/.telegram-mcp/main", + ) + + +if __name__ == "__main__": + unittest.main() +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd mcp && PYTHONPATH=src .venv/bin/python -m unittest tests.test_tdlib_download -v` +Expected: FAIL with `ModuleNotFoundError: No module named 'telegram_mcp.tdlib_download'` + +- [ ] **Step 3: Declare the optional dependency and env vars** + +In `mcp/pyproject.toml`, add an `[project.optional-dependencies]` table right after the existing `dependencies` list: + +```toml +[project.optional-dependencies] +tdlib = [ + "Pytdbot>=0.10.1", + "tdjson>=1.8.65", +] +``` + +In `mcp/.env.example`, add this block right after the `TELEGRAM_DOWNLOAD_*` group (after the `TELEGRAM_DOWNLOAD_CLEANUP_INTERVAL_SECONDS` line): + +``` +# Optional: route large media downloads on the main account through TDLib. +# Requires `pip install -e ".[tdlib]"` and a one-time `mcp/scripts/tdlib_login.py` run. +# TELEGRAM_TDLIB_ENABLED=false +# TELEGRAM_TDLIB_SESSION_DIR=~/.telegram-mcp-tdlib/main +# TELEGRAM_TDLIB_DOWNLOAD_THRESHOLD_MB=20 +``` + +- [ ] **Step 4: Implement `tdlib_download.py`** + +Create `mcp/src/telegram_mcp/tdlib_download.py`: + +```python +"""TDLib backend for large media downloads on the `main` account only. + +Graduated from the isolated POC (experiments/tdlib-media-poc/) per +mcp/docs/superpowers/specs/2026-07-01-tdlib-large-media-download-design.md. +`pytdbot` is an optional dependency (`telegram-mcp[tdlib]`) — every function +here that needs it imports it lazily, so importing this module never +requires pytdbot to be installed. +""" + +from __future__ import annotations + +import os +from pathlib import Path + +from .locking import FileSessionLock, try_acquire_with_timeout + +TELETHON_SESSION_DIR_MARKER = ".telegram-mcp" + +SUPPORTED_CONTENT_KINDS = frozenset({"video", "document", "photo", "audio"}) + + +class TdlibDownloadError(RuntimeError): + """Raised for any TDLib download failure; callers should fall back to Telethon.""" + + +def assert_isolated_from_telethon(files_directory: str) -> None: + if TELETHON_SESSION_DIR_MARKER in files_directory: + raise ValueError( + f"files_directory must not overlap the Telethon session tree " + f"(found {TELETHON_SESSION_DIR_MARKER!r})" + ) + + +def build_client( + api_id: int, + api_hash: str, + files_directory: str, + database_encryption_key: str = "telegram-mcp-tdlib", +): + assert_isolated_from_telethon(files_directory) + import pytdbot + + return pytdbot.Client( + api_id=api_id, + api_hash=api_hash, + files_directory=files_directory, + database_encryption_key=database_encryption_key, + use_file_database=True, + use_chat_info_database=False, + use_message_database=False, + ) + + +def raise_if_error(result): + import pytdbot + + if isinstance(result, pytdbot.types.Error): + raise TdlibDownloadError(f"TDLib error {result['code']}: {result['message']}") + return result + + +def extract_file_id_from_message(message) -> int: + content = message["content"] + content_type = content.getType() + if content_type == "messageVideo": + return content["video"]["video"]["id"] + if content_type == "messageDocument": + return content["document"]["document"]["id"] + if content_type == "messagePhoto": + sizes = content["photo"]["sizes"] + return sizes[-1]["photo"]["id"] + if content_type == "messageAudio": + return content["audio"]["audio"]["id"] + raise ValueError(f"unsupported message content type for download: {content_type!r}") + + +def should_route_to_tdlib( + *, + account: str, + tdlib_enabled: bool, + content_kind: str | None, + media_size_bytes: int | None, + threshold_mb: float, +) -> bool: + if account != "main": + return False + if not tdlib_enabled: + return False + if content_kind not in SUPPORTED_CONTENT_KINDS: + return False + if media_size_bytes is None: + return False + return media_size_bytes >= threshold_mb * 1024 * 1024 + + +async def download_via_tdlib(*, link: str, session_dir: Path) -> Path: + """Resolve `link` via TDLib and download it fully. Returns the local file + path on success. Raises TdlibDownloadError on any failure (lock timeout, + TDLib error, incomplete download) — the caller decides to fall back.""" + lock = FileSessionLock(session_dir / "download.lock") + if not try_acquire_with_timeout(lock, timeout_seconds=5.0): + raise TdlibDownloadError("could not acquire TDLib session lock within 5s") + + try: + client = build_client( + api_id=int(os.environ["TELEGRAM_API_ID"]), + api_hash=os.environ["TELEGRAM_API_HASH"], + files_directory=str(session_dir), + ) + await client.start() + try: + link_info = raise_if_error(await client.getMessageLinkInfo(url=link)) + message = raise_if_error( + await client.getMessage( + chat_id=link_info["chat_id"], message_id=link_info["message"]["id"] + ) + ) + file_id = extract_file_id_from_message(message) + result = raise_if_error( + await client.downloadFile( + file_id=file_id, priority=1, synchronous=True, offset=0, limit=0 + ) + ) + local = result["local"] + if not local or not bool(local["is_downloading_completed"]): + raise TdlibDownloadError("TDLib download did not complete") + return Path(local["path"]) + finally: + await client.stop() + finally: + lock.release() +``` + +- [ ] **Step 5: Run test to verify it passes** + +Run: `cd mcp && PYTHONPATH=src .venv/bin/python -m unittest tests.test_tdlib_download -v` +Expected: PASS — `ShouldRouteToTdlibTests` (6) and `AssertIsolatedFromTelethonTests` (2) run and pass; `PytdbotDependentTests` (5) show as `skipped` since `pytdbot` isn't installed in the portable dev/CI venv. + +- [ ] **Step 6: Run the full suite to confirm no regressions** + +Run: `cd mcp && TELEGRAM_API_ID=1 TELEGRAM_API_HASH=hash PYTHONPATH=src .venv/bin/python -m unittest discover -s tests -p 'test_*.py'` +Expected: `OK` (387 tests: 379 baseline + 8 new non-skipped) + +- [ ] **Step 7: Commit** + +```bash +git add mcp/src/telegram_mcp/tdlib_download.py mcp/tests/test_tdlib_download.py mcp/pyproject.toml mcp/.env.example +git commit -m "feat: add tdlib_download module with routing decision and optional pytdbot backend" +``` + +--- + +## Task 3: Wire TDLib routing and fallback into `download_post()` + +**Files:** +- Modify: `mcp/src/telegram_mcp/download_post.py` +- Test: `mcp/tests/test_download_post.py` + +**Interfaces:** +- Consumes: `telegram_mcp.tdlib_download.should_route_to_tdlib`, `telegram_mcp.tdlib_download.download_via_tdlib` (Task 2), `telegram_mcp.telemetry.record_telemetry` (existing — signature `record_telemetry(event: str, **fields) -> None`, never raises). +- Produces: `_telethon_media_kind(msg) -> str | None`, `_telethon_media_size_bytes(msg) -> int | None` — new pure helpers, unit-tested directly. `download_post()`'s returned dict gains no new keys (existing shape preserved); backend selection is internal + telemetry-only. + +- [ ] **Step 1: Write the failing tests for the new pure helpers** + +Add to `mcp/tests/test_download_post.py`, in a new test class (append before the `if __name__ == "__main__":` line), and add the two new names to the existing import line: + +```python +from telegram_mcp.download_post import ( + ParsedLink, + _ext_from_message, + _telethon_media_kind, + _telethon_media_size_bytes, + parse_post_link, +) +``` + +```python +class TelethonMediaKindTests(unittest.TestCase): + def test_photo_is_photo_kind(self): + msg = SimpleNamespace(media=SimpleNamespace(document=None, photo=object())) + self.assertEqual(_telethon_media_kind(msg), "photo") + + def test_video_mime_is_video_kind(self): + doc = SimpleNamespace(attributes=[], mime_type="video/mp4", size=123) + msg = SimpleNamespace(media=SimpleNamespace(document=doc, photo=None)) + self.assertEqual(_telethon_media_kind(msg), "video") + + def test_audio_mime_is_audio_kind(self): + doc = SimpleNamespace(attributes=[], mime_type="audio/mpeg", size=123) + msg = SimpleNamespace(media=SimpleNamespace(document=doc, photo=None)) + self.assertEqual(_telethon_media_kind(msg), "audio") + + def test_other_document_mime_is_document_kind(self): + doc = SimpleNamespace(attributes=[], mime_type="application/pdf", size=123) + msg = SimpleNamespace(media=SimpleNamespace(document=doc, photo=None)) + self.assertEqual(_telethon_media_kind(msg), "document") + + def test_no_media_is_unsupported(self): + msg = SimpleNamespace(media=None) + self.assertIsNone(_telethon_media_kind(msg)) + + +class TelethonMediaSizeBytesTests(unittest.TestCase): + def test_document_size(self): + doc = SimpleNamespace(size=123456) + msg = SimpleNamespace(media=SimpleNamespace(document=doc, photo=None)) + self.assertEqual(_telethon_media_size_bytes(msg), 123456) + + def test_photo_largest_size(self): + sizes = [SimpleNamespace(size=100), SimpleNamespace(size=9000)] + photo = SimpleNamespace(sizes=sizes) + msg = SimpleNamespace(media=SimpleNamespace(document=None, photo=photo)) + self.assertEqual(_telethon_media_size_bytes(msg), 9000) + + def test_no_media_is_none(self): + msg = SimpleNamespace(media=None) + self.assertIsNone(_telethon_media_size_bytes(msg)) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `cd mcp && PYTHONPATH=src .venv/bin/python -m unittest tests.test_download_post -v` +Expected: FAIL with `ImportError: cannot import name '_telethon_media_kind'` + +- [ ] **Step 3: Implement the two helpers and wire routing/fallback** + +In `mcp/src/telegram_mcp/download_post.py`, add these two functions right after `_ext_from_message` (same file, same style): + +```python +def _telethon_media_kind(msg) -> str | None: + media = getattr(msg, "media", None) + if media is None: + return None + if getattr(media, "photo", None) is not None: + return "photo" + doc = getattr(media, "document", None) + if doc is None: + return None + mime = getattr(doc, "mime_type", "") or "" + if mime.startswith("video/"): + return "video" + if mime.startswith("audio/"): + return "audio" + return "document" + + +def _telethon_media_size_bytes(msg) -> int | None: + media = getattr(msg, "media", None) + if media is None: + return None + photo = getattr(media, "photo", None) + if photo is not None: + sizes = getattr(photo, "sizes", None) or [] + if not sizes: + return None + return max(getattr(s, "size", 0) for s in sizes) + doc = getattr(media, "document", None) + if doc is not None: + return getattr(doc, "size", None) + return None +``` + +Add the telemetry import to the top-level imports (alongside the existing `.mcp_http_client` import): + +```python +from .telemetry import record_telemetry +``` + +Then, in `download_post()`, replace this block: + +```python + out = dest_dir / f"{parsed.label}_{parsed.message_id}{_ext_from_message(msg)}" + progress = None if quiet else _make_progress() + saved = await client.download_media(msg, file=str(out), progress_callback=progress) +``` + +with: + +```python + out = dest_dir / f"{parsed.label}_{parsed.message_id}{_ext_from_message(msg)}" + + from . import tdlib_download # lazy: keeps pytdbot fully optional at module load time + + tdlib_enabled = os.environ.get("TELEGRAM_TDLIB_ENABLED", "false").strip().lower() == "true" + threshold_mb = float(os.environ.get("TELEGRAM_TDLIB_DOWNLOAD_THRESHOLD_MB", "20")) + route_to_tdlib = tdlib_download.should_route_to_tdlib( + account=account, + tdlib_enabled=tdlib_enabled, + content_kind=_telethon_media_kind(msg), + media_size_bytes=_telethon_media_size_bytes(msg), + threshold_mb=threshold_mb, + ) + + tdlib_backend_used = False + fallback_reason: str | None = None + saved: str | None = None + + if route_to_tdlib: + session_dir = Path( + os.environ.get("TELEGRAM_TDLIB_SESSION_DIR", "~/.telegram-mcp-tdlib/main") + ).expanduser() + try: + tdlib_path = await tdlib_download.download_via_tdlib(link=link, session_dir=session_dir) + shutil.copy2(tdlib_path, out) + saved = str(out) + tdlib_backend_used = True + except Exception as exc: # noqa: BLE001 - any TDLib failure falls back to Telethon + fallback_reason = str(exc) + + if saved is None: + progress = None if quiet else _make_progress() + saved = await client.download_media(msg, file=str(out), progress_callback=progress) + + record_telemetry( + "download_post_backend", + backend="tdlib" if tdlib_backend_used else "telethon", + account=account, + route_attempted=route_to_tdlib, + fallback_reason=fallback_reason, + ) +``` + +Note: `saved` was previously assigned unconditionally by the last line of the replaced block; it is now assigned either by the TDLib branch or the Telethon fallback branch, so the function's final `saved_path = Path(saved) if saved else out` line below stays correct unchanged. + +- [ ] **Step 4: Run test to verify it passes** + +Run: `cd mcp && PYTHONPATH=src .venv/bin/python -m unittest tests.test_download_post -v` +Expected: PASS (13 tests: 6 existing + 5 kind tests + 3 size tests... run and check actual count in output; all green) + +- [ ] **Step 5: Run the full suite to confirm no regressions** + +Run: `cd mcp && TELEGRAM_API_ID=1 TELEGRAM_API_HASH=hash PYTHONPATH=src .venv/bin/python -m unittest discover -s tests -p 'test_*.py'` +Expected: `OK` + +- [ ] **Step 6: Commit** + +```bash +git add mcp/src/telegram_mcp/download_post.py mcp/tests/test_download_post.py +git commit -m "feat: route large main-account downloads through TDLib with Telethon fallback" +``` + +--- + +## Task 4: Production TDLib login script + +**Files:** +- Create: `mcp/scripts/tdlib_login.py` + +**Interfaces:** +- Consumes: `telegram_mcp.tdlib_download.build_client`, `telegram_mcp.tdlib_download.raise_if_error` (Task 2). +- Produces: a standalone CLI (`--phone`/`--code`/`--password`), no importable interface consumed by later tasks. + +- [ ] **Step 1: Port the script** + +Create `mcp/scripts/tdlib_login.py` (adapted from `experiments/tdlib-media-poc/benchmark/login_tdlib.py`, pointed at the production session dir and reusing `telegram_mcp.tdlib_download` instead of duplicating `build_client`/`raise_if_error`): + +```python +"""Interactive-by-invocation TDLib login for the production `main`-account +media-download backend. + +pytdbot's Client.start() only auto-drives the authorization state machine for +bot-token logins. For a real user account it does nothing at +authorizationStateWaitPhoneNumber, so this script drives the state machine +manually across multiple invocations, since a live phone/SMS code can't be fed +into a blocking input() call from outside a real TTY: + + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --phone +15551234567 + # (Telegram sends a login code to your other active sessions) + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --code 12345 + # (only if the account has 2FA enabled) + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --password ... + +Each invocation reconnects to the same persistent session under +TELEGRAM_TDLIB_SESSION_DIR (default ~/.telegram-mcp-tdlib/main), fully +separate from the Telethon session tree telegram-mcp owns. + +Requires the optional tdlib extra: pip install -e ".[tdlib]" +""" + +import argparse +import asyncio +import os +from pathlib import Path + +from dotenv import load_dotenv + +from telegram_mcp.tdlib_download import build_client, raise_if_error + +_STATE_TO_REQUIRED_ARG = { + "authorizationStateWaitPhoneNumber": "phone", + "authorizationStateWaitCode": "code", + "authorizationStateWaitPassword": "password", +} + + +def required_arg_for_state(state: str) -> str | None: + return _STATE_TO_REQUIRED_ARG.get(state) + + +async def wait_for_stable_state(client) -> str: + for _ in range(20): + state = client.authorization_state + if state and state != "authorizationStateWaitTdlibParameters": + return state + await asyncio.sleep(0.5) + return client.authorization_state + + +async def main(phone: str | None, code: str | None, password: str | None) -> None: + load_dotenv(Path.home() / ".telegram-mcp" / "launchd.env") + api_id = int(os.environ["TELEGRAM_API_ID"]) + api_hash = os.environ["TELEGRAM_API_HASH"] + session_dir = Path( + os.environ.get("TELEGRAM_TDLIB_SESSION_DIR", "~/.telegram-mcp-tdlib/main") + ).expanduser() + + client = build_client(api_id=api_id, api_hash=api_hash, files_directory=str(session_dir)) + await client.start(wait_login=False) + + state = await wait_for_stable_state(client) + print(f"Current authorization state: {state}") + + required_arg = required_arg_for_state(state) + supplied = {"phone": phone, "code": code, "password": password}.get(required_arg) + + if required_arg and not supplied: + print( + f"Need --{required_arg}. Re-run: " + f"PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --{required_arg} " + ) + await client.stop() + return + + if state == "authorizationStateWaitPhoneNumber": + raise_if_error(await client.setAuthenticationPhoneNumber(phone_number=phone)) + print("Code requested. Check Telegram on your other devices, then run with --code.") + elif state == "authorizationStateWaitCode": + raise_if_error(await client.checkAuthenticationCode(code=code)) + print("Code accepted.") + elif state == "authorizationStateWaitPassword": + raise_if_error(await client.checkAuthenticationPassword(password=password)) + print("Password accepted.") + elif state == "authorizationStateReady": + me = raise_if_error(await client.getMe()) + print(f"Already logged in: {me}") + + await asyncio.sleep(1) + print(f"Authorization state now: {client.authorization_state}") + await client.stop() + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--phone") + parser.add_argument("--code") + parser.add_argument("--password") + args = parser.parse_args() + asyncio.run(main(args.phone, args.code, args.password)) +``` + +- [ ] **Step 2: Verify it imports cleanly without pytdbot installed** + +Run: `cd mcp && PYTHONPATH=src .venv/bin/python -c "import ast; ast.parse(open('scripts/tdlib_login.py').read())"` +Expected: no output, exit code 0 (this only checks the file parses; it does not execute the pytdbot-requiring `main()`, consistent with the POC's own boundary: "script-only, live login deferred to manual run with operator") + +- [ ] **Step 3: Run the full suite to confirm no regressions** + +Run: `cd mcp && TELEGRAM_API_ID=1 TELEGRAM_API_HASH=hash PYTHONPATH=src .venv/bin/python -m unittest discover -s tests -p 'test_*.py'` +Expected: `OK` + +- [ ] **Step 4: Commit** + +```bash +git add mcp/scripts/tdlib_login.py +git commit -m "feat: add production TDLib login script for the main account" +``` + +--- + +## Task 5: Document the rollout in operator workflows + +**Files:** +- Modify: `docs/operator-workflows.md` + +**Interfaces:** +- Consumes: nothing (docs only). +- Produces: nothing consumed by other tasks — this is the last task. + +- [ ] **Step 1: Add a new section** + +Append this section to `docs/operator-workflows.md` (after the existing content, as a new `##` section): + +```markdown +## TDLib Large-Media Downloads (main account only) + +`tg download` (and `download_post()` under it) can optionally route large +media downloads on the `main` account through TDLib instead of Telethon, +based on a measured advantage confirmed in a live POC (+78.7% faster average +elapsed, resumability confirmed on 3 real files — see +`mcp/docs/superpowers/specs/2026-07-01-tdlib-large-media-download-design.md`). +This does **not** change any other Telegram operation or account: reads, +search, sends, and MCP tool downloads (`download_media_batch`, +`download_dialog_media`) stay on Telethon unchanged. + +The capability stays off until explicitly enabled. Rollout: + +1. Install the optional extra: `pip install -e ".[tdlib]"` (from `mcp/`). +2. Run the one-time interactive login for `main`: + ``` + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --phone + + # then, after Telegram sends a code to another active session: + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --code + # only if 2FA is enabled: + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --password + ``` +3. Set in `main`'s env (`~/.telegram-mcp/launchd.env` or `mcp/.env`): + ``` + TELEGRAM_TDLIB_ENABLED=true + ``` + Optional tuning: `TELEGRAM_TDLIB_SESSION_DIR` (default + `~/.telegram-mcp-tdlib/main`), `TELEGRAM_TDLIB_DOWNLOAD_THRESHOLD_MB` + (default `20`). +4. Watch telemetry (`download_post_backend` events: `backend`, + `route_attempted`, `fallback_reason`) for the backend-used distribution + and fallback rate before considering wider rollout (other accounts, lower + threshold) — each of those is a separate future decision, not part of + this change. + +Every TDLib failure mode (session not authorized, network error, unsupported +content type, lock not acquired within 5s) falls back to Telethon +automatically using the connection already open for routing — there is no +new failure mode a user can hit from this change. +``` + +- [ ] **Step 2: Verify the file still renders as valid markdown** + +Run: `cd /Users/sereja/Projects/tools/telegram/.claude/worktrees/tdlib-media-download-poc && python3 -c "import pathlib; text = pathlib.Path('docs/operator-workflows.md').read_text(); assert text.count('##') >= 1; print('ok')"` +Expected: `ok` + +- [ ] **Step 3: Run the full suite one final time** + +Run: `cd mcp && TELEGRAM_API_ID=1 TELEGRAM_API_HASH=hash PYTHONPATH=src .venv/bin/python -m unittest discover -s tests -p 'test_*.py'` +Expected: `OK` + +- [ ] **Step 4: Commit** + +```bash +git add docs/operator-workflows.md +git commit -m "docs: document TDLib main-account rollout in operator workflows" +``` + +--- + +## Self-Review Notes + +- **Spec coverage:** Architecture routing/fallback → Task 3. Components (`tdlib_download.py`, `download_post.py`, `locking.py`, `tdlib_login.py`) → Tasks 1/2/3/4. Configuration (env vars) → Task 2. Dependencies (optional extra, lazy import) → Task 2 + Global Constraints. Error Handling/Fallback → Task 3 (broad `except Exception`, telemetry `fallback_reason`). Testing boundary (pure logic tested, no live-network unit tests) → Tasks 1/2/3 test design; Task 4 explicitly mirrors the POC's "script-only, manual live run" boundary. Rollout steps → Task 5. +- **Placeholder scan:** no TBD/"add error handling"/"similar to Task N" — every step has literal code or an exact command with expected output. +- **Type consistency:** `should_route_to_tdlib` (Task 2) and its call site (Task 3) use identical keyword names (`account`, `tdlib_enabled`, `content_kind`, `media_size_bytes`, `threshold_mb`). `download_via_tdlib(*, link, session_dir) -> Path` (Task 2) matches its Task 3 call site exactly. `tdlib_download.build_client`/`raise_if_error` (Task 2) match the Task 4 script's imports exactly. diff --git a/mcp/docs/superpowers/specs/2026-07-01-tdlib-large-media-download-design.md b/mcp/docs/superpowers/specs/2026-07-01-tdlib-large-media-download-design.md new file mode 100644 index 0000000..b84f29c --- /dev/null +++ b/mcp/docs/superpowers/specs/2026-07-01-tdlib-large-media-download-design.md @@ -0,0 +1,193 @@ +# TDLib for Large Media Downloads — Design + +## Context + +[control-plane/docs/adr/2026-06-21-tdlib-is-not-default-runtime.md](../../../control-plane/docs/adr/2026-06-21-tdlib-is-not-default-runtime.md) +forbids TDLib as the default Telegram runtime, but allows an isolated lab +POC gated on a measured advantage over the current `telegram-mcp` (Telethon) +path. + +That POC ([experiments/tdlib-media-poc/](../../../experiments/tdlib-media-poc/), +plan at [control-plane/docs/superpowers/plans/2026-07-01-tdlib-media-download-poc.md](../../../control-plane/docs/superpowers/plans/2026-07-01-tdlib-media-download-poc.md)) +ran live on 2026-07-01 against 3 real large files (787MB / 454MB / 48MB) on +the `main` account and passed its gate: + +| | Telethon | TDLib | | +|---|---|---|---| +| Average elapsed | 298.43s | 63.53s | TDLib +78.7% faster | +| Resumability | not tested | confirmed (`resumed=true` on all 3) | | + +Full results: `experiments/tdlib-media-poc/data/RESULTS.md` (not committed — +local live-run artifact per the POC's isolation rules). + +This design covers graduating that result into a narrowly-scoped production +capability: **TDLib as an auto-routed backend for large media downloads on +the `main` account only**, everything else stays on Telethon. + +## Non-Goals + +- TDLib does **not** become a general runtime. Reads, search, sends, and all + other Telegram operations stay on Telethon, unchanged. +- No other account (`crwddy`, `pl`, `recklessou`, `teamsyncsage`, + `vermassov`) gets TDLib in this iteration. Each would need its own live + phone/code(/2FA) login, which is out of scope here. +- No persistent TDLib daemon. Every download gets a fresh, in-process TDLib + client reconnecting to an already-authorized on-disk session, then tears + down. +- This does not touch the MCP tool surface (`download_media_batch`, + `download_dialog_media` in `mcp/src/telegram_mcp/tools/media_tools.py`). + Those are a separate, general-purpose path bound by the 120s MCP tool + timeout. TDLib routing applies only to `download_post()` / + `tg download`, which already exists specifically to bypass that cap for + large post media. + +## Architecture + +``` +tg download --account main + │ + ▼ +cmd_download() [tg_cli.py] + │ + ▼ +download_post() [download_post.py] ── Telethon connects, resolves entity+msg (unchanged) + │ + ├─ account != "main" ──► existing Telethon download (unchanged) + ├─ TELEGRAM_TDLIB_ENABLED != "true" ──► existing Telethon download (unchanged) + ├─ media size < TELEGRAM_TDLIB_DOWNLOAD_THRESHOLD_MB ──► existing Telethon download (unchanged) + ├─ unsupported content type (not video/doc/photo/audio) ──► existing Telethon download (unchanged) + │ + └─ else: try TDLib (tdlib_download.py), in-process, same asyncio loop + │ + ├─ success ──► copy result into TELEGRAM_DOWNLOAD_DIR, return + └─ failure (any) ──► log reason, fall back to the *already-open* + Telethon connection + already-fetched msg + (no second round-trip to resolve the message) +``` + +The Telethon `client`/`msg` that `download_post()` already fetches to +determine routing doubles as the ready-made fallback path — no extra +Telegram round-trip is spent either deciding to route to TDLib or falling +back from it. + +## Components + +**`mcp/src/telegram_mcp/tdlib_download.py`** (new) — ports the POC's +proven pieces (`experiments/tdlib-media-poc/benchmark/tdlib_client.py`, +`tdlib_message.py`, and the `raise_if_error`/native-object-access pattern +from `run_tdlib.py`) into the production package: + +- `assert_isolated_from_telethon(files_directory: str) -> None` — same + isolation guard as the POC, marker `.telegram-mcp` (not + `.telegram-mcp-tdlib`). +- `build_client(...) -> pytdbot.Client` — constructed with + `use_file_database=True`, `use_chat_info_database=False`, + `use_message_database=False` (minimize state beyond what file download + needs — directly addresses the ADR's original worry about TDLib being "a + second source of truth for sessions, local state, files, and updates"). +- `extract_file_id_from_message(message) -> int` — same as the POC, + ported verbatim (native pytdbot object access, not `to_dict()`). +- `raise_if_error(result)` — same as the POC. +- `download_via_tdlib(link: str, dest_dir: Path) -> Path` — the new + orchestration function `download_post()` calls: resolves the link via + TDLib, downloads (`downloadFile(..., synchronous=True)`), returns the + path. Raises on any failure — the caller decides to fall back. +- `pytdbot` import is **lazy**, inside this module's functions, guarded by + `TELEGRAM_TDLIB_ENABLED` — nothing in the rest of the package imports + `tdlib_download.py` at module load time. + +**`mcp/src/telegram_mcp/download_post.py`** (modified) — after resolving +`msg`, add the routing check described above; call +`tdlib_download.download_via_tdlib(...)` inside a `try/except`, copy the +result into `dest_dir` on success, fall through to the existing Telethon +`client.download_media(...)` call on any exception. Add +`_record_tool_telemetry`-style logging of which backend served the +request (the CLI currently has no telemetry call for downloads at all — +this is a gap this change also closes). + +**`mcp/src/telegram_mcp/locking.py`** (extended) — reuse the existing +`FileSessionLock` pattern for a new lock file under the TDLib session +directory, so two concurrent large downloads on `main` serialize at the +TDLib layer instead of both opening the same SQLite+binlog database at +once. A download that can't acquire the lock within 5 seconds falls +back to Telethon immediately (same fallback path as any other TDLib +failure) rather than blocking indefinitely. + +**`mcp/scripts/tdlib_login.py`** (new) — the POC's +`experiments/tdlib-media-poc/benchmark/login_tdlib.py`, promoted: same +`--phone`/`--code`/`--password` multi-invocation CLI, pointed at the +production TDLib session directory. One-time manual step, documented in +`docs/operator-workflows.md`. + +## Configuration + +New environment variables (`mcp/.env.example`): + +``` +# Optional: route large media downloads on the main account through TDLib. +# TELEGRAM_TDLIB_ENABLED=false +# TELEGRAM_TDLIB_SESSION_DIR=~/.telegram-mcp-tdlib/main +# TELEGRAM_TDLIB_DOWNLOAD_THRESHOLD_MB=20 +``` + +`TELEGRAM_TDLIB_ENABLED` defaults to `false` — the capability stays off +until an operator turns it on explicitly, consistent with the ADR's "not +default" stance even after graduation. + +`TELEGRAM_API_ID`/`TELEGRAM_API_HASH` are reused as-is from the existing +`mcp/.env` (same my.telegram.org app, already established as safe to share +across sessions in the POC). + +## Dependencies + +`pytdbot`/`tdjson` become an **optional** extras group in +`mcp/pyproject.toml` (e.g. `telegram-mcp[tdlib]`), not a hard dependency. +Reasoning: CI (`.github/workflows/ci.yml`) runs on `ubuntu-latest`; the +POC only verified `tdjson`'s prebuilt native binary on macOS arm64. Making +the dependency optional plus lazily importing it only inside +`tdlib_download.py` means CI, and any install that never sets +`TELEGRAM_TDLIB_ENABLED=true`, never needs `pytdbot` to be importable at +all. + +## Error Handling / Fallback + +Every failure mode on the TDLib path falls back to Telethon, using the +connection/message already fetched during routing — never a hard failure +introduced by this change: + +- TDLib session not authorized / session directory missing +- Network error, timeout, or any `pytdbot.types.Error` response +- Unsupported message content type (sticker, round video, etc. — same + `extract_file_id_from_message` ValueError as the POC) +- Lock not acquired within timeout (concurrent large download in progress) + +Each fallback is logged via `_record_tool_telemetry` with the reason, so +production data accumulates on how often TDLib actually serves the +request vs. falls back — this is the ongoing signal for whether the +capability keeps earning its place, beyond the one-time POC measurement. + +## Testing + +- Unit tests (`mcp/tests/`) for the pure logic: routing-decision function + (size/account/content-type → route to TDLib or not), `raise_if_error`, + `extract_file_id_from_message` — same TDD approach as the POC, using + real `pytdbot.types` objects in fixtures (the POC's dict-fixture bug is + the concrete lesson here: fixtures must match what pytdbot actually + returns). +- No live-network unit tests for `download_via_tdlib` itself — same + boundary as the POC: pure decision/parsing logic is tested, live + TDLib/Telegram calls are verified manually. +- Before enabling in production, re-run the live comparison once against a + fresh set of large files to confirm the measured advantage holds outside + the original 3-file sample. + +## Rollout + +1. Land the code with `TELEGRAM_TDLIB_ENABLED` unset (off). +2. Run `mcp/scripts/tdlib_login.py` once for `main` (interactive, as done + for the POC). +3. Turn on `TELEGRAM_TDLIB_ENABLED=true` for `main`'s daemon. +4. Watch telemetry for a period (backend-used distribution, fallback + rate) before considering wider rollout (other accounts, lower + threshold) — each of those is a separate future decision, not part of + this change. diff --git a/mcp/pyproject.toml b/mcp/pyproject.toml index b9f8103..e4833d1 100644 --- a/mcp/pyproject.toml +++ b/mcp/pyproject.toml @@ -13,6 +13,12 @@ dependencies = [ "structlog>=24.4", ] +[project.optional-dependencies] +tdlib = [ + "Pytdbot>=0.10.1", + "tdjson>=1.8.65", +] + [project.scripts] telegram-mcp = "telegram_mcp.__main__:main" diff --git a/mcp/scripts/tdlib_login.py b/mcp/scripts/tdlib_login.py new file mode 100644 index 0000000..99bfa87 --- /dev/null +++ b/mcp/scripts/tdlib_login.py @@ -0,0 +1,101 @@ +"""Interactive-by-invocation TDLib login for the production `main`-account +media-download backend. + +pytdbot's Client.start() only auto-drives the authorization state machine for +bot-token logins. For a real user account it does nothing at +authorizationStateWaitPhoneNumber, so this script drives the state machine +manually across multiple invocations, since a live phone/SMS code can't be fed +into a blocking input() call from outside a real TTY: + + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --phone +15551234567 + # (Telegram sends a login code to your other active sessions) + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --code 12345 + # (only if the account has 2FA enabled) + PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --password ... + +Each invocation reconnects to the same persistent session under +TELEGRAM_TDLIB_SESSION_DIR (default ~/.telegram-mcp-tdlib/main), fully +separate from the Telethon session tree telegram-mcp owns. + +Requires the optional tdlib extra: pip install -e ".[tdlib]" +""" + +import argparse +import asyncio +import os +from pathlib import Path + +from dotenv import load_dotenv + +from telegram_mcp.tdlib_download import build_client, raise_if_error + +_STATE_TO_REQUIRED_ARG = { + "authorizationStateWaitPhoneNumber": "phone", + "authorizationStateWaitCode": "code", + "authorizationStateWaitPassword": "password", +} + + +def required_arg_for_state(state: str) -> str | None: + return _STATE_TO_REQUIRED_ARG.get(state) + + +async def wait_for_stable_state(client) -> str: + for _ in range(20): + state = client.authorization_state + if state and state != "authorizationStateWaitTdlibParameters": + return state + await asyncio.sleep(0.5) + return client.authorization_state + + +async def main(phone: str | None, code: str | None, password: str | None) -> None: + load_dotenv(Path.home() / ".telegram-mcp" / "launchd.env") + api_id = int(os.environ["TELEGRAM_API_ID"]) + api_hash = os.environ["TELEGRAM_API_HASH"] + session_dir = Path( + os.environ.get("TELEGRAM_TDLIB_SESSION_DIR", "~/.telegram-mcp-tdlib/main") + ).expanduser() + + client = build_client(api_id=api_id, api_hash=api_hash, files_directory=str(session_dir)) + await client.start(wait_login=False) + + state = await wait_for_stable_state(client) + print(f"Current authorization state: {state}") + + required_arg = required_arg_for_state(state) + supplied = {"phone": phone, "code": code, "password": password}.get(required_arg) + + if required_arg and not supplied: + print( + f"Need --{required_arg}. Re-run: " + f"PYTHONPATH=src .venv/bin/python scripts/tdlib_login.py --{required_arg} " + ) + await client.stop() + return + + if state == "authorizationStateWaitPhoneNumber": + raise_if_error(await client.setAuthenticationPhoneNumber(phone_number=phone)) + print("Code requested. Check Telegram on your other devices, then run with --code.") + elif state == "authorizationStateWaitCode": + raise_if_error(await client.checkAuthenticationCode(code=code)) + print("Code accepted.") + elif state == "authorizationStateWaitPassword": + raise_if_error(await client.checkAuthenticationPassword(password=password)) + print("Password accepted.") + elif state == "authorizationStateReady": + me = raise_if_error(await client.getMe()) + print(f"Already logged in: {me}") + + await asyncio.sleep(1) + print(f"Authorization state now: {client.authorization_state}") + await client.stop() + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("--phone") + parser.add_argument("--code") + parser.add_argument("--password") + args = parser.parse_args() + asyncio.run(main(args.phone, args.code, args.password)) diff --git a/mcp/src/telegram_mcp/download_post.py b/mcp/src/telegram_mcp/download_post.py index 2b5797f..e41bfc7 100644 --- a/mcp/src/telegram_mcp/download_post.py +++ b/mcp/src/telegram_mcp/download_post.py @@ -19,6 +19,7 @@ from telethon.sessions import StringSession from .mcp_http_client import ACCOUNT_ENDPOINTS, McpCliError, load_env_file +from .telemetry import record_telemetry DEFAULT_DEST_DIR = Path.home() / "Downloads" @@ -89,6 +90,39 @@ def _ext_from_message(msg) -> str: return ".bin" +def _telethon_media_kind(msg) -> str | None: + media = getattr(msg, "media", None) + if media is None: + return None + if getattr(media, "photo", None) is not None: + return "photo" + doc = getattr(media, "document", None) + if doc is None: + return None + mime = getattr(doc, "mime_type", "") or "" + if mime.startswith("video/"): + return "video" + if mime.startswith("audio/"): + return "audio" + return "document" + + +def _telethon_media_size_bytes(msg) -> int | None: + media = getattr(msg, "media", None) + if media is None: + return None + photo = getattr(media, "photo", None) + if photo is not None: + sizes = getattr(photo, "sizes", None) or [] + if not sizes: + return None + return max(getattr(s, "size", 0) for s in sizes) + doc = getattr(media, "document", None) + if doc is not None: + return getattr(doc, "size", None) + return None + + def _make_progress(): last = {"pct": -5} @@ -163,8 +197,46 @@ async def download_post( f"(чат {parsed.chat}, аккаунт {account})" ) out = dest_dir / f"{parsed.label}_{parsed.message_id}{_ext_from_message(msg)}" - progress = None if quiet else _make_progress() - saved = await client.download_media(msg, file=str(out), progress_callback=progress) + + from . import tdlib_download # lazy: keeps pytdbot fully optional at module load time + + tdlib_enabled = os.environ.get("TELEGRAM_TDLIB_ENABLED", "false").strip().lower() == "true" + threshold_mb = float(os.environ.get("TELEGRAM_TDLIB_DOWNLOAD_THRESHOLD_MB", "20")) + route_to_tdlib = tdlib_download.should_route_to_tdlib( + account=account, + tdlib_enabled=tdlib_enabled, + content_kind=_telethon_media_kind(msg), + media_size_bytes=_telethon_media_size_bytes(msg), + threshold_mb=threshold_mb, + ) + + tdlib_backend_used = False + fallback_reason: str | None = None + saved: str | None = None + + if route_to_tdlib: + session_dir = Path( + os.environ.get("TELEGRAM_TDLIB_SESSION_DIR", "~/.telegram-mcp-tdlib/main") + ).expanduser() + try: + tdlib_path = await tdlib_download.download_via_tdlib(link=link, session_dir=session_dir) + shutil.copy2(tdlib_path, out) + saved = str(out) + tdlib_backend_used = True + except Exception as exc: # noqa: BLE001 - any TDLib failure falls back to Telethon + fallback_reason = str(exc) + + if saved is None: + progress = None if quiet else _make_progress() + saved = await client.download_media(msg, file=str(out), progress_callback=progress) + + record_telemetry( + "download_post_backend", + backend="tdlib" if tdlib_backend_used else "telethon", + account=account, + route_attempted=route_to_tdlib, + fallback_reason=fallback_reason, + ) finally: await client.disconnect() if tmp_session is not None: diff --git a/mcp/src/telegram_mcp/locking.py b/mcp/src/telegram_mcp/locking.py index 4c5a1d4..7312b32 100644 --- a/mcp/src/telegram_mcp/locking.py +++ b/mcp/src/telegram_mcp/locking.py @@ -4,6 +4,7 @@ import fcntl import os +import time from pathlib import Path from typing import TextIO @@ -50,3 +51,26 @@ def release(self) -> None: finally: self._handle.close() self._handle = None + + +def try_acquire_with_timeout( + lock: FileSessionLock, + *, + timeout_seconds: float = 5.0, + poll_interval_seconds: float = 0.2, +) -> bool: + """Retry ``lock.acquire()`` until it succeeds or ``timeout_seconds`` elapse. + + Returns True if acquired (caller owns the lock and must call release()). + Returns False if the timeout elapsed without acquiring — the caller + should fall back rather than treat this as an error. + """ + deadline = time.monotonic() + timeout_seconds + while True: + try: + lock.acquire() + return True + except SessionLockError: + if time.monotonic() >= deadline: + return False + time.sleep(poll_interval_seconds) diff --git a/mcp/src/telegram_mcp/tdlib_download.py b/mcp/src/telegram_mcp/tdlib_download.py new file mode 100644 index 0000000..2dc55c2 --- /dev/null +++ b/mcp/src/telegram_mcp/tdlib_download.py @@ -0,0 +1,132 @@ +"""TDLib backend for large media downloads on the `main` account only. + +Graduated from the isolated POC (experiments/tdlib-media-poc/) per +mcp/docs/superpowers/specs/2026-07-01-tdlib-large-media-download-design.md. +`pytdbot` is an optional dependency (`telegram-mcp[tdlib]`) — every function +here that needs it imports it lazily, so importing this module never +requires pytdbot to be installed. +""" + +from __future__ import annotations + +import os +from pathlib import Path + +from .locking import FileSessionLock, try_acquire_with_timeout + +TELETHON_SESSION_DIR_MARKER = ".telegram-mcp" + +SUPPORTED_CONTENT_KINDS = frozenset({"video", "document", "photo", "audio"}) + + +class TdlibDownloadError(RuntimeError): + """Raised for any TDLib download failure; callers should fall back to Telethon.""" + + +def assert_isolated_from_telethon(files_directory: str) -> None: + segments = Path(files_directory).parts + if TELETHON_SESSION_DIR_MARKER in segments: + raise ValueError( + f"files_directory must not overlap the Telethon session tree " + f"(found {TELETHON_SESSION_DIR_MARKER!r})" + ) + + +def build_client( + api_id: int, + api_hash: str, + files_directory: str, + database_encryption_key: str = "telegram-mcp-tdlib", +): + assert_isolated_from_telethon(files_directory) + import pytdbot + + return pytdbot.Client( + api_id=api_id, + api_hash=api_hash, + files_directory=files_directory, + database_encryption_key=database_encryption_key, + use_file_database=True, + use_chat_info_database=False, + use_message_database=False, + ) + + +def raise_if_error(result): + import pytdbot + + if isinstance(result, pytdbot.types.Error): + raise TdlibDownloadError(f"TDLib error {result['code']}: {result['message']}") + return result + + +def extract_file_id_from_message(message) -> int: + content = message["content"] + content_type = content.getType() + if content_type == "messageVideo": + return content["video"]["video"]["id"] + if content_type == "messageDocument": + return content["document"]["document"]["id"] + if content_type == "messagePhoto": + sizes = content["photo"]["sizes"] + return sizes[-1]["photo"]["id"] + if content_type == "messageAudio": + return content["audio"]["audio"]["id"] + raise ValueError(f"unsupported message content type for download: {content_type!r}") + + +def should_route_to_tdlib( + *, + account: str, + tdlib_enabled: bool, + content_kind: str | None, + media_size_bytes: int | None, + threshold_mb: float, +) -> bool: + if account != "main": + return False + if not tdlib_enabled: + return False + if content_kind not in SUPPORTED_CONTENT_KINDS: + return False + if media_size_bytes is None: + return False + return media_size_bytes >= threshold_mb * 1024 * 1024 + + +async def download_via_tdlib(*, link: str, session_dir: Path) -> Path: + """Resolve `link` via TDLib and download it fully. Returns the local file + path on success. Raises TdlibDownloadError on any failure (lock timeout, + TDLib error, incomplete download) — the caller decides to fall back.""" + lock = FileSessionLock(session_dir / "download.lock") + if not try_acquire_with_timeout(lock, timeout_seconds=5.0): + raise TdlibDownloadError("could not acquire TDLib session lock within 5s") + + try: + client = build_client( + api_id=int(os.environ["TELEGRAM_API_ID"]), + api_hash=os.environ["TELEGRAM_API_HASH"], + files_directory=str(session_dir), + ) + await client.start() + try: + link_info = raise_if_error(await client.getMessageLinkInfo(url=link)) + message = raise_if_error( + await client.getMessage( + chat_id=link_info["chat_id"], message_id=link_info["message"]["id"] + ) + ) + file_id = extract_file_id_from_message(message) + result = raise_if_error( + await client.downloadFile( + file_id=file_id, priority=1, synchronous=True, offset=0, limit=0 + ) + ) + local = result["local"] + if not local or not bool(local["is_downloading_completed"]): + raise TdlibDownloadError("TDLib download did not complete") + return Path(local["path"]) + finally: + await client.stop() + finally: + lock.release() diff --git a/mcp/tests/test_download_post.py b/mcp/tests/test_download_post.py index 3f1ca51..937a2f0 100644 --- a/mcp/tests/test_download_post.py +++ b/mcp/tests/test_download_post.py @@ -1,7 +1,13 @@ import unittest from types import SimpleNamespace -from telegram_mcp.download_post import ParsedLink, _ext_from_message, parse_post_link +from telegram_mcp.download_post import ( + ParsedLink, + _ext_from_message, + _telethon_media_kind, + _telethon_media_size_bytes, + parse_post_link, +) from telegram_mcp.mcp_http_client import McpCliError @@ -70,5 +76,47 @@ def test_falls_back_to_bin_when_no_media(self): self.assertEqual(_ext_from_message(msg), ".bin") +class TelethonMediaKindTests(unittest.TestCase): + def test_photo_is_photo_kind(self): + msg = SimpleNamespace(media=SimpleNamespace(document=None, photo=object())) + self.assertEqual(_telethon_media_kind(msg), "photo") + + def test_video_mime_is_video_kind(self): + doc = SimpleNamespace(attributes=[], mime_type="video/mp4", size=123) + msg = SimpleNamespace(media=SimpleNamespace(document=doc, photo=None)) + self.assertEqual(_telethon_media_kind(msg), "video") + + def test_audio_mime_is_audio_kind(self): + doc = SimpleNamespace(attributes=[], mime_type="audio/mpeg", size=123) + msg = SimpleNamespace(media=SimpleNamespace(document=doc, photo=None)) + self.assertEqual(_telethon_media_kind(msg), "audio") + + def test_other_document_mime_is_document_kind(self): + doc = SimpleNamespace(attributes=[], mime_type="application/pdf", size=123) + msg = SimpleNamespace(media=SimpleNamespace(document=doc, photo=None)) + self.assertEqual(_telethon_media_kind(msg), "document") + + def test_no_media_is_unsupported(self): + msg = SimpleNamespace(media=None) + self.assertIsNone(_telethon_media_kind(msg)) + + +class TelethonMediaSizeBytesTests(unittest.TestCase): + def test_document_size(self): + doc = SimpleNamespace(size=123456) + msg = SimpleNamespace(media=SimpleNamespace(document=doc, photo=None)) + self.assertEqual(_telethon_media_size_bytes(msg), 123456) + + def test_photo_largest_size(self): + sizes = [SimpleNamespace(size=100), SimpleNamespace(size=9000)] + photo = SimpleNamespace(sizes=sizes) + msg = SimpleNamespace(media=SimpleNamespace(document=None, photo=photo)) + self.assertEqual(_telethon_media_size_bytes(msg), 9000) + + def test_no_media_is_none(self): + msg = SimpleNamespace(media=None) + self.assertIsNone(_telethon_media_size_bytes(msg)) + + if __name__ == "__main__": unittest.main() diff --git a/mcp/tests/test_locking.py b/mcp/tests/test_locking.py index f8d7504..2261ee9 100644 --- a/mcp/tests/test_locking.py +++ b/mcp/tests/test_locking.py @@ -2,7 +2,7 @@ import unittest from pathlib import Path -from telegram_mcp.locking import FileSessionLock, SessionLockError +from telegram_mcp.locking import FileSessionLock, SessionLockError, try_acquire_with_timeout class LockingTests(unittest.TestCase): @@ -19,3 +19,29 @@ def test_file_session_lock_rejects_second_holder(self): second.acquire() finally: first.release() + + def test_try_acquire_with_timeout_returns_false_when_held(self): + with tempfile.TemporaryDirectory() as tmp: + lock_path = Path(tmp) / "telegram.lock" + + holder = FileSessionLock(lock_path) + holder.acquire() + try: + waiter = FileSessionLock(lock_path) + acquired = try_acquire_with_timeout( + waiter, timeout_seconds=0.3, poll_interval_seconds=0.1 + ) + self.assertFalse(acquired) + finally: + holder.release() + + def test_try_acquire_with_timeout_succeeds_once_free(self): + with tempfile.TemporaryDirectory() as tmp: + lock_path = Path(tmp) / "telegram.lock" + + lock = FileSessionLock(lock_path) + acquired = try_acquire_with_timeout(lock, timeout_seconds=0.3) + try: + self.assertTrue(acquired) + finally: + lock.release() diff --git a/mcp/tests/test_tdlib_download.py b/mcp/tests/test_tdlib_download.py new file mode 100644 index 0000000..bad309b --- /dev/null +++ b/mcp/tests/test_tdlib_download.py @@ -0,0 +1,135 @@ +import importlib.util +import unittest + +from telegram_mcp.tdlib_download import assert_isolated_from_telethon, should_route_to_tdlib + +PYTDBOT_AVAILABLE = importlib.util.find_spec("pytdbot") is not None + + +class ShouldRouteToTdlibTests(unittest.TestCase): + def test_routes_when_all_conditions_met(self): + self.assertTrue( + should_route_to_tdlib( + account="main", + tdlib_enabled=True, + content_kind="video", + media_size_bytes=30 * 1024 * 1024, + threshold_mb=20, + ) + ) + + def test_rejects_non_main_account(self): + self.assertFalse( + should_route_to_tdlib( + account="pl", + tdlib_enabled=True, + content_kind="video", + media_size_bytes=30 * 1024 * 1024, + threshold_mb=20, + ) + ) + + def test_rejects_when_disabled(self): + self.assertFalse( + should_route_to_tdlib( + account="main", + tdlib_enabled=False, + content_kind="video", + media_size_bytes=30 * 1024 * 1024, + threshold_mb=20, + ) + ) + + def test_rejects_unsupported_content_kind(self): + self.assertFalse( + should_route_to_tdlib( + account="main", + tdlib_enabled=True, + content_kind=None, + media_size_bytes=30 * 1024 * 1024, + threshold_mb=20, + ) + ) + + def test_rejects_below_threshold(self): + self.assertFalse( + should_route_to_tdlib( + account="main", + tdlib_enabled=True, + content_kind="video", + media_size_bytes=1 * 1024 * 1024, + threshold_mb=20, + ) + ) + + def test_rejects_unknown_size(self): + self.assertFalse( + should_route_to_tdlib( + account="main", + tdlib_enabled=True, + content_kind="video", + media_size_bytes=None, + threshold_mb=20, + ) + ) + + +class AssertIsolatedFromTelethonTests(unittest.TestCase): + def test_rejects_telethon_session_tree(self): + with self.assertRaises(ValueError): + assert_isolated_from_telethon("/Users/x/.telegram-mcp/main") + + def test_accepts_isolated_directory(self): + assert_isolated_from_telethon("/Users/x/.telegram-mcp-tdlib/main") + + +@unittest.skipUnless(PYTDBOT_AVAILABLE, "pytdbot not installed (optional [tdlib] extra)") +class PytdbotDependentTests(unittest.TestCase): + def test_raise_if_error_passes_through_non_error_result(self): + from telegram_mcp.tdlib_download import raise_if_error + + self.assertEqual(raise_if_error("some value"), "some value") + + def test_raise_if_error_raises_on_pytdbot_error(self): + import pytdbot + + from telegram_mcp.tdlib_download import raise_if_error + + error = pytdbot.types.Error(code=400, message="MESSAGE_ID_INVALID") + with self.assertRaisesRegex(RuntimeError, "MESSAGE_ID_INVALID"): + raise_if_error(error) + + def test_extract_file_id_from_message_video(self): + import pytdbot + + from telegram_mcp.tdlib_download import extract_file_id_from_message + + message = pytdbot.types.Message( + content=pytdbot.types.MessageVideo( + video=pytdbot.types.Video(video=pytdbot.types.File(id=555, size=52_428_800)) + ) + ) + self.assertEqual(extract_file_id_from_message(message), 555) + + def test_extract_file_id_from_message_unsupported_type(self): + import pytdbot + + from telegram_mcp.tdlib_download import extract_file_id_from_message + + message = pytdbot.types.Message(content=pytdbot.types.MessageText()) + with self.assertRaisesRegex(ValueError, "unsupported message content type"): + extract_file_id_from_message(message) + + def test_build_client_rejects_telethon_session_tree(self): + from telegram_mcp.tdlib_download import build_client + + with self.assertRaises(ValueError): + build_client( + api_id=1, + api_hash="hash", + files_directory="/Users/x/.telegram-mcp/main", + ) + + +if __name__ == "__main__": + unittest.main()