VidXP

Search video by what was said, what appeared on screen, and recurring faces.

VidXP is a local-first video indexing and search engine distributed as a Python package.

---

You can use it:

• From the command line
• Through its browser interface
• As a desktop app
• As an API
• As an MCP, with your agents (coming soon ⚡️)
• As an indexing and retrieval layer inside another application

Dialogue search · Scene search · Actor grouping · CLI · Browser UI · Python API

Why VidXP

Finding one moment in a video should not require scrubbing through the entire timeline. VidXP builds a searchable index from three kinds of evidence:

• Dialogue: semantic search over timestamped transcripts.
• Scenes: text-to-frame search.
• Actors: groups similar detected faces and exports a highlighted video for a selected cluster.

After the required model weights are available, video processing and search run completely locally, for your privacy and security.

Some ideas on how to use VidXP:

• Use it as way to find your favorite relatives in a huge folder of wedding videos (been there, done that)
• Use it in your application, allow users to search videos (an idea: use it alongside a video-editing application)
• Use it as an "understanding" layer so your LLM / agent can understand videos

Current capabilities

Capability	Available now	Result
Dialogue search	Transcription, word alignment, semantic phrase indexing	Matching video time
Scene search	Text search over sampled video frames	Matching frame and time
Actor grouping	Within-video face detection and clustering	Clustered detections and highlighted output video
Interfaces	Typer CLI, Streamlit browser interface, Python API	Interactive or programmatic use
Index management	Saved progress, ready/failed state, cancellation, isolated programmatic runs	Traceable and reusable indexes

Quick start

VidXP supports Python 3.10 through 3.13 and requires FFmpeg. See the installation guide for the dlib compiler requirements, source installation, model preparation, and troubleshooting.

Install the command line and browser interface with pipx. The command is available on your PATH while VidXP and its dependencies remain isolated:

pipx install "vidxp[all,frontend]"

Install only the capabilities you need with a smaller selection such as pipx install "vidxp[scene,frontend]" or pipx install "vidxp[dialogue,scene]". To import VidXP from another Python project, install it into that project's environment instead; the installation guide covers that path.

Confirm the installed package and its runtime dependencies:

vidxp --version
vidxp doctor

The first use of each capability downloads its model weights. Download the fixed dialogue, transcription, and scene models in advance with:

vidxp prepare

Index and search

Build an index containing dialogue, scene, and actor information:

vidxp index create samplevideo.mp4

Search the completed index:

vidxp search dialogue "the bread just came out of the oven"
vidxp search scene "a yellow taxi on a city street" --top-k 5
vidxp actors list
vidxp actors render 1 samplevideo.mp4

Index only selected capabilities or sample fewer visual frames:

vidxp index create samplevideo.mp4 --modality scene --frame-stride 5

Repeat --modality to combine dialogue, scene, and actor. Run vidxp --help or any command followed by --help for the complete command reference.

Use named repositories to keep index locations and devices centrally configured:

vidxp repositories add team --index-dir ./indexes/team --device cuda --use
vidxp repositories list

Browser interface

Install the frontend extra and start:

vidxp ui

The command uses the active named repository, starts a local Streamlit server, and remains active until stopped. The interface can upload a video, start or cancel indexing, restore saved progress after a page reload, and search the capabilities available in the completed index.

Container

Stable releases are available from GitHub Container Registry. Start the local interface with persistent index and model storage by running:

docker compose up

See the installation guide for model preparation, configuration, and direct docker run usage.

Use VidXP as a Python package

The programmatic API supports isolated multi-video runs, supplied timestamped transcripts, resumable per-video checkpoints, and metadata-rich top-k results.

from vidxp.core import IndexConfig, VideoSource
from vidxp.core.runner import run_index
from vidxp.capabilities.scene.operations import search_scene

config = IndexConfig(
    dataset="my-library",
    split="local",
    run_id="demo",
    enabled_modalities=("scene",),
    frame_stride=5,
)

run_index(
    [
        VideoSource(video_id="video-1", path="videos/first.mp4"),
        VideoSource(video_id="video-2", path="videos/second.mp4"),
    ],
    config,
)

results = search_scene("a person enters a taxi", config=config, top_k=5)
for hit in results.hits:
    print(hit.video_id, hit.start, hit.end, hit.score)

The Python indexing and retrieval contract documents configuration, stored metadata, result fields, and run layout.

Recommended specs

Coming soon

---

Roadmap

VidXP is an evolving beta. We'd love to hear your feedback and where you'd like to see the project go.

Area	Current foundation	Direction
Search results	Top result in the CLI; structured top-k Python results	Rich ranked results, metadata, previews, and filtering across interfaces
Temporal search	Frame and transcript-phrase timestamps	Better time ranges, scene boundaries, aggregation, and ranking
Video collections	One local CLI/UI index; isolated multi-video Python runs	User-facing persistent multi-video libraries and index management
Actor workflows	Face clustering and highlighted video export	Cluster browsing, labeling, actor search, and stronger tracking
Speaker context	Timestamped dialogue search	Active-speaker detection and links between speech and visible people
Product experience	CLI and browser indexing/search	Clearer progress, result navigation, recovery, and long-running job controls
Evaluation	DiDeMo and HiREST baselines	Combined and component benchmarks, beginning with a LongVALE pilot

Models and local data

Capability	Model
Dialogue embeddings	sentence-transformers/all-MiniLM-L6-v2
Transcription	WhisperX large-v2
Scene search	CLIP ViT-B/32
Word alignment	WhisperX model selected for the detected language

VidXP maintains the standard local CLI/UI index in chroma_data/. Starting a new local indexing run replaces the previous or incomplete local index. Model caches normally live outside this directory and outside the virtual environment.

Documentation and project links

• Installation and troubleshooting
• Benchmarking status and results
• Adding a capability
• Changelog
• Issue tracker
• MIT license

Contributing

See CONTRIBUTING.md for guidelines, maintainers, and how to submit PRs. AI/vibe-coded PRs welcome!

Credits

Built by Grayhat Developers PVT Ltd. 2026. Maintained by the community.

Email: info@grayhat.studio