ClipTraining

Semantic video search infrastructure powered by AI

Overview

ClipTraining’s platform teaches corporate employees software skills, Excel, Teams, Outlook, through a library of thousands of short video clips. The real problem was never the videos, it was that the right 40-second answer was buried inside a 20-minute tutorial, with no way to search for it. I approached that as a product problem before it was an engineering one, and because I have the technical fluency to take it all the way to production myself, I designed and built the full pipeline rather than just specifying it for an engineering team.

ClipTraining is the result: AI-powered search infrastructure that transforms the video library into a queryable knowledge base, so a user can search in natural language and land on the exact moment (minute + second) where the answer actually appears.

The Problem

An employee with a specific work question, “how do I share my screen in Teams”, shouldn’t have to scrub through a 20-minute video to find a 15-second answer.

  • No semantic understanding of what’s actually said in a video
  • Metadata is incomplete or inconsistent, so keyword search fails
  • Users must manually scrub through videos to find the moment they need
  • Scaling search across thousands of clips has to stay affordable, or it never ships

The Solution

I designed and implemented an end-to-end pipeline that:

  • Transcribes videos with timestamps
  • Converts content into vector embeddings
  • Enables semantic search via natural language
  • Returns precise video segments

System Overview

Key Contributions

Transcription & Time Alignment

  • Evaluated multiple ASR providers (Whisper, AssemblyAI, GCP, AWS)
  • Generated accurate timestamped transcripts
  • Improved alignment for precise segment retrieval

Metadata Enrichment

  • Identified gaps in existing metadata
  • Generated contextual metadata using LLMs
  • Improved search relevance and discoverability

Chunking Strategy (A/B Testing)

  • Tested multiple chunk sizes and overlaps
  • Evaluated impact on retrieval accuracy and latency
  • Selected 12s window / 3s overlap, tuned against how someone actually phrases a spoken work answer, not just abstract benchmark accuracy

Embeddings & Retrieval

  • Benchmarked OpenAI vs Gemini embeddings
  • Optimized for semantic accuracy vs performance tradeoffs
  • Enabled similarity-based search beyond keywords

Vector Database Architecture

  • Evaluated Pinecone, FAISS, ChromaDB, and pgvector
  • Implemented scalable storage and retrieval layer
  • Designed for fast similarity search across large datasets

Incremental Indexing

  • Avoided full reprocessing of ~20k video clips
  • Implemented hashing strategy on VTT files
  • Re-index only modified content

Search API (Production Deployment)

  • Built FastAPI service deployed on Azure Functions
  • Endpoints:
    • /search
    • /index
    • /health
  • Integrated with PostgreSQL (pgvector) and Azure infrastructure

Cost Optimization

  • Modeled API costs for transcription, embeddings, and metadata
  • Identified low per-video cost (~$0.04 at scale)
  • Balanced accuracy vs compute efficiency

Example Output

Query: “How do I share my screen in Teams?”

Result:

  • Video: Teams Meetings Tutorial
  • Timestamp: 02:34
  • Segment: “Click the share button in the meeting controls…”

Outcome

  • An employee’s 40-second answer is now a search away, not a 20-minute video away
  • Production-ready semantic video search system, scaling to ~20k video clips
  • Enables conversational and API-based search experiences
  • Ready for frontend integration and enterprise use

This is the most technically demanding project I’ve built end to end, and I’d make the same call again: a product designer who can take a real user problem all the way through infrastructure decisions to a shipped system finds different tradeoffs than a pure engineering brief would.

Key Learnings

  • Chunking strategy has a major impact on retrieval quality
  • Metadata + semantic search (hybrid) significantly improves precision
  • Cost modeling is critical for scaling AI systems
  • Incremental pipelines are essential for real-world deployments