Asset Intelligence Pipeline

For the first time we can point an AI at a video and have it actually watch the whole thing, frame by frame, plus the audio. Our current pipeline can only read text we already have (transcripts, PDFs, existing tags), which leaves silent video, visual-heavy explainers, thin transcripts, and everything that happens on screen but is never said out loud effectively invisible. Gemini's multimodal models close that gap.

The output is a deep metadata layer on every asset, alongside what editors have entered:

• Quality scores. Seven dimensions (production, currency, instructional intent, visual design, engagement, grade fit, sensitivity), each scored with a rationale and confidence, plus a composite, a deal-breaker check, and a Keep / Edit / Remove recommendation with reasoning.

• Measured picture and sound quality. Sharpness, noise, contrast, blurry-frame count, and audio loudness measured off the file itself, not AI judgment.

• Titles and descriptions. An accuracy score on the current title, a suggested title and description, and notes on what specifically should change.

• Content analysis. Plain-language summary, estimated grade level, dated-content indicators (4:3 aspect ratio, SD resolution), specific quality concerns, standout strengths, and which rubric criteria the asset meets (learning phases, Core Four subject, engagement strategies).

• Taxonomy and grade level. Suggested nodes and grade bands timestamped to the moment that supports them, with confidence, rationale, and an agree/disagree explanation against our current tags.

• Standards alignment. CCSS, NGSS, and C3 matched directly, then crosswalked to 23+ states, rated and timestamped per standard, including which states are covered, which are missing, and why an existing alignment didn't match.

• Instructional intent. Primary and secondary intent (Build Knowledge, Engage, Practice/Review, Apply/Extend) with rationale, mapped to Science of Teaching strategies.

• Teaching support. Key concepts, classroom activities, real-world connections, and career connections (cluster, pathway, role).

• Sensitive topics. Framing-aware tags following DE's rules, flagged at specific timestamps rather than as blanket labels.

• Curriculum placement. Suggested publisher, program, grade, unit, and learning phase with confidence.

• Captions, translations, audio description. English captions plus Spanish and French, audio description in all three, and a diff against the existing human transcript.

• Chapter-level detail. The same metadata regenerated per chapter and time-sliced, so long assets are usable in pieces.

Video is where this starts, but the pipeline is built to extend to other asset types. Which types we take on next is still an open decision.

Richer asset metadata is upstream of almost everything we want to build. This work drives product development, standards alignment, search, and MCP/chat all at once.

• Search and AI chat get more to work with. Retrieval is only as good as the metadata behind it, and quality scores let us surface strong assets and suppress weak ones.

• Standards coverage expands. State-by-state alignment at library scale is work we cannot staff by hand.

• Product teams get signals that don't exist today. Differentiation, playlists, recommendations, and personalization all need structured asset-level data.

• Content and curriculum teams get a map of the library. Quality scores show what to refresh, retire, or promote. Nobody can watch 65,000 assets.

• Every video asset has a report, refreshed when the asset changes.

• Other systems (search, chat, MCP) consume the metadata without a bespoke integration.

• Editors accept or reject suggestions field by field, in the tools they already use.

• AI-written data is always distinguishable from human-entered data and fully reversible.

• Quality scores are trusted enough to drive a real content decision.

• Measurable lift in standards coverage and in assets with complete metadata, at a cost that works at full library scale.