For the first time we can point an AI at a video and have it actually watch the whole thing, frame by frame, plus the audio. Our current pipeline can only read text we already have (transcripts, PDFs, existing tags), which leaves silent video, visual-heavy explainers, thin transcripts, and everything that happens on screen but is never said out loud effectively invisible. Gemini's multimodal models close that gap.

The output is a deep metadata layer on every asset, alongside what editors have entered:

Quality scores. Seven dimensions (production, currency, instructional intent, visual design, engagement, grade fit, sensitivity), each scored with a rationale and confidence, plus a composite, a deal-breaker check, and a Keep / Edit / Remove recommendation with reasoning.
Measured picture and sound quality. Sharpness, noise, contrast, blurry-frame count, and audio loudness measured off the file itself, not AI judgment.
Titles and descriptions. An accuracy score on the current title, a suggested title and description, and notes on what specifically should change.
Content analysis. Plain-language summary, estimated grade level, dated-content indicators (4:3 aspect ratio, SD resolution), specific quality concerns, standout strengths, and which rubric criteria the asset meets (learning phases, Core Four subject, engagement strategies).
Taxonomy and grade level. Suggested nodes and grade bands timestamped to the moment that supports them, with confidence, rationale, and an agree/disagree explanation against our current tags.
Standards alignment. CCSS, NGSS, and C3 matched directly, then crosswalked to 23+ states, rated and timestamped per standard, including which states are covered, which are missing, and why an existing alignment didn't match.
Instructional intent. Primary and secondary intent (Build Knowledge, Engage, Practice/Review, Apply/Extend) with rationale, mapped to Science of Teaching strategies.
Teaching support. Key concepts, classroom activities, real-world connections, and career connections (cluster, pathway, role).
Sensitive topics. Framing-aware tags following DE's rules, flagged at specific timestamps rather than as blanket labels.
Curriculum placement. Suggested publisher, program, grade, unit, and learning phase with confidence.
Captions, translations, audio description. English captions plus Spanish and French, audio description in all three, and a diff against the existing human transcript.
Chapter-level detail. The same metadata regenerated per chapter and time-sliced, so long assets are usable in pieces.

Video is where this starts, but the pipeline is built to extend to other asset types. Which types we take on next is still an open decision.

DE API Layer

Key dates

Decision on API Framework

First API available with new api layer