EXP-002PlannedCreative tool

AI Video Editor

A real browser video editor with a multitrack timeline, AI captions, plain-English edits and a genuine MP4 export.

Time budget
8h per agent
Spec
v1.1 committed Nov 7, 2026
Agents
3 runs not started
Runs recorded
—
Evaluation
Pending

01 The hypothesis

The hypothesis

Coding agents that can ship a CRUD-style SaaS app will struggle far more with media engineering, where timebase math, preview/export parity and real encoding can't be faked.

Questions this experiment asks

  1. Can an agent produce an export that really plays and matches the timeline?
  2. Will the agents find a workable encoding architecture on a Workers-based platform?
  3. Do the AI features return real, validated edit plans, or canned output?

The challenge

Video editing is where “it looks done” and “it works” drift furthest apart. The timeline has to keep consistent time without floating-point drift. The preview and the export have to come from one canonical project model. The export has to be a real H.264/AAC file whose audio lines up, and Cloudflare Workers can’t host native FFmpeg. Each agent has to choose a viable encoding architecture and prove it works.

The evaluator runs a fixed 10-step editing task with a shared media pack, then inspects the exported file.

02 The rules

The challenge, as specified

A deployed app in which a tester can import supplied media, edit a short video, generate and edit captions, save and reopen the project, and export an MP4 whose video and audio match the timeline.

Required scope Spec: A requirement from the versioned specification.

  • Project dashboard, email/password accounts, guest editing and an admin role
  • Import MP4, JPEG/PNG and MP3/WAV with metadata, thumbnails and waveforms
  • Multitrack timeline (at least 2 video and 2 audio tracks) with trim, split, snapping and undo/redo
  • Composed canvas preview that matches the export, with 16:9, 9:16 and 1:1 presets
  • Text overlays, transforms, fades, transitions, color adjustments, speed and audio mixing
  • OpenAI transcription into editable, burned-in captions
  • Natural-language edit assistant producing validated, confirmable, undoable edit plans
  • Real H.264/AAC MP4 export at 720p and 1080p, with progress and export history

Constraints Spec: A requirement from the versioned specification.

  • Eight hours of wall-clock time per agent, including planning, installs, testing, deploying and reporting
  • No human-written code, fixes, design decisions or architecture guidance during a run
  • Human help limited to credentials, permissions and approval prompts, and every intervention is recorded
  • Do not embed, fork or white-label an existing video editor
  • No silent fallbacks: never rename WebM to .mp4, and no canned AI output or fake progress
  • Never invent test results; mark untested features as unverified

Deliverables Spec: A requirement from the versioned specification.

  • Source repository with reproducible setup
  • Deployed application URL and health endpoint
  • README.md, BUILD_REPORT.md, FEATURE_MATRIX.md and TEST_RESULTS.md
  • Example project or reproducible sample data
  • A beautiful UI without working editing and export does not meet the mission.spec v1.1
  • An export job reporting "completed" is not sufficient proof.spec v1.1

03 The contenders

Who will take this on

The spec names these agents. Runs haven't started, so there are no repositories, timelines or results yet. Each run will get its own page here once it's recorded.

04 Head-to-head evaluation

Evaluation pending

Scoring rubric · 100 points

Spec: A requirement from the versioned specification.
  1. End-to-end editing and export30
  2. Timeline and preview correctness20
  3. Product usability and design15
  4. AI capabilities10
  5. Persistence and backend reliability10
  6. Engineering quality10
  7. Agent autonomy and reporting5
  • Hard-failure flags: no real playable export, preview does not match the timeline, cannot save and reopen, cross-user access, fake AI, or broken deployment.
  • Exports are checked by playback and metadata inspection. A "completed" status is not proof.

05 Evidence

Inspect the artifacts yourself

06 Observations & verdict

What we can say so far

No observations yet. They're recorded during the runs.

No verdict yet.

A winner is only named once every build has been independently tested against the shared criteria, with the score components shown.

Episode

Not published yet

The video for this experiment will be embedded here once it is published.

Get notified on YouTube (opens in a new tab)

Explore more

Other experiments

Discover screen of the Codex-built travel planner, "Elsewhere", from a screenshot the agent committedEXP-001
Runs recordedWeb app

AI Travel Planner

Three coding agents, one 1,245-line product spec, eight hours each. Build a real AI travel SaaS and deploy it.

  • CCClaude Code
  • CXOpenAI Codex
  • CUCursor CLI
Time budget
8h / agent
Spec version
Spec v1.7
Runs
Runs not started
EXP-003
PlannedGame

3D Basketball Game

Five-on-five basketball in the browser, with real ball physics, keyboard control and an AI opponent.

  • CCClaude Code
  • CXOpenAI Codex
  • CUCursor CLI
Time budget
8h / agent
Spec version
Spec v1.0
Runs
Runs not started
EXP-004
PlannedGame

Project Bulletstorm

A stationary 3D lane shooter where you steer an auto-firing squad, recruit soldiers and survive ten levels.

  • CCClaude Code
  • CXOpenAI Codex
  • CUCursor CLI
Time budget
8h / agent
Spec version
Spec v1.0
Runs
Runs not started