Vector-Bench · a thetalab benchmark

theta labs / task

sv_015very_hard

scenic_multi

instructionnatural language

Make the train door pale yellow again, remove the extra clock hand, and slide the blue poster back into line. The green signal should be green, the platform crack should follow its original shallow zigzag, and the passenger's bag should sit beside them again.

initial · brokeninput

target · expected fixcanonical

published model outputs

What each model produced.

34 endpoints

model result explorerSeed 2.0 Mini

Seed 2.0 Mini

bytedance-seed/seed-2.0-mini

TracePARTIAL

requested repairs

4/7

semantic preservation

clean

SVG validity

valid

source fidelity

unchanged

target match

different

Full pass requires the first three gates. Source fidelity and canonical target match are diagnostics only.

outcome

partial

progress

73.8%

UCR

0.0%

preserve

100.0%

elapsed

2593.6s

cost

$0.0054

requested repairs4/7

passenger-bag.xpassed

expected 104 / produced 102

numeric-delta: 2 <= 5 user units / 80.0% repaired

passenger-bag.ymissed

expected 108 / produced 105

numeric-delta: 3 <= 1.5 user units / 0.0% repaired

platform-crack.dmissed

expected M18 108 C32 104 42 111 56 108 S82 106 96 110 / produced M18 109 L24 111 L32 109 L40 111 L48 109 L56 111 L64 109 L72 111 L80 109 L88 111 L96 110

path-shape: 4.204 <= 4.2 sampled geometry / 55.7% repaired

signal-green.fillmissed

expected #22c55e / produced #10b981

color-lab: 24.778 <= 18 Delta E76 / 80.5% repaired

poster-blue.xpassed

expected 178 / produced 178

numeric-delta: 0 <= 3.5 user units / 100.0% repaired

clock-minute-dup.existspassed

expected false / produced false

required presence matches

train-door.fillpassed

expected #fef3c7 / produced #fef3c7

color-lab: 0 <= 10.442 Delta E76 / 100.0% repaired

Why it did not pass

requested repairs incomplete

Unexpected changes

None

Semantic preservation failures

None

Source fidelity diagnostics

None

prompt 3,575 tokoutput 12,619 tokreasoning 9,199 tokceiling 4,231 tokfinish stopcheap control

resolved bytedance-seed/seed-2.0-mini

response gen-1784547417-gV8vDYNapAItYFU04MSy

expected diff7 edits

Part	Attr	Before	After
passenger-bag	x	114	104
passenger-bag	y	105	108
platform-crack	d	M18 109 C30 116 44 101 58 111 S82 114 96 105	M18 108 C32 104 42 111 56 108 S82 106 96 110
signal-green	fill	#ef4444	#22c55e
poster-blue	x	185	178
clock-minute-dup	exists	true	false
train-door	fill	#ffffff	#fef3c7

partsscope

Target (6)

passenger-bag

platform-crack

signal-green

poster-blue

clock-minute-dup

train-door

Should preserve (63)

sky-band

overhead-wire

station-wall

platform

platform-edge

rail-back

rail-front

track-tie-1

track-tie-2

track-tie-3

track-tie-4

track-tie-5

train-body

train-roof

window-left

window-center

window-right

door-handle

wheel-left

wheel-right

signal-post

signal-box

signal-red

signal-amber

clock-face

clock-hour

clock-minute

poster-pink

bench-seat

bench-leg-left

bench-leg-right

passenger-head

passenger-body

skin-15-snowbank-wide

skin-15-flake-wide-a

skin-15-flake-wide-b

skin-15-flake-wide-c

identity-15-snowbank

identity-15-flake-a

identity-15-flake-b

identity-15-flake-c

variant-15-dot

variant-15-rule

variant-15-chip-01

variant-15-chip-02

variant-15-chip-03

variant-15-chip-04

variant-15-chip-05

variant-15-chip-06

variant-15-chip-07

variant-15-chip-08

variant-15-chip-09

variant-15-chip-10

variant-15-chip-11

variant-15-chip-12

variant-15-pin-01

variant-15-pin-02

variant-15-pin-03

variant-15-pin-04

variant-15-pin-05

variant-15-pin-06

variant-15-pin-07

variant-15-pin-08

initial svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 220 140" width="220" height="140">
<rect id="__bg" x="0" y="0" width="220" height="140" fill="#fff8fa" />
  <rect id="sky-band" fill="#ffe4e6" x="0" y="0" width="220" height="64" />
  <path id="overhead-wire" fill="none" stroke="#334155" stroke-width="1" d="M8 18 C42 10 74 10 108 18 S176 26 212 18" />
  <rect id="station-wall" fill="#e0f2fe" x="0" y="64" width="220" height="30" />
  <rect id="platform" fill="#f8fafc" x="0" y="94" width="220" height="46" />
  <rect id="platform-edge" fill="#bae6fd" x="0" y="94" width="220" height="5" />
  <path id="platform-crack" fill="none" stroke="#94a3b8" stroke-width="1" d="M18 109 C30 116 44 101 58 111 S82 114 96 105" />
  <line id="rail-back" stroke="#64748b" stroke-width="3" x1="0" y1="118" x2="220" y2="118" />
  <line id="rail-front" stroke="#475569" stroke-width="3" x1="0" y1="130" x2="220" y2="130" />
  <rect id="track-tie-1" fill="#7c2d12" x="18" y="116" width="7" height="18" />
  <rect id="track-tie-2" fill="#7c2d12" x="56" y="116" width="7" height="18" />
  <rect id="track-tie-3" fill="#7c2d12" x="94" y="116" width="7" height="18" />
  <rect id="track-tie-4" fill="#7c2d12" x="132" y="116" width="7" height="18" />
  <rect id="track-tie-5" fill="#7c2d12" x="170" y="116" width="7" height="18" />
  <rect id="train-body" fill="#ef4444" stroke="#222" stroke-width="1.5" x="26" y="38" width="126" height="46" rx="6" ry="6" />
  <rect id="train-roof" fill="#991b1b" stroke="#222" stroke-width="1" x="34" y="30" width="110" height="12" rx="4" ry="4" />
  <rect id="window-left" fill="#bfdbfe" stroke="#222" stroke-width="1" x="38" y="46" width="24" height="16" />
  <rect id="window-center" fill="#bfdbfe" stroke="#222" stroke-width="1" x="68" y="46" width="24" height="16" />
  <rect id="window-right" fill="#bfdbfe" stroke="#222" stroke-width="1" x="98" y="46" width="24" height="16" />
  <rect id="train-door" fill="#ffffff" stroke="#222" stroke-width="1" x="128" y="44" width="16" height="34" />
  <circle id="door-handle" fill="#222" cx="140" cy="61" r="1.6" />
  <circle id="wheel-left" fill="#111827" cx="48" cy="86" r="6" />
  <circle id="wheel-right" fill="#111827" cx="126" cy="86" r="6" />
  <line id="signal-post" stroke="#334155" stroke-width="4" x1="170" y1="32" x2="170" y2="94" />
  <rect id="signal-box" fill="#111827" x="162" y="22" width="16" height="34" rx="3" ry="3" />
  <circle id="signal-red" fill="#ef4444" cx="170" cy="30" r="3" />
  <circle id="signal-amber" fill="#f59e0b" cx="170" cy="39" r="3" />
  <circle id="signal-green" fill="#ef4444" cx="170" cy="48" r="3" />
  <circle id="clock-face" fill="#ffffff" stroke="#222" stroke-width="1.5" cx="196" cy="31" r="12" />
  <line id="clock-hour" stroke="#222" stroke-width="1.5" x1="196" y1="31" x2="196" y2="23" />
  <line id="clock-minute" stroke="#222" stroke-width="1" x1="196" y1="31" x2="203" y2="31" />
  <rect id="poster-blue" fill="#3b82f6" stroke="#222" stroke-width="0.8" x="185" y="62" width="14" height="20" />
  <rect id="poster-pink" fill="#f43f5e" stroke="#222" stroke-width="0.8" x="198" y="64" width="12" height="18" />
  <rect id="bench-seat" fill="#92400e" x="38" y="100" width="42" height="5" />
  <line id="bench-leg-left" stroke="#451a03" stroke-width="2" x1="46" y1="105" x2="43" y2="114" />
  <line id="bench-leg-right" stroke="#451a03" stroke-width="2" x1="72" y1="105" x2="75" y2="114" />
  <circle id="passenger-head" fill="#f2c078" cx="98" cy="96" r="4" />
  <rect id="passenger-body" fill="#2563eb" x="94" y="100" width="8" height="16" />
  <rect id="passenger-bag" fill="#a16207" x="114" y="105" width="8" height="7" />
  <path id="skin-15-snowbank-wide" fill="#e0f2fe" opacity="0.88" d="M0 118 C38 104 72 130 110 114 S172 102 220 118 L220 150 L0 150 Z" />
  <circle id="skin-15-flake-wide-a" fill="#f8fafc" opacity="0.95" cx="38" cy="34" r="6" />
  <circle id="skin-15-flake-wide-b" fill="#f8fafc" opacity="0.95" cx="132" cy="24" r="5" />
  <circle id="skin-15-flake-wide-c" fill="#f8fafc" opacity="0.95" cx="196" cy="42" r="6" />
  <path id="identity-15-snowbank" fill="#e0f2fe" opacity="0.9" d="M0 124 C34 114 62 130 94 120 S158 112 220 124 L220 150 L0 150 Z" />
  <circle id="identity-15-flake-a" fill="#f8fafc" cx="44" cy="31" r="3" />
  <circle id="identity-15-flake-b" fill="#f8fafc" cx="128" cy="22" r="2.5" />
  <circle id="identity-15-flake-c" fill="#f8fafc" cx="186" cy="38" r="3" />
  <circle id="variant-15-dot" fill="#f43f5e" opacity="0.72" cx="25" cy="35" r="2.2" />
  <line id="variant-15-rule" stroke="#f43f5e" stroke-width="1.2" opacity="0.58" x1="31" y1="38" x2="47" y2="36" />
  <rect id="variant-15-chip-01" fill="#84cc16" opacity="0.68" x="11" y="4" width="7" height="3" rx="1" />
  <rect id="variant-15-chip-02" fill="#22c55e" opacity="0.68" x="42" y="5" width="8" height="4" rx="1" />
  <rect id="variant-15-chip-03" fill="#14b8a6" opacity="0.68" x="79" y="6" width="9" height="3" rx="1" />
  <rect id="variant-15-chip-04" fill="#06b6d4" opacity="0.68" x="110" y="4" width="7" height="4" rx="1" />
  <rect id="variant-15-chip-05" fill="#3b82f6" opacity="0.68" x="147" y="5" width="8" height="3" rx="1" />
  <rect id="variant-15-chip-06" fill="#6366f1" opacity="0.68" x="178" y="6" width="9" height="4" rx="1" />
  <rect id="variant-15-chip-07" fill="#8b5cf6" opacity="0.68" x="11" y="141" width="7" height="3" rx="1" />
  <rect id="variant-15-chip-08" fill="#d946ef" opacity="0.68" x="42" y="140" width="8" height="4" rx="1" />
  <rect id="variant-15-chip-09" fill="#ec4899" opacity="0.68" x="79" y="139" width="9" height="3" rx="1" />
  <rect id="variant-15-chip-10" fill="#ef4444" opacity="0.68" x="110" y="141" width="7" height="4" rx="1" />
  <rect id="variant-15-chip-11" fill="#f97316" opacity="0.68" x="147" y="140" width="8" height="3" rx="1" />
  <rect id="variant-15-chip-12" fill="#f59e0b" opacity="0.68" x="178" y="139" width="9" height="4" rx="1" />
  <circle id="variant-15-pin-01" fill="#06b6d4" opacity="0.66" cx="17" cy="22" r="2.55" />
  <circle id="variant-15-pin-02" fill="#3b82f6" opacity="0.66" cx="202" cy="36" r="1.8" />
  <circle id="variant-15-pin-03" fill="#6366f1" opacity="0.66" cx="27" cy="50" r="2.05" />
  <circle id="variant-15-pin-04" fill="#8b5cf6" opacity="0.66" cx="212" cy="64" r="2.3" />
  <circle id="variant-15-pin-05" fill="#d946ef" opacity="0.66" cx="13" cy="78" r="2.55" />
  <circle id="variant-15-pin-06" fill="#ec4899" opacity="0.66" cx="198" cy="92" r="1.8" />
  <circle id="variant-15-pin-07" fill="#ef4444" opacity="0.66" cx="23" cy="106" r="2.05" />
  <circle id="variant-15-pin-08" fill="#f97316" opacity="0.66" cx="208" cy="120" r="2.3" />
  <line id="clock-minute-dup" stroke="#222" stroke-width="1" x1="191" y1="38" x2="198" y2="38" />
</svg>

target svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 220 140" width="220" height="140">
<rect id="__bg" x="0" y="0" width="220" height="140" fill="#fff8fa" />
  <rect id="sky-band" fill="#ffe4e6" x="0" y="0" width="220" height="64" />
  <path id="overhead-wire" fill="none" stroke="#334155" stroke-width="1" d="M8 18 C42 10 74 10 108 18 S176 26 212 18" />
  <rect id="station-wall" fill="#e0f2fe" x="0" y="64" width="220" height="30" />
  <rect id="platform" fill="#f8fafc" x="0" y="94" width="220" height="46" />
  <rect id="platform-edge" fill="#bae6fd" x="0" y="94" width="220" height="5" />
  <path id="platform-crack" fill="none" stroke="#94a3b8" stroke-width="1" d="M18 108 C32 104 42 111 56 108 S82 106 96 110" />
  <line id="rail-back" stroke="#64748b" stroke-width="3" x1="0" y1="118" x2="220" y2="118" />
  <line id="rail-front" stroke="#475569" stroke-width="3" x1="0" y1="130" x2="220" y2="130" />
  <rect id="track-tie-1" fill="#7c2d12" x="18" y="116" width="7" height="18" />
  <rect id="track-tie-2" fill="#7c2d12" x="56" y="116" width="7" height="18" />
  <rect id="track-tie-3" fill="#7c2d12" x="94" y="116" width="7" height="18" />
  <rect id="track-tie-4" fill="#7c2d12" x="132" y="116" width="7" height="18" />
  <rect id="track-tie-5" fill="#7c2d12" x="170" y="116" width="7" height="18" />
  <rect id="train-body" fill="#ef4444" stroke="#222" stroke-width="1.5" x="26" y="38" width="126" height="46" rx="6" ry="6" />
  <rect id="train-roof" fill="#991b1b" stroke="#222" stroke-width="1" x="34" y="30" width="110" height="12" rx="4" ry="4" />
  <rect id="window-left" fill="#bfdbfe" stroke="#222" stroke-width="1" x="38" y="46" width="24" height="16" />
  <rect id="window-center" fill="#bfdbfe" stroke="#222" stroke-width="1" x="68" y="46" width="24" height="16" />
  <rect id="window-right" fill="#bfdbfe" stroke="#222" stroke-width="1" x="98" y="46" width="24" height="16" />
  <rect id="train-door" fill="#fef3c7" stroke="#222" stroke-width="1" x="128" y="44" width="16" height="34" />
  <circle id="door-handle" fill="#222" cx="140" cy="61" r="1.6" />
  <circle id="wheel-left" fill="#111827" cx="48" cy="86" r="6" />
  <circle id="wheel-right" fill="#111827" cx="126" cy="86" r="6" />
  <line id="signal-post" stroke="#334155" stroke-width="4" x1="170" y1="32" x2="170" y2="94" />
  <rect id="signal-box" fill="#111827" x="162" y="22" width="16" height="34" rx="3" ry="3" />
  <circle id="signal-red" fill="#ef4444" cx="170" cy="30" r="3" />
  <circle id="signal-amber" fill="#f59e0b" cx="170" cy="39" r="3" />
  <circle id="signal-green" fill="#22c55e" cx="170" cy="48" r="3" />
  <circle id="clock-face" fill="#ffffff" stroke="#222" stroke-width="1.5" cx="196" cy="31" r="12" />
  <line id="clock-hour" stroke="#222" stroke-width="1.5" x1="196" y1="31" x2="196" y2="23" />
  <line id="clock-minute" stroke="#222" stroke-width="1" x1="196" y1="31" x2="203" y2="31" />
  <rect id="poster-blue" fill="#3b82f6" stroke="#222" stroke-width="0.8" x="178" y="62" width="14" height="20" />
  <rect id="poster-pink" fill="#f43f5e" stroke="#222" stroke-width="0.8" x="198" y="64" width="12" height="18" />
  <rect id="bench-seat" fill="#92400e" x="38" y="100" width="42" height="5" />
  <line id="bench-leg-left" stroke="#451a03" stroke-width="2" x1="46" y1="105" x2="43" y2="114" />
  <line id="bench-leg-right" stroke="#451a03" stroke-width="2" x1="72" y1="105" x2="75" y2="114" />
  <circle id="passenger-head" fill="#f2c078" cx="98" cy="96" r="4" />
  <rect id="passenger-body" fill="#2563eb" x="94" y="100" width="8" height="16" />
  <rect id="passenger-bag" fill="#a16207" x="104" y="108" width="8" height="7" />
  <path id="skin-15-snowbank-wide" fill="#e0f2fe" opacity="0.88" d="M0 118 C38 104 72 130 110 114 S172 102 220 118 L220 150 L0 150 Z" />
  <circle id="skin-15-flake-wide-a" fill="#f8fafc" opacity="0.95" cx="38" cy="34" r="6" />
  <circle id="skin-15-flake-wide-b" fill="#f8fafc" opacity="0.95" cx="132" cy="24" r="5" />
  <circle id="skin-15-flake-wide-c" fill="#f8fafc" opacity="0.95" cx="196" cy="42" r="6" />
  <path id="identity-15-snowbank" fill="#e0f2fe" opacity="0.9" d="M0 124 C34 114 62 130 94 120 S158 112 220 124 L220 150 L0 150 Z" />
  <circle id="identity-15-flake-a" fill="#f8fafc" cx="44" cy="31" r="3" />
  <circle id="identity-15-flake-b" fill="#f8fafc" cx="128" cy="22" r="2.5" />
  <circle id="identity-15-flake-c" fill="#f8fafc" cx="186" cy="38" r="3" />
  <circle id="variant-15-dot" fill="#f43f5e" opacity="0.72" cx="25" cy="35" r="2.2" />
  <line id="variant-15-rule" stroke="#f43f5e" stroke-width="1.2" opacity="0.58" x1="31" y1="38" x2="47" y2="36" />
  <rect id="variant-15-chip-01" fill="#84cc16" opacity="0.68" x="11" y="4" width="7" height="3" rx="1" />
  <rect id="variant-15-chip-02" fill="#22c55e" opacity="0.68" x="42" y="5" width="8" height="4" rx="1" />
  <rect id="variant-15-chip-03" fill="#14b8a6" opacity="0.68" x="79" y="6" width="9" height="3" rx="1" />
  <rect id="variant-15-chip-04" fill="#06b6d4" opacity="0.68" x="110" y="4" width="7" height="4" rx="1" />
  <rect id="variant-15-chip-05" fill="#3b82f6" opacity="0.68" x="147" y="5" width="8" height="3" rx="1" />
  <rect id="variant-15-chip-06" fill="#6366f1" opacity="0.68" x="178" y="6" width="9" height="4" rx="1" />
  <rect id="variant-15-chip-07" fill="#8b5cf6" opacity="0.68" x="11" y="141" width="7" height="3" rx="1" />
  <rect id="variant-15-chip-08" fill="#d946ef" opacity="0.68" x="42" y="140" width="8" height="4" rx="1" />
  <rect id="variant-15-chip-09" fill="#ec4899" opacity="0.68" x="79" y="139" width="9" height="3" rx="1" />
  <rect id="variant-15-chip-10" fill="#ef4444" opacity="0.68" x="110" y="141" width="7" height="4" rx="1" />
  <rect id="variant-15-chip-11" fill="#f97316" opacity="0.68" x="147" y="140" width="8" height="3" rx="1" />
  <rect id="variant-15-chip-12" fill="#f59e0b" opacity="0.68" x="178" y="139" width="9" height="4" rx="1" />
  <circle id="variant-15-pin-01" fill="#06b6d4" opacity="0.66" cx="17" cy="22" r="2.55" />
  <circle id="variant-15-pin-02" fill="#3b82f6" opacity="0.66" cx="202" cy="36" r="1.8" />
  <circle id="variant-15-pin-03" fill="#6366f1" opacity="0.66" cx="27" cy="50" r="2.05" />
  <circle id="variant-15-pin-04" fill="#8b5cf6" opacity="0.66" cx="212" cy="64" r="2.3" />
  <circle id="variant-15-pin-05" fill="#d946ef" opacity="0.66" cx="13" cy="78" r="2.55" />
  <circle id="variant-15-pin-06" fill="#ec4899" opacity="0.66" cx="198" cy="92" r="1.8" />
  <circle id="variant-15-pin-07" fill="#ef4444" opacity="0.66" cx="23" cy="106" r="2.05" />
  <circle id="variant-15-pin-08" fill="#f97316" opacity="0.66" cx="208" cy="120" r="2.3" />
</svg>