Vector-Bench · a thetalab benchmark

theta labs / task

sv_016very_hard

scenic_multi

instructionnatural language

The lighthouse window should glow yellow and its beam should point cleanly across the harbor. Remove the extra buoy near the pier, make the small sail light blue, smooth the long wave, and return the left crate to the cargo stack. Do not alter the rest.

initial · brokeninput

target · expected fixcanonical

published model outputs

What each model produced.

34 endpoints

model result explorerLaguna M.1

Laguna M.1

poolside/laguna-m.1

TracePARTIAL

requested repairs

5/7

semantic preservation

clean

SVG validity

valid

source fidelity

unchanged

target match

different

Full pass requires the first three gates. Source fidelity and canonical target match are diagnostics only.

outcome

partial

progress

69.3%

UCR

0.0%

preserve

100.0%

elapsed

224.9s

cost

$0.0022

requested repairs5/7

crate-left.xpassed

expected 12 / produced 14

numeric-delta: 2 <= 4 user units / 75.0% repaired

crate-left.ypassed

expected 76 / produced 76

numeric-delta: 0 <= 2.5 user units / 100.0% repaired

harbor-wave-long.dmissed

expected M82 118 C98 112 112 124 128 118 S158 112 174 118 / produced M82 116 C98 122 112 116 128 116 C144 116 158 122 174 116

path-shape: 4.848 <= 4.2 sampled geometry / 47.0% repaired

sail-small.fillpassed

expected #e0f2fe / produced #bae6fd

color-lab: 11.209 <= 13.679 Delta E76 / 63.1% repaired

extra-pier-buoy.existspassed

expected false / produced false

required presence matches

light-beam.pointsmissed

expected [[180,30],[132,18],[132,42]] / produced [[191,30],[60,70],[58,90]]

points-rms: 51.298 <= 3.479 coordinate RMS / 0.0% repaired

lighthouse-window.fillpassed

expected #fde047 / produced #fde047

color-lab: 0 <= 18 Delta E76 / 100.0% repaired

Why it did not pass

requested repairs incomplete

Unexpected changes

None

Semantic preservation failures

None

Source fidelity diagnostics

None

prompt 3,402 tokoutput 3,933 tokreasoning 701 tokceiling 4,119 tokfinish stopopen weight

resolved poolside/laguna-m.1

response gen-1784544092-oolEeTpvEnLbDrn3966K

expected diff7 edits

Part	Attr	Before	After
crate-left	x	4	12
crate-left	y	81	76
harbor-wave-long	d	M82 116 C96 126 112 108 128 116 S158 126 174 116	M82 118 C98 112 112 124 128 118 S158 112 174 118
sail-small	fill	#fef3c7	#e0f2fe
extra-pier-buoy	exists	true	false
light-beam	points	[[188,26],[140,14],[140,38]]	[[180,30],[132,18],[132,42]]
lighthouse-window	fill	#38bdf8	#fde047

partsscope

Target (6)

crate-left

harbor-wave-long

sail-small

extra-pier-buoy

light-beam

lighthouse-window

Should preserve (56)

sky

water

horizon

sun

cloud-left

cloud-right

pier-deck

rope-coil

pier-leg-1

pier-leg-2

boat-hull

boat-cabin

cabin-window

mast

sail-main

flag

lighthouse-body

lighthouse-stripe-1

lighthouse-stripe-2

lighthouse-cap

buoy-red

buoy-red-stripe

buoy-green

buoy-green-stripe

wave-1

wave-2

crate-right

skin-16-sub-wide-body

skin-16-sub-wide-window

skin-16-sub-wide-tail

identity-16-sub-body

identity-16-sub-window

identity-16-sub-tail

identity-16-fish

variant-16-dot

variant-16-rule

variant-16-chip-01

variant-16-chip-02

variant-16-chip-03

variant-16-chip-04

variant-16-chip-05

variant-16-chip-06

variant-16-chip-07

variant-16-chip-08

variant-16-chip-09

variant-16-chip-10

variant-16-chip-11

variant-16-chip-12

variant-16-pin-01

variant-16-pin-02

variant-16-pin-03

variant-16-pin-04

variant-16-pin-05

variant-16-pin-06

variant-16-pin-07

variant-16-pin-08

initial svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 220 140" width="220" height="140">
<rect id="__bg" x="0" y="0" width="220" height="140" fill="#f5fffd" />
  <rect id="sky" fill="#dbeafe" x="0" y="0" width="220" height="70" />
  <rect id="water" fill="#06b6d4" x="0" y="70" width="220" height="70" />
  <line id="horizon" stroke="#0284c7" stroke-width="2" x1="0" y1="70" x2="220" y2="70" />
  <circle id="sun" fill="#fde047" cx="190" cy="22" r="12" />
  <ellipse id="cloud-left" fill="#f8fafc" cx="36" cy="26" rx="18" ry="7" />
  <ellipse id="cloud-right" fill="#f8fafc" cx="56" cy="25" rx="14" ry="6" />
  <rect id="pier-deck" fill="#92400e" x="0" y="88" width="72" height="10" />
  <path id="rope-coil" fill="none" stroke="#f8fafc" stroke-width="2" d="M48 82 C58 72 70 74 70 84 C70 94 54 96 50 88 C46 80 60 78 62 86" />
  <line id="pier-leg-1" stroke="#451a03" stroke-width="4" x1="16" y1="96" x2="13" y2="132" />
  <line id="pier-leg-2" stroke="#451a03" stroke-width="4" x1="56" y1="96" x2="60" y2="132" />
  <polygon id="boat-hull" fill="#f59e0b" stroke="#222" stroke-width="1" points="84,84 158,84 146,104 96,104" />
  <rect id="boat-cabin" fill="#fefce8" stroke="#222" stroke-width="1" x="106" y="62" width="30" height="20" />
  <rect id="cabin-window" fill="#93c5fd" stroke="#222" stroke-width="0.8" x="114" y="67" width="14" height="8" />
  <line id="mast" stroke="#222" stroke-width="3" x1="126" y1="84" x2="126" y2="26" />
  <polygon id="sail-main" fill="#ecfeff" stroke="#222" stroke-width="1" points="126,30 154,76 126,76" />
  <polygon id="sail-small" fill="#fef3c7" stroke="#222" stroke-width="1" points="124,34 100,76 124,76" />
  <polygon id="flag" fill="#0f766e" points="126,25 140,29 126,33" />
  <polygon id="lighthouse-body" fill="#f8fafc" stroke="#222" stroke-width="1" points="176,82 206,82 200,28 182,28" />
  <rect id="lighthouse-stripe-1" fill="#ef4444" x="181" y="48" width="20" height="7" />
  <rect id="lighthouse-stripe-2" fill="#ef4444" x="179" y="64" width="24" height="7" />
  <rect id="lighthouse-cap" fill="#0f172a" x="180" y="22" width="22" height="8" />
  <rect id="lighthouse-window" fill="#38bdf8" stroke="#222" stroke-width="0.7" x="188" y="34" width="6" height="8" />
  <polygon id="light-beam" fill="#fde68a" opacity="0.45" points="188,26 140,14 140,38" />
  <circle id="buoy-red" fill="#f8fafc" stroke="#222" stroke-width="1" cx="42" cy="112" r="8" />
  <rect id="buoy-red-stripe" fill="#ef4444" x="35" y="110" width="14" height="4" />
  <circle id="buoy-green" fill="#f8fafc" stroke="#222" stroke-width="1" cx="184" cy="112" r="7" />
  <rect id="buoy-green-stripe" fill="#22c55e" x="178" y="110" width="12" height="4" />
  <polyline id="wave-1" fill="none" stroke="#0ea5e9" stroke-width="2" points="6,122 22,118 38,122 54,118 70,122" />
  <path id="harbor-wave-long" fill="none" stroke="#0ea5e9" stroke-width="2" d="M82 116 C96 126 112 108 128 116 S158 126 174 116" />
  <polyline id="wave-2" fill="none" stroke="#0ea5e9" stroke-width="2" points="128,120 146,116 164,120 182,116 200,120" />
  <rect id="crate-left" fill="#a16207" stroke="#451a03" stroke-width="1" x="4" y="81" width="14" height="12" />
  <rect id="crate-right" fill="#b45309" stroke="#451a03" stroke-width="1" x="28" y="76" width="14" height="12" />
  <ellipse id="skin-16-sub-wide-body" fill="#facc15" stroke="#854d0e" stroke-width="2.2" opacity="0.9" cx="53" cy="112" rx="43" ry="18" />
  <circle id="skin-16-sub-wide-window" fill="#bae6fd" stroke="#075985" stroke-width="1.5" opacity="0.95" cx="42" cy="109" r="7" />
  <polygon id="skin-16-sub-wide-tail" fill="#f59e0b" stroke="#854d0e" stroke-width="1.5" opacity="0.9" points="93,112 116,96 116,128" />
  <ellipse id="identity-16-sub-body" fill="#facc15" stroke="#854d0e" stroke-width="1.5" cx="52" cy="112" rx="28" ry="11" />
  <circle id="identity-16-sub-window" fill="#bae6fd" stroke="#075985" stroke-width="1" cx="43" cy="110" r="4" />
  <polygon id="identity-16-sub-tail" fill="#f59e0b" stroke="#854d0e" stroke-width="1" points="80,112 94,104 94,120" />
  <polygon id="identity-16-fish" fill="#fb7185" opacity="0.95" points="152,122 166,116 166,128" />
  <circle id="variant-16-dot" fill="#0f766e" opacity="0.72" cx="42" cy="18" r="2.6" />
  <line id="variant-16-rule" stroke="#0f766e" stroke-width="1.2" opacity="0.58" x1="48" y1="21" x2="64" y2="24" />
  <rect id="variant-16-chip-01" fill="#22c55e" opacity="0.68" x="12" y="5" width="8" height="3" rx="1" />
  <rect id="variant-16-chip-02" fill="#14b8a6" opacity="0.68" x="43" y="6" width="9" height="4" rx="1" />
  <rect id="variant-16-chip-03" fill="#06b6d4" opacity="0.68" x="80" y="4" width="7" height="3" rx="1" />
  <rect id="variant-16-chip-04" fill="#3b82f6" opacity="0.68" x="111" y="5" width="8" height="4" rx="1" />
  <rect id="variant-16-chip-05" fill="#6366f1" opacity="0.68" x="148" y="6" width="9" height="3" rx="1" />
  <rect id="variant-16-chip-06" fill="#8b5cf6" opacity="0.68" x="179" y="4" width="7" height="4" rx="1" />
  <rect id="variant-16-chip-07" fill="#d946ef" opacity="0.68" x="12" y="140" width="8" height="3" rx="1" />
  <rect id="variant-16-chip-08" fill="#ec4899" opacity="0.68" x="43" y="139" width="9" height="4" rx="1" />
  <rect id="variant-16-chip-09" fill="#ef4444" opacity="0.68" x="80" y="141" width="7" height="3" rx="1" />
  <rect id="variant-16-chip-10" fill="#f97316" opacity="0.68" x="111" y="140" width="8" height="4" rx="1" />
  <rect id="variant-16-chip-11" fill="#f59e0b" opacity="0.68" x="148" y="139" width="9" height="3" rx="1" />
  <rect id="variant-16-chip-12" fill="#84cc16" opacity="0.68" x="179" y="141" width="7" height="4" rx="1" />
  <circle id="variant-16-pin-01" fill="#6366f1" opacity="0.66" cx="24" cy="23" r="1.8" />
  <circle id="variant-16-pin-02" fill="#8b5cf6" opacity="0.66" cx="197" cy="37" r="2.05" />
  <circle id="variant-16-pin-03" fill="#d946ef" opacity="0.66" cx="10" cy="51" r="2.3" />
  <circle id="variant-16-pin-04" fill="#ec4899" opacity="0.66" cx="207" cy="65" r="2.55" />
  <circle id="variant-16-pin-05" fill="#ef4444" opacity="0.66" cx="20" cy="79" r="1.8" />
  <circle id="variant-16-pin-06" fill="#f97316" opacity="0.66" cx="193" cy="93" r="2.05" />
  <circle id="variant-16-pin-07" fill="#f59e0b" opacity="0.66" cx="30" cy="107" r="2.3" />
  <circle id="variant-16-pin-08" fill="#84cc16" opacity="0.66" cx="203" cy="121" r="2.55" />
  <circle id="extra-pier-buoy" fill="#facc15" stroke="#222" stroke-width="0.8" cx="70" cy="112" r="5" />
</svg>

target svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 220 140" width="220" height="140">
<rect id="__bg" x="0" y="0" width="220" height="140" fill="#f5fffd" />
  <rect id="sky" fill="#dbeafe" x="0" y="0" width="220" height="70" />
  <rect id="water" fill="#06b6d4" x="0" y="70" width="220" height="70" />
  <line id="horizon" stroke="#0284c7" stroke-width="2" x1="0" y1="70" x2="220" y2="70" />
  <circle id="sun" fill="#fde047" cx="190" cy="22" r="12" />
  <ellipse id="cloud-left" fill="#f8fafc" cx="36" cy="26" rx="18" ry="7" />
  <ellipse id="cloud-right" fill="#f8fafc" cx="56" cy="25" rx="14" ry="6" />
  <rect id="pier-deck" fill="#92400e" x="0" y="88" width="72" height="10" />
  <path id="rope-coil" fill="none" stroke="#f8fafc" stroke-width="2" d="M48 82 C58 72 70 74 70 84 C70 94 54 96 50 88 C46 80 60 78 62 86" />
  <line id="pier-leg-1" stroke="#451a03" stroke-width="4" x1="16" y1="96" x2="13" y2="132" />
  <line id="pier-leg-2" stroke="#451a03" stroke-width="4" x1="56" y1="96" x2="60" y2="132" />
  <polygon id="boat-hull" fill="#f59e0b" stroke="#222" stroke-width="1" points="84,84 158,84 146,104 96,104" />
  <rect id="boat-cabin" fill="#fefce8" stroke="#222" stroke-width="1" x="106" y="62" width="30" height="20" />
  <rect id="cabin-window" fill="#93c5fd" stroke="#222" stroke-width="0.8" x="114" y="67" width="14" height="8" />
  <line id="mast" stroke="#222" stroke-width="3" x1="126" y1="84" x2="126" y2="26" />
  <polygon id="sail-main" fill="#ecfeff" stroke="#222" stroke-width="1" points="126,30 154,76 126,76" />
  <polygon id="sail-small" fill="#e0f2fe" stroke="#222" stroke-width="1" points="124,34 100,76 124,76" />
  <polygon id="flag" fill="#0f766e" points="126,25 140,29 126,33" />
  <polygon id="lighthouse-body" fill="#f8fafc" stroke="#222" stroke-width="1" points="176,82 206,82 200,28 182,28" />
  <rect id="lighthouse-stripe-1" fill="#ef4444" x="181" y="48" width="20" height="7" />
  <rect id="lighthouse-stripe-2" fill="#ef4444" x="179" y="64" width="24" height="7" />
  <rect id="lighthouse-cap" fill="#0f172a" x="180" y="22" width="22" height="8" />
  <rect id="lighthouse-window" fill="#fde047" stroke="#222" stroke-width="0.7" x="188" y="34" width="6" height="8" />
  <polygon id="light-beam" fill="#fde68a" opacity="0.45" points="180,30 132,18 132,42" />
  <circle id="buoy-red" fill="#f8fafc" stroke="#222" stroke-width="1" cx="42" cy="112" r="8" />
  <rect id="buoy-red-stripe" fill="#ef4444" x="35" y="110" width="14" height="4" />
  <circle id="buoy-green" fill="#f8fafc" stroke="#222" stroke-width="1" cx="184" cy="112" r="7" />
  <rect id="buoy-green-stripe" fill="#22c55e" x="178" y="110" width="12" height="4" />
  <polyline id="wave-1" fill="none" stroke="#0ea5e9" stroke-width="2" points="6,122 22,118 38,122 54,118 70,122" />
  <path id="harbor-wave-long" fill="none" stroke="#0ea5e9" stroke-width="2" d="M82 118 C98 112 112 124 128 118 S158 112 174 118" />
  <polyline id="wave-2" fill="none" stroke="#0ea5e9" stroke-width="2" points="128,120 146,116 164,120 182,116 200,120" />
  <rect id="crate-left" fill="#a16207" stroke="#451a03" stroke-width="1" x="12" y="76" width="14" height="12" />
  <rect id="crate-right" fill="#b45309" stroke="#451a03" stroke-width="1" x="28" y="76" width="14" height="12" />
  <ellipse id="skin-16-sub-wide-body" fill="#facc15" stroke="#854d0e" stroke-width="2.2" opacity="0.9" cx="53" cy="112" rx="43" ry="18" />
  <circle id="skin-16-sub-wide-window" fill="#bae6fd" stroke="#075985" stroke-width="1.5" opacity="0.95" cx="42" cy="109" r="7" />
  <polygon id="skin-16-sub-wide-tail" fill="#f59e0b" stroke="#854d0e" stroke-width="1.5" opacity="0.9" points="93,112 116,96 116,128" />
  <ellipse id="identity-16-sub-body" fill="#facc15" stroke="#854d0e" stroke-width="1.5" cx="52" cy="112" rx="28" ry="11" />
  <circle id="identity-16-sub-window" fill="#bae6fd" stroke="#075985" stroke-width="1" cx="43" cy="110" r="4" />
  <polygon id="identity-16-sub-tail" fill="#f59e0b" stroke="#854d0e" stroke-width="1" points="80,112 94,104 94,120" />
  <polygon id="identity-16-fish" fill="#fb7185" opacity="0.95" points="152,122 166,116 166,128" />
  <circle id="variant-16-dot" fill="#0f766e" opacity="0.72" cx="42" cy="18" r="2.6" />
  <line id="variant-16-rule" stroke="#0f766e" stroke-width="1.2" opacity="0.58" x1="48" y1="21" x2="64" y2="24" />
  <rect id="variant-16-chip-01" fill="#22c55e" opacity="0.68" x="12" y="5" width="8" height="3" rx="1" />
  <rect id="variant-16-chip-02" fill="#14b8a6" opacity="0.68" x="43" y="6" width="9" height="4" rx="1" />
  <rect id="variant-16-chip-03" fill="#06b6d4" opacity="0.68" x="80" y="4" width="7" height="3" rx="1" />
  <rect id="variant-16-chip-04" fill="#3b82f6" opacity="0.68" x="111" y="5" width="8" height="4" rx="1" />
  <rect id="variant-16-chip-05" fill="#6366f1" opacity="0.68" x="148" y="6" width="9" height="3" rx="1" />
  <rect id="variant-16-chip-06" fill="#8b5cf6" opacity="0.68" x="179" y="4" width="7" height="4" rx="1" />
  <rect id="variant-16-chip-07" fill="#d946ef" opacity="0.68" x="12" y="140" width="8" height="3" rx="1" />
  <rect id="variant-16-chip-08" fill="#ec4899" opacity="0.68" x="43" y="139" width="9" height="4" rx="1" />
  <rect id="variant-16-chip-09" fill="#ef4444" opacity="0.68" x="80" y="141" width="7" height="3" rx="1" />
  <rect id="variant-16-chip-10" fill="#f97316" opacity="0.68" x="111" y="140" width="8" height="4" rx="1" />
  <rect id="variant-16-chip-11" fill="#f59e0b" opacity="0.68" x="148" y="139" width="9" height="3" rx="1" />
  <rect id="variant-16-chip-12" fill="#84cc16" opacity="0.68" x="179" y="141" width="7" height="4" rx="1" />
  <circle id="variant-16-pin-01" fill="#6366f1" opacity="0.66" cx="24" cy="23" r="1.8" />
  <circle id="variant-16-pin-02" fill="#8b5cf6" opacity="0.66" cx="197" cy="37" r="2.05" />
  <circle id="variant-16-pin-03" fill="#d946ef" opacity="0.66" cx="10" cy="51" r="2.3" />
  <circle id="variant-16-pin-04" fill="#ec4899" opacity="0.66" cx="207" cy="65" r="2.55" />
  <circle id="variant-16-pin-05" fill="#ef4444" opacity="0.66" cx="20" cy="79" r="1.8" />
  <circle id="variant-16-pin-06" fill="#f97316" opacity="0.66" cx="193" cy="93" r="2.05" />
  <circle id="variant-16-pin-07" fill="#f59e0b" opacity="0.66" cx="30" cy="107" r="2.3" />
  <circle id="variant-16-pin-08" fill="#84cc16" opacity="0.66" cx="203" cy="121" r="2.55" />
</svg>