Vector-Bench · a thetalab benchmark

theta labs / task

sv_019very_hard

scenic_multi

instructionnatural language

Repair the harbor by turning the main sail around, making the red buoy stripe red again, and placing the cabin window back inside the cabin. Remove the stray pier rope, slim down the mast, neaten the rope coil, and return the green buoy to its proper spot.

initial · brokeninput

target · expected fixcanonical

published model outputs

What each model produced.

34 endpoints

model result explorerKimi K2.6

Kimi K2.6

moonshotai/kimi-k2.6

TraceNEAR

requested repairs

8/9

semantic preservation

clean

SVG validity

valid

source fidelity

unchanged

target match

different

Full pass requires the first three gates. Source fidelity and canonical target match are diagnostics only.

outcome

near

progress

84.3%

UCR

0.0%

preserve

100.0%

elapsed

1201.2s

cost

$0.1094

requested repairs8/9

buoy-green.cxpassed

expected 184 / produced 184

numeric-delta: 0 <= 5.5 user units / 100.0% repaired

buoy-green.cypassed

expected 112 / produced 112

numeric-delta: 0 <= 2 user units / 100.0% repaired

rope-coil.dmissed

expected M48 82 C58 72 70 74 70 84 C70 94 54 96 50 88 C46 80 60 78 62 86 / produced M44 92 C44 88 56 88 56 92 C56 96 44 96 44 92 M47 92 C47 90 53 90 53 92 C53 94 47 94 47 92

path-shape: 17.363 <= 1.981 sampled geometry / 0.0% repaired

mast.stroke-widthpassed

expected 3 / produced 3

line-weight-delta: 0 <= 1.2 user units / 100.0% repaired

stray-pier-rope.existspassed

expected false / produced false

required presence matches

cabin-window.xpassed

expected 114 / produced 114

numeric-delta: 0 <= 4 user units / 100.0% repaired

cabin-window.ypassed

expected 67 / produced 68

numeric-delta: 1 <= 1.5 user units / 66.7% repaired

buoy-red-stripe.fillpassed

expected #ef4444 / produced #ef4444

color-lab: 0 <= 18 Delta E76 / 100.0% repaired

sail-main.pointspassed

expected [[126,30],[154,76],[126,76]] / produced [[128,30],[156,76],[128,76]]

points-rms: 1.414 <= 4.2 coordinate RMS / 92.4% repaired

Why it did not pass

requested repairs incomplete

Unexpected changes

None

Semantic preservation failures

None

Source fidelity diagnostics

None

prompt 2,776 tokoutput 26,696 tokreasoning 24,082 tokceiling 4,096 tokfinish stopopen weight

resolved moonshotai/kimi-k2.6

response gen-1784544427-NtQkq4DfGBMg3P9tHL0M

expected diff9 edits

Part	Attr	Before	After
buoy-green	cx	172	184
buoy-green	cy	116	112
rope-coil	d	M48 82 C58 70 72 76 70 88 C68 100 50 94 52 84 C54 76 64 78 66 88	M48 82 C58 72 70 74 70 84 C70 94 54 96 50 88 C46 80 60 78 62 86
mast	stroke-width	6	3
stray-pier-rope	exists	true	false
cabin-window	x	106	114
cabin-window	y	70	67
buoy-red-stripe	fill	#22c55e	#ef4444
sail-main	points	[[144.66666666666669,30],[116.66666666666669,76],[144.66666666666669,76]]	[[126,30],[154,76],[126,76]]

partsscope

Target (7)

buoy-green

rope-coil

mast

stray-pier-rope

cabin-window

buoy-red-stripe

sail-main

Should preserve (55)

sky

water

horizon

sun

cloud-left

cloud-right

pier-deck

pier-leg-1

pier-leg-2

boat-hull

boat-cabin

sail-small

flag

lighthouse-body

lighthouse-stripe-1

lighthouse-stripe-2

lighthouse-cap

lighthouse-window

light-beam

buoy-red

buoy-green-stripe

wave-1

harbor-wave-long

wave-2

crate-left

crate-right

skin-19-thunder-cloud-a

skin-19-thunder-cloud-b

skin-19-thunder-bolt

identity-19-storm-cloud-a

identity-19-storm-cloud-b

identity-19-lightning

identity-19-storm-rain

variant-19-dot

variant-19-rule

variant-19-chip-01

variant-19-chip-02

variant-19-chip-03

variant-19-chip-04

variant-19-chip-05

variant-19-chip-06

variant-19-chip-07

variant-19-chip-08

variant-19-chip-09

variant-19-chip-10

variant-19-chip-11

variant-19-chip-12

variant-19-pin-01

variant-19-pin-02

variant-19-pin-03

variant-19-pin-04

variant-19-pin-05

variant-19-pin-06

variant-19-pin-07

variant-19-pin-08

initial svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 220 140" width="220" height="140">
<rect id="__bg" x="0" y="0" width="220" height="140" fill="#f5fbff" />
  <rect id="sky" fill="#cbd5e1" x="0" y="0" width="220" height="70" />
  <rect id="water" fill="#0f766e" x="0" y="70" width="220" height="70" />
  <line id="horizon" stroke="#0284c7" stroke-width="2" x1="0" y1="70" x2="220" y2="70" />
  <circle id="sun" fill="#fde047" cx="190" cy="22" r="12" />
  <ellipse id="cloud-left" fill="#ffffff" cx="36" cy="26" rx="18" ry="7" />
  <ellipse id="cloud-right" fill="#ffffff" cx="56" cy="25" rx="14" ry="6" />
  <rect id="pier-deck" fill="#92400e" x="0" y="88" width="72" height="10" />
  <path id="rope-coil" fill="none" stroke="#f8fafc" stroke-width="2" d="M48 82 C58 70 72 76 70 88 C68 100 50 94 52 84 C54 76 64 78 66 88" />
  <line id="pier-leg-1" stroke="#451a03" stroke-width="4" x1="16" y1="96" x2="13" y2="132" />
  <line id="pier-leg-2" stroke="#451a03" stroke-width="4" x1="56" y1="96" x2="60" y2="132" />
  <polygon id="boat-hull" fill="#1e293b" stroke="#222" stroke-width="1" points="84,84 158,84 146,104 96,104" />
  <rect id="boat-cabin" fill="#e2e8f0" stroke="#222" stroke-width="1" x="106" y="62" width="30" height="20" />
  <rect id="cabin-window" fill="#93c5fd" stroke="#222" stroke-width="0.8" x="106" y="70" width="14" height="8" />
  <line id="mast" stroke="#222" stroke-width="6" x1="126" y1="84" x2="126" y2="26" />
  <polygon id="sail-main" fill="#dbeafe" stroke="#222" stroke-width="1" points="144.66666666666669,30 116.66666666666669,76 144.66666666666669,76" />
  <polygon id="sail-small" fill="#94a3b8" stroke="#222" stroke-width="1" points="124,34 100,76 124,76" />
  <polygon id="flag" fill="#0284c7" points="126,25 140,29 126,33" />
  <polygon id="lighthouse-body" fill="#e5e7eb" stroke="#222" stroke-width="1" points="176,82 206,82 200,28 182,28" />
  <rect id="lighthouse-stripe-1" fill="#334155" x="181" y="48" width="20" height="7" />
  <rect id="lighthouse-stripe-2" fill="#334155" x="179" y="64" width="24" height="7" />
  <rect id="lighthouse-cap" fill="#0f172a" x="180" y="22" width="22" height="8" />
  <rect id="lighthouse-window" fill="#fde047" stroke="#222" stroke-width="0.7" x="188" y="34" width="6" height="8" />
  <polygon id="light-beam" fill="#fde68a" opacity="0.45" points="180,30 132,18 132,42" />
  <circle id="buoy-red" fill="#f8fafc" stroke="#222" stroke-width="1" cx="42" cy="112" r="8" />
  <rect id="buoy-red-stripe" fill="#22c55e" x="35" y="110" width="14" height="4" />
  <circle id="buoy-green" fill="#f8fafc" stroke="#222" stroke-width="1" cx="172" cy="116" r="7" />
  <rect id="buoy-green-stripe" fill="#22c55e" x="178" y="110" width="12" height="4" />
  <polyline id="wave-1" fill="none" stroke="#0ea5e9" stroke-width="2" points="6,122 22,118 38,122 54,118 70,122" />
  <path id="harbor-wave-long" fill="none" stroke="#0ea5e9" stroke-width="2" d="M82 118 C98 112 112 124 128 118 S158 112 174 118" />
  <polyline id="wave-2" fill="none" stroke="#0ea5e9" stroke-width="2" points="128,120 146,116 164,120 182,116 200,120" />
  <rect id="crate-left" fill="#a16207" stroke="#451a03" stroke-width="1" x="12" y="76" width="14" height="12" />
  <rect id="crate-right" fill="#b45309" stroke="#451a03" stroke-width="1" x="28" y="76" width="14" height="12" />
  <ellipse id="skin-19-thunder-cloud-a" fill="#64748b" opacity="0.9" cx="88" cy="28" rx="39" ry="18" />
  <ellipse id="skin-19-thunder-cloud-b" fill="#475569" opacity="0.9" cx="130" cy="32" rx="35" ry="22" />
  <polygon id="skin-19-thunder-bolt" fill="#fde047" opacity="0.95" points="120,48 100,92 124,82 108,132 151,68 128,76" />
  <ellipse id="identity-19-storm-cloud-a" fill="#64748b" opacity="0.95" cx="92" cy="24" rx="24" ry="11" />
  <ellipse id="identity-19-storm-cloud-b" fill="#475569" opacity="0.95" cx="118" cy="27" rx="20" ry="13" />
  <polygon id="identity-19-lightning" fill="#fde047" points="112,37 101,62 114,57 105,82 130,48 116,52" />
  <line id="identity-19-storm-rain" stroke="#38bdf8" stroke-width="2" x1="86" y1="45" x2="78" y2="65" />
  <circle id="variant-19-dot" fill="#0284c7" opacity="0.72" cx="45" cy="23" r="2.6" />
  <line id="variant-19-rule" stroke="#0284c7" stroke-width="1.2" opacity="0.58" x1="51" y1="26" x2="67" y2="24" />
  <rect id="variant-19-chip-01" fill="#3b82f6" opacity="0.68" x="9" y="5" width="8" height="3" rx="1" />
  <rect id="variant-19-chip-02" fill="#6366f1" opacity="0.68" x="46" y="6" width="9" height="4" rx="1" />
  <rect id="variant-19-chip-03" fill="#8b5cf6" opacity="0.68" x="77" y="4" width="7" height="3" rx="1" />
  <rect id="variant-19-chip-04" fill="#d946ef" opacity="0.68" x="114" y="5" width="8" height="4" rx="1" />
  <rect id="variant-19-chip-05" fill="#ec4899" opacity="0.68" x="145" y="6" width="9" height="3" rx="1" />
  <rect id="variant-19-chip-06" fill="#ef4444" opacity="0.68" x="182" y="4" width="7" height="4" rx="1" />
  <rect id="variant-19-chip-07" fill="#f97316" opacity="0.68" x="9" y="140" width="8" height="3" rx="1" />
  <rect id="variant-19-chip-08" fill="#f59e0b" opacity="0.68" x="46" y="139" width="9" height="4" rx="1" />
  <rect id="variant-19-chip-09" fill="#84cc16" opacity="0.68" x="77" y="141" width="7" height="3" rx="1" />
  <rect id="variant-19-chip-10" fill="#22c55e" opacity="0.68" x="114" y="140" width="8" height="4" rx="1" />
  <rect id="variant-19-chip-11" fill="#14b8a6" opacity="0.68" x="145" y="139" width="9" height="3" rx="1" />
  <rect id="variant-19-chip-12" fill="#06b6d4" opacity="0.68" x="182" y="141" width="7" height="4" rx="1" />
  <circle id="variant-19-pin-01" fill="#f59e0b" opacity="0.66" cx="21" cy="26" r="2.55" />
  <circle id="variant-19-pin-02" fill="#84cc16" opacity="0.66" cx="206" cy="40" r="1.8" />
  <circle id="variant-19-pin-03" fill="#22c55e" opacity="0.66" cx="31" cy="54" r="2.05" />
  <circle id="variant-19-pin-04" fill="#14b8a6" opacity="0.66" cx="192" cy="68" r="2.3" />
  <circle id="variant-19-pin-05" fill="#06b6d4" opacity="0.66" cx="17" cy="82" r="2.55" />
  <circle id="variant-19-pin-06" fill="#3b82f6" opacity="0.66" cx="202" cy="96" r="1.8" />
  <circle id="variant-19-pin-07" fill="#6366f1" opacity="0.66" cx="27" cy="110" r="2.05" />
  <circle id="variant-19-pin-08" fill="#8b5cf6" opacity="0.66" cx="212" cy="124" r="2.3" />
  <polyline id="stray-pier-rope" fill="none" stroke="#f8fafc" stroke-width="2" points="8,84 20,80 32,84" />
</svg>

target svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 220 140" width="220" height="140">
<rect id="__bg" x="0" y="0" width="220" height="140" fill="#f5fbff" />
  <rect id="sky" fill="#cbd5e1" x="0" y="0" width="220" height="70" />
  <rect id="water" fill="#0f766e" x="0" y="70" width="220" height="70" />
  <line id="horizon" stroke="#0284c7" stroke-width="2" x1="0" y1="70" x2="220" y2="70" />
  <circle id="sun" fill="#fde047" cx="190" cy="22" r="12" />
  <ellipse id="cloud-left" fill="#ffffff" cx="36" cy="26" rx="18" ry="7" />
  <ellipse id="cloud-right" fill="#ffffff" cx="56" cy="25" rx="14" ry="6" />
  <rect id="pier-deck" fill="#92400e" x="0" y="88" width="72" height="10" />
  <path id="rope-coil" fill="none" stroke="#f8fafc" stroke-width="2" d="M48 82 C58 72 70 74 70 84 C70 94 54 96 50 88 C46 80 60 78 62 86" />
  <line id="pier-leg-1" stroke="#451a03" stroke-width="4" x1="16" y1="96" x2="13" y2="132" />
  <line id="pier-leg-2" stroke="#451a03" stroke-width="4" x1="56" y1="96" x2="60" y2="132" />
  <polygon id="boat-hull" fill="#1e293b" stroke="#222" stroke-width="1" points="84,84 158,84 146,104 96,104" />
  <rect id="boat-cabin" fill="#e2e8f0" stroke="#222" stroke-width="1" x="106" y="62" width="30" height="20" />
  <rect id="cabin-window" fill="#93c5fd" stroke="#222" stroke-width="0.8" x="114" y="67" width="14" height="8" />
  <line id="mast" stroke="#222" stroke-width="3" x1="126" y1="84" x2="126" y2="26" />
  <polygon id="sail-main" fill="#dbeafe" stroke="#222" stroke-width="1" points="126,30 154,76 126,76" />
  <polygon id="sail-small" fill="#94a3b8" stroke="#222" stroke-width="1" points="124,34 100,76 124,76" />
  <polygon id="flag" fill="#0284c7" points="126,25 140,29 126,33" />
  <polygon id="lighthouse-body" fill="#e5e7eb" stroke="#222" stroke-width="1" points="176,82 206,82 200,28 182,28" />
  <rect id="lighthouse-stripe-1" fill="#334155" x="181" y="48" width="20" height="7" />
  <rect id="lighthouse-stripe-2" fill="#334155" x="179" y="64" width="24" height="7" />
  <rect id="lighthouse-cap" fill="#0f172a" x="180" y="22" width="22" height="8" />
  <rect id="lighthouse-window" fill="#fde047" stroke="#222" stroke-width="0.7" x="188" y="34" width="6" height="8" />
  <polygon id="light-beam" fill="#fde68a" opacity="0.45" points="180,30 132,18 132,42" />
  <circle id="buoy-red" fill="#f8fafc" stroke="#222" stroke-width="1" cx="42" cy="112" r="8" />
  <rect id="buoy-red-stripe" fill="#ef4444" x="35" y="110" width="14" height="4" />
  <circle id="buoy-green" fill="#f8fafc" stroke="#222" stroke-width="1" cx="184" cy="112" r="7" />
  <rect id="buoy-green-stripe" fill="#22c55e" x="178" y="110" width="12" height="4" />
  <polyline id="wave-1" fill="none" stroke="#0ea5e9" stroke-width="2" points="6,122 22,118 38,122 54,118 70,122" />
  <path id="harbor-wave-long" fill="none" stroke="#0ea5e9" stroke-width="2" d="M82 118 C98 112 112 124 128 118 S158 112 174 118" />
  <polyline id="wave-2" fill="none" stroke="#0ea5e9" stroke-width="2" points="128,120 146,116 164,120 182,116 200,120" />
  <rect id="crate-left" fill="#a16207" stroke="#451a03" stroke-width="1" x="12" y="76" width="14" height="12" />
  <rect id="crate-right" fill="#b45309" stroke="#451a03" stroke-width="1" x="28" y="76" width="14" height="12" />
  <ellipse id="skin-19-thunder-cloud-a" fill="#64748b" opacity="0.9" cx="88" cy="28" rx="39" ry="18" />
  <ellipse id="skin-19-thunder-cloud-b" fill="#475569" opacity="0.9" cx="130" cy="32" rx="35" ry="22" />
  <polygon id="skin-19-thunder-bolt" fill="#fde047" opacity="0.95" points="120,48 100,92 124,82 108,132 151,68 128,76" />
  <ellipse id="identity-19-storm-cloud-a" fill="#64748b" opacity="0.95" cx="92" cy="24" rx="24" ry="11" />
  <ellipse id="identity-19-storm-cloud-b" fill="#475569" opacity="0.95" cx="118" cy="27" rx="20" ry="13" />
  <polygon id="identity-19-lightning" fill="#fde047" points="112,37 101,62 114,57 105,82 130,48 116,52" />
  <line id="identity-19-storm-rain" stroke="#38bdf8" stroke-width="2" x1="86" y1="45" x2="78" y2="65" />
  <circle id="variant-19-dot" fill="#0284c7" opacity="0.72" cx="45" cy="23" r="2.6" />
  <line id="variant-19-rule" stroke="#0284c7" stroke-width="1.2" opacity="0.58" x1="51" y1="26" x2="67" y2="24" />
  <rect id="variant-19-chip-01" fill="#3b82f6" opacity="0.68" x="9" y="5" width="8" height="3" rx="1" />
  <rect id="variant-19-chip-02" fill="#6366f1" opacity="0.68" x="46" y="6" width="9" height="4" rx="1" />
  <rect id="variant-19-chip-03" fill="#8b5cf6" opacity="0.68" x="77" y="4" width="7" height="3" rx="1" />
  <rect id="variant-19-chip-04" fill="#d946ef" opacity="0.68" x="114" y="5" width="8" height="4" rx="1" />
  <rect id="variant-19-chip-05" fill="#ec4899" opacity="0.68" x="145" y="6" width="9" height="3" rx="1" />
  <rect id="variant-19-chip-06" fill="#ef4444" opacity="0.68" x="182" y="4" width="7" height="4" rx="1" />
  <rect id="variant-19-chip-07" fill="#f97316" opacity="0.68" x="9" y="140" width="8" height="3" rx="1" />
  <rect id="variant-19-chip-08" fill="#f59e0b" opacity="0.68" x="46" y="139" width="9" height="4" rx="1" />
  <rect id="variant-19-chip-09" fill="#84cc16" opacity="0.68" x="77" y="141" width="7" height="3" rx="1" />
  <rect id="variant-19-chip-10" fill="#22c55e" opacity="0.68" x="114" y="140" width="8" height="4" rx="1" />
  <rect id="variant-19-chip-11" fill="#14b8a6" opacity="0.68" x="145" y="139" width="9" height="3" rx="1" />
  <rect id="variant-19-chip-12" fill="#06b6d4" opacity="0.68" x="182" y="141" width="7" height="4" rx="1" />
  <circle id="variant-19-pin-01" fill="#f59e0b" opacity="0.66" cx="21" cy="26" r="2.55" />
  <circle id="variant-19-pin-02" fill="#84cc16" opacity="0.66" cx="206" cy="40" r="1.8" />
  <circle id="variant-19-pin-03" fill="#22c55e" opacity="0.66" cx="31" cy="54" r="2.05" />
  <circle id="variant-19-pin-04" fill="#14b8a6" opacity="0.66" cx="192" cy="68" r="2.3" />
  <circle id="variant-19-pin-05" fill="#06b6d4" opacity="0.66" cx="17" cy="82" r="2.55" />
  <circle id="variant-19-pin-06" fill="#3b82f6" opacity="0.66" cx="202" cy="96" r="1.8" />
  <circle id="variant-19-pin-07" fill="#6366f1" opacity="0.66" cx="27" cy="110" r="2.05" />
  <circle id="variant-19-pin-08" fill="#8b5cf6" opacity="0.66" cx="212" cy="124" r="2.3" />
</svg>