Vector-Bench · a thetalab benchmark

theta labs / task

sv_012very_hard

scenic_very_hard

instructionnatural language

The bicycle frame is almost invisible while its left wheel is drawn too heavily. Remove the copied blue awning panel, make the odd red pear green again, and smooth out the handlebar. Leave the rest of the market untouched.

initial · brokeninput

target · expected fixcanonical

published model outputs

What each model produced.

34 endpoints

model result explorerMiniMax M3

MiniMax M3

minimax/minimax-m3

TracePASS

requested repairs

5/5

semantic preservation

clean

SVG validity

valid

source fidelity

unchanged

target match

different

Full pass requires the first three gates. Source fidelity and canonical target match are diagnostics only.

outcome

pass

progress

89.8%

UCR

0.0%

preserve

100.0%

elapsed

3563.1s

cost

$0.0048

requested repairs5/5

bike-handlebar.dpassed

expected M122 96 C130 88 136 92 136 98 / produced M122 96 C130 92 132 92 140 94

path-coordinate-rms: 2.828 <= 3.5 coordinate RMS / 49.2% repaired

pear-1.fillpassed

expected #a3e635 / produced #a3e635

color-lab: 0 <= 18 Delta E76 / 100.0% repaired

right-awning-blue-2-dup.existspassed

expected false / produced false

required presence matches

bike-wheel-left.stroke-widthpassed

expected 2 / produced 2

line-weight-delta: 0 <= 1 user units / 100.0% repaired

bike-frame.stroke-widthpassed

expected 2 / produced 2

line-weight-delta: 0 <= 0.75 user units / 100.0% repaired

Specification outcome

All required gates passed.

Unexpected changes

None

Semantic preservation failures

None

Source fidelity diagnostics

None

prompt 3,068 tokoutput 3,249 tokreasoning 430 tokceiling 4,225 tokfinish stopopen weight

resolved minimax/minimax-m3

response gen-1784548449-3j8PxYFYQ7fnYArgY2uA

expected diff5 edits

Part	Attr	Before	After
bike-handlebar	d	M122 96 C128 100 134 84 140 94	M122 96 C130 88 136 92 136 98
pear-1	fill	#dc2626	#a3e635
right-awning-blue-2-dup	exists	true	false
bike-wheel-left	stroke-width	4	2
bike-frame	stroke-width	0.5	2

partsscope

Target (5)

bike-handlebar

pear-1

right-awning-blue-2-dup

bike-wheel-left

bike-frame

Should preserve (62)

back-wall

street

left-stall-counter

left-awning-base

left-awning-red-1

left-awning-red-2

left-awning-red-3

left-awning-scallop

left-post

left-post-right

right-stall-counter

right-awning-base

right-awning-blue-1

right-awning-blue-2

right-awning-blue-3

right-post-left

right-post-right

apple-1

apple-2

apple-3

pear-2

pear-3

crate-apples

crate-pears

hanging-sign

sign-pin-left

sign-pin-right

lamp

bike-wheel-right

shopper-head

shopper-body

shopper-bag

skin-12-rug-wide

skin-12-rug-wide-a

skin-12-rug-wide-b

skin-12-rug-wide-c

identity-12-rug-base

identity-12-rug-band-a

identity-12-rug-band-b

identity-12-rug-band-c

variant-12-dot

variant-12-rule

variant-12-chip-01

variant-12-chip-02

variant-12-chip-03

variant-12-chip-04

variant-12-chip-05

variant-12-chip-06

variant-12-chip-07

variant-12-chip-08

variant-12-chip-09

variant-12-chip-10

variant-12-chip-11

variant-12-chip-12

variant-12-pin-01

variant-12-pin-02

variant-12-pin-03

variant-12-pin-04

variant-12-pin-05

variant-12-pin-06

variant-12-pin-07

variant-12-pin-08

initial svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 220 140" width="220" height="140">
<rect id="__bg" x="0" y="0" width="220" height="140" fill="#fff7fb" />
  <rect id="back-wall" fill="#fed7aa" x="0" y="0" width="220" height="82" />
  <rect id="street" fill="#fde68a" x="0" y="82" width="220" height="58" />
  <rect id="left-stall-counter" fill="#7c2d12" stroke="#451a03" stroke-width="1" x="18" y="76" width="74" height="34" />
  <rect id="left-awning-base" fill="#f8fafc" stroke="#222" stroke-width="1" x="14" y="42" width="82" height="18" />
  <rect id="left-awning-red-1" fill="#be123c" x="14" y="42" width="14" height="18" />
  <rect id="left-awning-red-2" fill="#be123c" x="42" y="42" width="14" height="18" />
  <rect id="left-awning-red-3" fill="#be123c" x="70" y="42" width="14" height="18" />
  <path id="left-awning-scallop" fill="none" stroke="#222" stroke-width="1" d="M14 60 C22 68 30 52 38 60 S54 68 62 60 S78 52 86 60 S94 68 96 60" />
  <line id="left-post" stroke="#451a03" stroke-width="3" x1="22" y1="60" x2="22" y2="118" />
  <line id="left-post-right" stroke="#451a03" stroke-width="3" x1="88" y1="60" x2="88" y2="118" />
  <rect id="right-stall-counter" fill="#7c2d12" stroke="#451a03" stroke-width="1" x="128" y="78" width="72" height="32" />
  <rect id="right-awning-base" fill="#f8fafc" stroke="#222" stroke-width="1" x="124" y="44" width="80" height="18" />
  <rect id="right-awning-blue-1" fill="#0284c7" x="124" y="44" width="16" height="18" />
  <rect id="right-awning-blue-2" fill="#0284c7" x="156" y="44" width="16" height="18" />
  <rect id="right-awning-blue-3" fill="#0284c7" x="188" y="44" width="16" height="18" />
  <line id="right-post-left" stroke="#451a03" stroke-width="3" x1="134" y1="62" x2="134" y2="118" />
  <line id="right-post-right" stroke="#451a03" stroke-width="3" x1="196" y1="62" x2="196" y2="118" />
  <circle id="apple-1" fill="#dc2626" cx="34" cy="72" r="5" />
  <circle id="apple-2" fill="#dc2626" cx="48" cy="70" r="5" />
  <circle id="apple-3" fill="#dc2626" cx="62" cy="72" r="5" />
  <ellipse id="pear-1" fill="#dc2626" cx="148" cy="72" rx="5" ry="7" />
  <ellipse id="pear-2" fill="#a3e635" cx="164" cy="70" rx="5" ry="7" />
  <ellipse id="pear-3" fill="#a3e635" cx="180" cy="72" rx="5" ry="7" />
  <rect id="crate-apples" fill="#a16207" stroke="#451a03" stroke-width="1" x="28" y="76" width="44" height="12" />
  <rect id="crate-pears" fill="#a16207" stroke="#451a03" stroke-width="1" x="140" y="76" width="48" height="12" />
  <rect id="hanging-sign" fill="#fff7ed" stroke="#222" stroke-width="1" x="92" y="18" width="36" height="18" />
  <line id="sign-pin-left" stroke="#222" stroke-width="1" x1="100" y1="18" x2="100" y2="8" />
  <line id="sign-pin-right" stroke="#222" stroke-width="1" x1="120" y1="18" x2="120" y2="8" />
  <circle id="lamp" fill="#facc15" stroke="#222" stroke-width="1" cx="110" cy="10" r="5" />
  <polyline id="bike-frame" fill="none" stroke="#334155" stroke-width="0.5" points="84,114 104,96 122,114 96,114 104,96" />
  <path id="bike-handlebar" fill="none" stroke="#334155" stroke-width="2" d="M122 96 C128 100 134 84 140 94" />
  <circle id="bike-wheel-left" fill="none" stroke="#334155" stroke-width="4" cx="84" cy="114" r="10" />
  <circle id="bike-wheel-right" fill="none" stroke="#334155" stroke-width="2" cx="122" cy="114" r="10" />
  <circle id="shopper-head" fill="#f2c078" cx="112" cy="78" r="4" />
  <rect id="shopper-body" fill="#2563eb" x="108" y="82" width="8" height="18" />
  <rect id="shopper-bag" fill="#f59e0b" x="118" y="94" width="8" height="10" />
  <rect id="skin-12-rug-wide" fill="#f97316" opacity="0.82" x="78" y="105" width="76" height="31" rx="3" />
  <rect id="skin-12-rug-wide-a" fill="#fef3c7" opacity="0.9" x="90" y="105" width="10" height="31" />
  <rect id="skin-12-rug-wide-b" fill="#fef3c7" opacity="0.9" x="123" y="105" width="10" height="31" />
  <rect id="skin-12-rug-wide-c" fill="#fef3c7" opacity="0.9" x="146" y="105" width="8" height="31" />
  <rect id="identity-12-rug-base" fill="#f97316" opacity="0.85" x="86" y="114" width="58" height="17" rx="2" />
  <rect id="identity-12-rug-band-a" fill="#fef3c7" opacity="0.9" x="92" y="114" width="7" height="17" />
  <rect id="identity-12-rug-band-b" fill="#fef3c7" opacity="0.9" x="116" y="114" width="7" height="17" />
  <rect id="identity-12-rug-band-c" fill="#fef3c7" opacity="0.9" x="137" y="114" width="7" height="17" />
  <circle id="variant-12-dot" fill="#ec4899" opacity="0.72" cx="22" cy="30" r="2.2" />
  <line id="variant-12-rule" stroke="#ec4899" stroke-width="1.2" opacity="0.58" x1="28" y1="33" x2="44" y2="36" />
  <rect id="variant-12-chip-01" fill="#ef4444" opacity="0.68" x="8" y="4" width="7" height="3" rx="1" />
  <rect id="variant-12-chip-02" fill="#f97316" opacity="0.68" x="45" y="5" width="8" height="4" rx="1" />
  <rect id="variant-12-chip-03" fill="#f59e0b" opacity="0.68" x="76" y="6" width="9" height="3" rx="1" />
  <rect id="variant-12-chip-04" fill="#84cc16" opacity="0.68" x="113" y="4" width="7" height="4" rx="1" />
  <rect id="variant-12-chip-05" fill="#22c55e" opacity="0.68" x="144" y="5" width="8" height="3" rx="1" />
  <rect id="variant-12-chip-06" fill="#14b8a6" opacity="0.68" x="181" y="6" width="9" height="4" rx="1" />
  <rect id="variant-12-chip-07" fill="#06b6d4" opacity="0.68" x="8" y="141" width="7" height="3" rx="1" />
  <rect id="variant-12-chip-08" fill="#3b82f6" opacity="0.68" x="45" y="140" width="8" height="4" rx="1" />
  <rect id="variant-12-chip-09" fill="#6366f1" opacity="0.68" x="76" y="139" width="9" height="3" rx="1" />
  <rect id="variant-12-chip-10" fill="#8b5cf6" opacity="0.68" x="113" y="141" width="7" height="4" rx="1" />
  <rect id="variant-12-chip-11" fill="#d946ef" opacity="0.68" x="144" y="140" width="8" height="3" rx="1" />
  <rect id="variant-12-chip-12" fill="#ec4899" opacity="0.68" x="181" y="139" width="9" height="4" rx="1" />
  <circle id="variant-12-pin-01" fill="#ef4444" opacity="0.66" cx="20" cy="24" r="1.8" />
  <circle id="variant-12-pin-02" fill="#f97316" opacity="0.66" cx="193" cy="38" r="2.05" />
  <circle id="variant-12-pin-03" fill="#f59e0b" opacity="0.66" cx="30" cy="52" r="2.3" />
  <circle id="variant-12-pin-04" fill="#84cc16" opacity="0.66" cx="203" cy="66" r="2.55" />
  <circle id="variant-12-pin-05" fill="#22c55e" opacity="0.66" cx="16" cy="80" r="1.8" />
  <circle id="variant-12-pin-06" fill="#14b8a6" opacity="0.66" cx="189" cy="94" r="2.05" />
  <circle id="variant-12-pin-07" fill="#06b6d4" opacity="0.66" cx="26" cy="108" r="2.3" />
  <circle id="variant-12-pin-08" fill="#3b82f6" opacity="0.66" cx="199" cy="122" r="2.55" />
  <rect id="right-awning-blue-2-dup" fill="#0284c7" x="164" y="44" width="16" height="18" />
</svg>

target svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 220 140" width="220" height="140">
<rect id="__bg" x="0" y="0" width="220" height="140" fill="#fff7fb" />
  <rect id="back-wall" fill="#fed7aa" x="0" y="0" width="220" height="82" />
  <rect id="street" fill="#fde68a" x="0" y="82" width="220" height="58" />
  <rect id="left-stall-counter" fill="#7c2d12" stroke="#451a03" stroke-width="1" x="18" y="76" width="74" height="34" />
  <rect id="left-awning-base" fill="#f8fafc" stroke="#222" stroke-width="1" x="14" y="42" width="82" height="18" />
  <rect id="left-awning-red-1" fill="#be123c" x="14" y="42" width="14" height="18" />
  <rect id="left-awning-red-2" fill="#be123c" x="42" y="42" width="14" height="18" />
  <rect id="left-awning-red-3" fill="#be123c" x="70" y="42" width="14" height="18" />
  <path id="left-awning-scallop" fill="none" stroke="#222" stroke-width="1" d="M14 60 C22 68 30 52 38 60 S54 68 62 60 S78 52 86 60 S94 68 96 60" />
  <line id="left-post" stroke="#451a03" stroke-width="3" x1="22" y1="60" x2="22" y2="118" />
  <line id="left-post-right" stroke="#451a03" stroke-width="3" x1="88" y1="60" x2="88" y2="118" />
  <rect id="right-stall-counter" fill="#7c2d12" stroke="#451a03" stroke-width="1" x="128" y="78" width="72" height="32" />
  <rect id="right-awning-base" fill="#f8fafc" stroke="#222" stroke-width="1" x="124" y="44" width="80" height="18" />
  <rect id="right-awning-blue-1" fill="#0284c7" x="124" y="44" width="16" height="18" />
  <rect id="right-awning-blue-2" fill="#0284c7" x="156" y="44" width="16" height="18" />
  <rect id="right-awning-blue-3" fill="#0284c7" x="188" y="44" width="16" height="18" />
  <line id="right-post-left" stroke="#451a03" stroke-width="3" x1="134" y1="62" x2="134" y2="118" />
  <line id="right-post-right" stroke="#451a03" stroke-width="3" x1="196" y1="62" x2="196" y2="118" />
  <circle id="apple-1" fill="#dc2626" cx="34" cy="72" r="5" />
  <circle id="apple-2" fill="#dc2626" cx="48" cy="70" r="5" />
  <circle id="apple-3" fill="#dc2626" cx="62" cy="72" r="5" />
  <ellipse id="pear-1" fill="#a3e635" cx="148" cy="72" rx="5" ry="7" />
  <ellipse id="pear-2" fill="#a3e635" cx="164" cy="70" rx="5" ry="7" />
  <ellipse id="pear-3" fill="#a3e635" cx="180" cy="72" rx="5" ry="7" />
  <rect id="crate-apples" fill="#a16207" stroke="#451a03" stroke-width="1" x="28" y="76" width="44" height="12" />
  <rect id="crate-pears" fill="#a16207" stroke="#451a03" stroke-width="1" x="140" y="76" width="48" height="12" />
  <rect id="hanging-sign" fill="#fff7ed" stroke="#222" stroke-width="1" x="92" y="18" width="36" height="18" />
  <line id="sign-pin-left" stroke="#222" stroke-width="1" x1="100" y1="18" x2="100" y2="8" />
  <line id="sign-pin-right" stroke="#222" stroke-width="1" x1="120" y1="18" x2="120" y2="8" />
  <circle id="lamp" fill="#facc15" stroke="#222" stroke-width="1" cx="110" cy="10" r="5" />
  <polyline id="bike-frame" fill="none" stroke="#334155" stroke-width="2" points="84,114 104,96 122,114 96,114 104,96" />
  <path id="bike-handlebar" fill="none" stroke="#334155" stroke-width="2" d="M122 96 C130 88 136 92 136 98" />
  <circle id="bike-wheel-left" fill="none" stroke="#334155" stroke-width="2" cx="84" cy="114" r="10" />
  <circle id="bike-wheel-right" fill="none" stroke="#334155" stroke-width="2" cx="122" cy="114" r="10" />
  <circle id="shopper-head" fill="#f2c078" cx="112" cy="78" r="4" />
  <rect id="shopper-body" fill="#2563eb" x="108" y="82" width="8" height="18" />
  <rect id="shopper-bag" fill="#f59e0b" x="118" y="94" width="8" height="10" />
  <rect id="skin-12-rug-wide" fill="#f97316" opacity="0.82" x="78" y="105" width="76" height="31" rx="3" />
  <rect id="skin-12-rug-wide-a" fill="#fef3c7" opacity="0.9" x="90" y="105" width="10" height="31" />
  <rect id="skin-12-rug-wide-b" fill="#fef3c7" opacity="0.9" x="123" y="105" width="10" height="31" />
  <rect id="skin-12-rug-wide-c" fill="#fef3c7" opacity="0.9" x="146" y="105" width="8" height="31" />
  <rect id="identity-12-rug-base" fill="#f97316" opacity="0.85" x="86" y="114" width="58" height="17" rx="2" />
  <rect id="identity-12-rug-band-a" fill="#fef3c7" opacity="0.9" x="92" y="114" width="7" height="17" />
  <rect id="identity-12-rug-band-b" fill="#fef3c7" opacity="0.9" x="116" y="114" width="7" height="17" />
  <rect id="identity-12-rug-band-c" fill="#fef3c7" opacity="0.9" x="137" y="114" width="7" height="17" />
  <circle id="variant-12-dot" fill="#ec4899" opacity="0.72" cx="22" cy="30" r="2.2" />
  <line id="variant-12-rule" stroke="#ec4899" stroke-width="1.2" opacity="0.58" x1="28" y1="33" x2="44" y2="36" />
  <rect id="variant-12-chip-01" fill="#ef4444" opacity="0.68" x="8" y="4" width="7" height="3" rx="1" />
  <rect id="variant-12-chip-02" fill="#f97316" opacity="0.68" x="45" y="5" width="8" height="4" rx="1" />
  <rect id="variant-12-chip-03" fill="#f59e0b" opacity="0.68" x="76" y="6" width="9" height="3" rx="1" />
  <rect id="variant-12-chip-04" fill="#84cc16" opacity="0.68" x="113" y="4" width="7" height="4" rx="1" />
  <rect id="variant-12-chip-05" fill="#22c55e" opacity="0.68" x="144" y="5" width="8" height="3" rx="1" />
  <rect id="variant-12-chip-06" fill="#14b8a6" opacity="0.68" x="181" y="6" width="9" height="4" rx="1" />
  <rect id="variant-12-chip-07" fill="#06b6d4" opacity="0.68" x="8" y="141" width="7" height="3" rx="1" />
  <rect id="variant-12-chip-08" fill="#3b82f6" opacity="0.68" x="45" y="140" width="8" height="4" rx="1" />
  <rect id="variant-12-chip-09" fill="#6366f1" opacity="0.68" x="76" y="139" width="9" height="3" rx="1" />
  <rect id="variant-12-chip-10" fill="#8b5cf6" opacity="0.68" x="113" y="141" width="7" height="4" rx="1" />
  <rect id="variant-12-chip-11" fill="#d946ef" opacity="0.68" x="144" y="140" width="8" height="3" rx="1" />
  <rect id="variant-12-chip-12" fill="#ec4899" opacity="0.68" x="181" y="139" width="9" height="4" rx="1" />
  <circle id="variant-12-pin-01" fill="#ef4444" opacity="0.66" cx="20" cy="24" r="1.8" />
  <circle id="variant-12-pin-02" fill="#f97316" opacity="0.66" cx="193" cy="38" r="2.05" />
  <circle id="variant-12-pin-03" fill="#f59e0b" opacity="0.66" cx="30" cy="52" r="2.3" />
  <circle id="variant-12-pin-04" fill="#84cc16" opacity="0.66" cx="203" cy="66" r="2.55" />
  <circle id="variant-12-pin-05" fill="#22c55e" opacity="0.66" cx="16" cy="80" r="1.8" />
  <circle id="variant-12-pin-06" fill="#14b8a6" opacity="0.66" cx="189" cy="94" r="2.05" />
  <circle id="variant-12-pin-07" fill="#06b6d4" opacity="0.66" cx="26" cy="108" r="2.3" />
  <circle id="variant-12-pin-08" fill="#3b82f6" opacity="0.66" cx="199" cy="122" r="2.55" />
</svg>