Vector-Bench · a thetalab benchmark

theta labs / task

sv_018very_hard

scenic_multi

instructionnatural language

One pear should be green, one apple should be red, and the shopper's bag should hang higher. Remove the copied blue awning panel, restore the scalloped edge on the left awning, and give the bicycle frame its normal line weight. Preserve the rest of the market.

initial · brokeninput

target · expected fixcanonical

published model outputs

What each model produced.

34 endpoints

model result explorerHY 3

HY 3

tencent/hy3

TraceNEAR

requested repairs

5/6

semantic preservation

clean

SVG validity

valid

source fidelity

unchanged

target match

different

Full pass requires the first three gates. Source fidelity and canonical target match are diagnostics only.

outcome

near

progress

81.5%

UCR

0.0%

preserve

100.0%

elapsed

3236.3s

cost

$0.0021

requested repairs5/6

apple-3.fillpassed

expected #dc2626 / produced #dc2626

color-lab: 0 <= 18 Delta E76 / 100.0% repaired

bike-frame.stroke-widthpassed

expected 2 / produced 2

line-weight-delta: 0 <= 0.75 user units / 100.0% repaired

left-awning-scallop.dmissed

expected M14 60 C22 68 30 52 38 60 S54 68 62 60 S78 52 86 60 S94 68 96 60 / produced M14 60 C20 70 32 50 38 62 S54 70 62 58 S78 50 86 62 S94 70 96 58

path-shape: 3.495 <= 2.446 sampled geometry / 0.0% repaired

right-awning-blue-2-dup.existspassed

expected false / produced false

required presence matches

shopper-bag.ypassed

expected 94 / produced 93

numeric-delta: 1 <= 3.5 user units / 88.9% repaired

pear-2.fillpassed

expected #a3e635 / produced #a3e635

color-lab: 0 <= 18 Delta E76 / 100.0% repaired

Why it did not pass

requested repairs incomplete

Unexpected changes

None

Semantic preservation failures

None

Source fidelity diagnostics

None

prompt 2,998 tokoutput 2,833 tokreasoning 0 tokceiling 4,295 tokfinish stopopen weight

resolved tencent/hy3

response gen-1784548109-DjCFkhs7Ovu0b0EREEJo

expected diff6 edits

Part	Attr	Before	After
apple-3	fill	#a3e635	#dc2626
bike-frame	stroke-width	0.5	2
left-awning-scallop	d	M14 60 C20 70 32 50 38 62 S54 70 62 58 S78 50 86 62 S94 70 96 58	M14 60 C22 68 30 52 38 60 S54 68 62 60 S78 52 86 60 S94 68 96 60
right-awning-blue-2-dup	exists	true	false
shopper-bag	y	103	94
pear-2	fill	#dc2626	#a3e635

partsscope

Target (6)

apple-3

bike-frame

left-awning-scallop

right-awning-blue-2-dup

shopper-bag

pear-2

Should preserve (63)

back-wall

street

left-stall-counter

left-awning-base

left-awning-red-1

left-awning-red-2

left-awning-red-3

left-post

left-post-right

right-stall-counter

right-awning-base

right-awning-blue-1

right-awning-blue-2

right-awning-blue-3

right-post-left

right-post-right

apple-1

apple-2

pear-1

pear-3

crate-apples

crate-pears

hanging-sign

sign-pin-left

sign-pin-right

lamp

bike-handlebar

bike-wheel-left

bike-wheel-right

shopper-head

shopper-body

skin-18-flower-cart-wide

skin-18-flower-wide-a

skin-18-flower-wide-b

skin-18-flower-wide-c

identity-18-flower-cart

identity-18-cart-wheel-a

identity-18-cart-wheel-b

identity-18-flower-a

identity-18-flower-b

identity-18-flower-c

variant-18-dot

variant-18-rule

variant-18-chip-01

variant-18-chip-02

variant-18-chip-03

variant-18-chip-04

variant-18-chip-05

variant-18-chip-06

variant-18-chip-07

variant-18-chip-08

variant-18-chip-09

variant-18-chip-10

variant-18-chip-11

variant-18-chip-12

variant-18-pin-01

variant-18-pin-02

variant-18-pin-03

variant-18-pin-04

variant-18-pin-05

variant-18-pin-06

variant-18-pin-07

variant-18-pin-08

initial svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 220 140" width="220" height="140">
<rect id="__bg" x="0" y="0" width="220" height="140" fill="#fffef5" />
  <rect id="back-wall" fill="#fce7f3" x="0" y="0" width="220" height="82" />
  <rect id="street" fill="#d1d5db" x="0" y="82" width="220" height="58" />
  <rect id="left-stall-counter" fill="#6b21a8" stroke="#451a03" stroke-width="1" x="18" y="76" width="74" height="34" />
  <rect id="left-awning-base" fill="#f8fafc" stroke="#222" stroke-width="1" x="14" y="42" width="82" height="18" />
  <rect id="left-awning-red-1" fill="#ec4899" x="14" y="42" width="14" height="18" />
  <rect id="left-awning-red-2" fill="#ec4899" x="42" y="42" width="14" height="18" />
  <rect id="left-awning-red-3" fill="#ec4899" x="70" y="42" width="14" height="18" />
  <path id="left-awning-scallop" fill="none" stroke="#222" stroke-width="1" d="M14 60 C20 70 32 50 38 62 S54 70 62 58 S78 50 86 62 S94 70 96 58" />
  <line id="left-post" stroke="#451a03" stroke-width="3" x1="22" y1="60" x2="22" y2="118" />
  <line id="left-post-right" stroke="#451a03" stroke-width="3" x1="88" y1="60" x2="88" y2="118" />
  <rect id="right-stall-counter" fill="#6b21a8" stroke="#451a03" stroke-width="1" x="128" y="78" width="72" height="32" />
  <rect id="right-awning-base" fill="#f8fafc" stroke="#222" stroke-width="1" x="124" y="44" width="80" height="18" />
  <rect id="right-awning-blue-1" fill="#84cc16" x="124" y="44" width="16" height="18" />
  <rect id="right-awning-blue-2" fill="#84cc16" x="156" y="44" width="16" height="18" />
  <rect id="right-awning-blue-3" fill="#84cc16" x="188" y="44" width="16" height="18" />
  <line id="right-post-left" stroke="#451a03" stroke-width="3" x1="134" y1="62" x2="134" y2="118" />
  <line id="right-post-right" stroke="#451a03" stroke-width="3" x1="196" y1="62" x2="196" y2="118" />
  <circle id="apple-1" fill="#dc2626" cx="34" cy="72" r="5" />
  <circle id="apple-2" fill="#dc2626" cx="48" cy="70" r="5" />
  <circle id="apple-3" fill="#a3e635" cx="62" cy="72" r="5" />
  <ellipse id="pear-1" fill="#a3e635" cx="148" cy="72" rx="5" ry="7" />
  <ellipse id="pear-2" fill="#dc2626" cx="164" cy="70" rx="5" ry="7" />
  <ellipse id="pear-3" fill="#a3e635" cx="180" cy="72" rx="5" ry="7" />
  <rect id="crate-apples" fill="#a16207" stroke="#451a03" stroke-width="1" x="28" y="76" width="44" height="12" />
  <rect id="crate-pears" fill="#a16207" stroke="#451a03" stroke-width="1" x="140" y="76" width="48" height="12" />
  <rect id="hanging-sign" fill="#fff7ed" stroke="#222" stroke-width="1" x="92" y="18" width="36" height="18" />
  <line id="sign-pin-left" stroke="#222" stroke-width="1" x1="100" y1="18" x2="100" y2="8" />
  <line id="sign-pin-right" stroke="#222" stroke-width="1" x1="120" y1="18" x2="120" y2="8" />
  <circle id="lamp" fill="#facc15" stroke="#222" stroke-width="1" cx="110" cy="10" r="5" />
  <polyline id="bike-frame" fill="none" stroke="#334155" stroke-width="0.5" points="84,114 104,96 122,114 96,114 104,96" />
  <path id="bike-handlebar" fill="none" stroke="#334155" stroke-width="2" d="M122 96 C130 88 136 92 136 98" />
  <circle id="bike-wheel-left" fill="none" stroke="#334155" stroke-width="2" cx="84" cy="114" r="10" />
  <circle id="bike-wheel-right" fill="none" stroke="#334155" stroke-width="2" cx="122" cy="114" r="10" />
  <circle id="shopper-head" fill="#f2c078" cx="112" cy="78" r="4" />
  <rect id="shopper-body" fill="#2563eb" x="108" y="82" width="8" height="18" />
  <rect id="shopper-bag" fill="#f59e0b" x="118" y="103" width="8" height="10" />
  <rect id="skin-18-flower-cart-wide" fill="#a855f7" stroke="#581c87" stroke-width="2" opacity="0.9" x="8" y="96" width="70" height="28" rx="3" />
  <circle id="skin-18-flower-wide-a" fill="#f472b6" opacity="0.95" cx="28" cy="88" r="10" />
  <circle id="skin-18-flower-wide-b" fill="#fde047" opacity="0.95" cx="48" cy="82" r="10" />
  <circle id="skin-18-flower-wide-c" fill="#22c55e" opacity="0.95" cx="68" cy="90" r="10" />
  <rect id="identity-18-flower-cart" fill="#a855f7" stroke="#581c87" stroke-width="1.3" x="20" y="103" width="42" height="17" rx="2" />
  <circle id="identity-18-cart-wheel-a" fill="none" stroke="#334155" stroke-width="1.6" cx="29" cy="123" r="5" />
  <circle id="identity-18-cart-wheel-b" fill="none" stroke="#334155" stroke-width="1.6" cx="54" cy="123" r="5" />
  <circle id="identity-18-flower-a" fill="#f472b6" cx="31" cy="99" r="5" />
  <circle id="identity-18-flower-b" fill="#fde047" cx="44" cy="96" r="5" />
  <circle id="identity-18-flower-c" fill="#22c55e" cx="56" cy="100" r="5" />
  <circle id="variant-18-dot" fill="#ca8a04" opacity="0.72" cx="28" cy="12" r="2.2" />
  <line id="variant-18-rule" stroke="#ca8a04" stroke-width="1.2" opacity="0.58" x1="34" y1="15" x2="50" y2="18" />
  <rect id="variant-18-chip-01" fill="#06b6d4" opacity="0.68" x="8" y="4" width="7" height="3" rx="1" />
  <rect id="variant-18-chip-02" fill="#3b82f6" opacity="0.68" x="45" y="5" width="8" height="4" rx="1" />
  <rect id="variant-18-chip-03" fill="#6366f1" opacity="0.68" x="76" y="6" width="9" height="3" rx="1" />
  <rect id="variant-18-chip-04" fill="#8b5cf6" opacity="0.68" x="113" y="4" width="7" height="4" rx="1" />
  <rect id="variant-18-chip-05" fill="#d946ef" opacity="0.68" x="144" y="5" width="8" height="3" rx="1" />
  <rect id="variant-18-chip-06" fill="#ec4899" opacity="0.68" x="181" y="6" width="9" height="4" rx="1" />
  <rect id="variant-18-chip-07" fill="#ef4444" opacity="0.68" x="8" y="141" width="7" height="3" rx="1" />
  <rect id="variant-18-chip-08" fill="#f97316" opacity="0.68" x="45" y="140" width="8" height="4" rx="1" />
  <rect id="variant-18-chip-09" fill="#f59e0b" opacity="0.68" x="76" y="139" width="9" height="3" rx="1" />
  <rect id="variant-18-chip-10" fill="#84cc16" opacity="0.68" x="113" y="141" width="7" height="4" rx="1" />
  <rect id="variant-18-chip-11" fill="#22c55e" opacity="0.68" x="144" y="140" width="8" height="3" rx="1" />
  <rect id="variant-18-chip-12" fill="#14b8a6" opacity="0.68" x="181" y="139" width="9" height="4" rx="1" />
  <circle id="variant-18-pin-01" fill="#ef4444" opacity="0.66" cx="14" cy="25" r="2.3" />
  <circle id="variant-18-pin-02" fill="#f97316" opacity="0.66" cx="211" cy="39" r="2.55" />
  <circle id="variant-18-pin-03" fill="#f59e0b" opacity="0.66" cx="24" cy="53" r="1.8" />
  <circle id="variant-18-pin-04" fill="#84cc16" opacity="0.66" cx="197" cy="67" r="2.05" />
  <circle id="variant-18-pin-05" fill="#22c55e" opacity="0.66" cx="10" cy="81" r="2.3" />
  <circle id="variant-18-pin-06" fill="#14b8a6" opacity="0.66" cx="207" cy="95" r="2.55" />
  <circle id="variant-18-pin-07" fill="#06b6d4" opacity="0.66" cx="20" cy="109" r="1.8" />
  <circle id="variant-18-pin-08" fill="#3b82f6" opacity="0.66" cx="193" cy="123" r="2.05" />
  <rect id="right-awning-blue-2-dup" fill="#84cc16" x="164" y="44" width="16" height="18" />
</svg>

target svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 220 140" width="220" height="140">
<rect id="__bg" x="0" y="0" width="220" height="140" fill="#fffef5" />
  <rect id="back-wall" fill="#fce7f3" x="0" y="0" width="220" height="82" />
  <rect id="street" fill="#d1d5db" x="0" y="82" width="220" height="58" />
  <rect id="left-stall-counter" fill="#6b21a8" stroke="#451a03" stroke-width="1" x="18" y="76" width="74" height="34" />
  <rect id="left-awning-base" fill="#f8fafc" stroke="#222" stroke-width="1" x="14" y="42" width="82" height="18" />
  <rect id="left-awning-red-1" fill="#ec4899" x="14" y="42" width="14" height="18" />
  <rect id="left-awning-red-2" fill="#ec4899" x="42" y="42" width="14" height="18" />
  <rect id="left-awning-red-3" fill="#ec4899" x="70" y="42" width="14" height="18" />
  <path id="left-awning-scallop" fill="none" stroke="#222" stroke-width="1" d="M14 60 C22 68 30 52 38 60 S54 68 62 60 S78 52 86 60 S94 68 96 60" />
  <line id="left-post" stroke="#451a03" stroke-width="3" x1="22" y1="60" x2="22" y2="118" />
  <line id="left-post-right" stroke="#451a03" stroke-width="3" x1="88" y1="60" x2="88" y2="118" />
  <rect id="right-stall-counter" fill="#6b21a8" stroke="#451a03" stroke-width="1" x="128" y="78" width="72" height="32" />
  <rect id="right-awning-base" fill="#f8fafc" stroke="#222" stroke-width="1" x="124" y="44" width="80" height="18" />
  <rect id="right-awning-blue-1" fill="#84cc16" x="124" y="44" width="16" height="18" />
  <rect id="right-awning-blue-2" fill="#84cc16" x="156" y="44" width="16" height="18" />
  <rect id="right-awning-blue-3" fill="#84cc16" x="188" y="44" width="16" height="18" />
  <line id="right-post-left" stroke="#451a03" stroke-width="3" x1="134" y1="62" x2="134" y2="118" />
  <line id="right-post-right" stroke="#451a03" stroke-width="3" x1="196" y1="62" x2="196" y2="118" />
  <circle id="apple-1" fill="#dc2626" cx="34" cy="72" r="5" />
  <circle id="apple-2" fill="#dc2626" cx="48" cy="70" r="5" />
  <circle id="apple-3" fill="#dc2626" cx="62" cy="72" r="5" />
  <ellipse id="pear-1" fill="#a3e635" cx="148" cy="72" rx="5" ry="7" />
  <ellipse id="pear-2" fill="#a3e635" cx="164" cy="70" rx="5" ry="7" />
  <ellipse id="pear-3" fill="#a3e635" cx="180" cy="72" rx="5" ry="7" />
  <rect id="crate-apples" fill="#a16207" stroke="#451a03" stroke-width="1" x="28" y="76" width="44" height="12" />
  <rect id="crate-pears" fill="#a16207" stroke="#451a03" stroke-width="1" x="140" y="76" width="48" height="12" />
  <rect id="hanging-sign" fill="#fff7ed" stroke="#222" stroke-width="1" x="92" y="18" width="36" height="18" />
  <line id="sign-pin-left" stroke="#222" stroke-width="1" x1="100" y1="18" x2="100" y2="8" />
  <line id="sign-pin-right" stroke="#222" stroke-width="1" x1="120" y1="18" x2="120" y2="8" />
  <circle id="lamp" fill="#facc15" stroke="#222" stroke-width="1" cx="110" cy="10" r="5" />
  <polyline id="bike-frame" fill="none" stroke="#334155" stroke-width="2" points="84,114 104,96 122,114 96,114 104,96" />
  <path id="bike-handlebar" fill="none" stroke="#334155" stroke-width="2" d="M122 96 C130 88 136 92 136 98" />
  <circle id="bike-wheel-left" fill="none" stroke="#334155" stroke-width="2" cx="84" cy="114" r="10" />
  <circle id="bike-wheel-right" fill="none" stroke="#334155" stroke-width="2" cx="122" cy="114" r="10" />
  <circle id="shopper-head" fill="#f2c078" cx="112" cy="78" r="4" />
  <rect id="shopper-body" fill="#2563eb" x="108" y="82" width="8" height="18" />
  <rect id="shopper-bag" fill="#f59e0b" x="118" y="94" width="8" height="10" />
  <rect id="skin-18-flower-cart-wide" fill="#a855f7" stroke="#581c87" stroke-width="2" opacity="0.9" x="8" y="96" width="70" height="28" rx="3" />
  <circle id="skin-18-flower-wide-a" fill="#f472b6" opacity="0.95" cx="28" cy="88" r="10" />
  <circle id="skin-18-flower-wide-b" fill="#fde047" opacity="0.95" cx="48" cy="82" r="10" />
  <circle id="skin-18-flower-wide-c" fill="#22c55e" opacity="0.95" cx="68" cy="90" r="10" />
  <rect id="identity-18-flower-cart" fill="#a855f7" stroke="#581c87" stroke-width="1.3" x="20" y="103" width="42" height="17" rx="2" />
  <circle id="identity-18-cart-wheel-a" fill="none" stroke="#334155" stroke-width="1.6" cx="29" cy="123" r="5" />
  <circle id="identity-18-cart-wheel-b" fill="none" stroke="#334155" stroke-width="1.6" cx="54" cy="123" r="5" />
  <circle id="identity-18-flower-a" fill="#f472b6" cx="31" cy="99" r="5" />
  <circle id="identity-18-flower-b" fill="#fde047" cx="44" cy="96" r="5" />
  <circle id="identity-18-flower-c" fill="#22c55e" cx="56" cy="100" r="5" />
  <circle id="variant-18-dot" fill="#ca8a04" opacity="0.72" cx="28" cy="12" r="2.2" />
  <line id="variant-18-rule" stroke="#ca8a04" stroke-width="1.2" opacity="0.58" x1="34" y1="15" x2="50" y2="18" />
  <rect id="variant-18-chip-01" fill="#06b6d4" opacity="0.68" x="8" y="4" width="7" height="3" rx="1" />
  <rect id="variant-18-chip-02" fill="#3b82f6" opacity="0.68" x="45" y="5" width="8" height="4" rx="1" />
  <rect id="variant-18-chip-03" fill="#6366f1" opacity="0.68" x="76" y="6" width="9" height="3" rx="1" />
  <rect id="variant-18-chip-04" fill="#8b5cf6" opacity="0.68" x="113" y="4" width="7" height="4" rx="1" />
  <rect id="variant-18-chip-05" fill="#d946ef" opacity="0.68" x="144" y="5" width="8" height="3" rx="1" />
  <rect id="variant-18-chip-06" fill="#ec4899" opacity="0.68" x="181" y="6" width="9" height="4" rx="1" />
  <rect id="variant-18-chip-07" fill="#ef4444" opacity="0.68" x="8" y="141" width="7" height="3" rx="1" />
  <rect id="variant-18-chip-08" fill="#f97316" opacity="0.68" x="45" y="140" width="8" height="4" rx="1" />
  <rect id="variant-18-chip-09" fill="#f59e0b" opacity="0.68" x="76" y="139" width="9" height="3" rx="1" />
  <rect id="variant-18-chip-10" fill="#84cc16" opacity="0.68" x="113" y="141" width="7" height="4" rx="1" />
  <rect id="variant-18-chip-11" fill="#22c55e" opacity="0.68" x="144" y="140" width="8" height="3" rx="1" />
  <rect id="variant-18-chip-12" fill="#14b8a6" opacity="0.68" x="181" y="139" width="9" height="4" rx="1" />
  <circle id="variant-18-pin-01" fill="#ef4444" opacity="0.66" cx="14" cy="25" r="2.3" />
  <circle id="variant-18-pin-02" fill="#f97316" opacity="0.66" cx="211" cy="39" r="2.55" />
  <circle id="variant-18-pin-03" fill="#f59e0b" opacity="0.66" cx="24" cy="53" r="1.8" />
  <circle id="variant-18-pin-04" fill="#84cc16" opacity="0.66" cx="197" cy="67" r="2.05" />
  <circle id="variant-18-pin-05" fill="#22c55e" opacity="0.66" cx="10" cy="81" r="2.3" />
  <circle id="variant-18-pin-06" fill="#14b8a6" opacity="0.66" cx="207" cy="95" r="2.55" />
  <circle id="variant-18-pin-07" fill="#06b6d4" opacity="0.66" cx="20" cy="109" r="1.8" />
  <circle id="variant-18-pin-08" fill="#3b82f6" opacity="0.66" cx="193" cy="123" r="2.05" />
</svg>