Vector-Bench · a thetalab benchmark

theta labs / task

sv_028very_hard

downloaded_scenic_surgical

instructionnatural language

Remove the colored marker from the reef and return the loose bubble to the small group at upper left. One fish in the school should be orange again, the turtle shell outline should be light rather than heavy, and the second underwater sunbeam should regain its brightness. Preserve the remaining coral and fish.

initial · brokeninput

target · expected fixcanonical

published model outputs

What each model produced.

34 endpoints

model result explorerQwen3.5 397B A17B

Qwen3.5 397B A17B

qwen/qwen3.5-397b-a17b

TracePARTIAL

requested repairs

4/5

semantic preservation

changed

SVG validity

valid

source fidelity

different

target match

different

Full pass requires the first three gates. Source fidelity and canonical target match are diagnostics only.

outcome

partial

progress

74.7%

UCR

0.0%

preserve

100.0%

elapsed

1295.4s

cost

$0.0131

requested repairs4/5

sv_028-extra-survey-marker.existspassed

expected false / produced false

required presence matches

download-sv_028-polygon-004.fillpassed

expected #ff8c1a / produced #ff8c1a

color-lab: 0 <= 18 Delta E76 / 100.0% repaired

download-sv_028-circle-001.cxmissed

expected 300 / produced 343

numeric-delta: 43 <= 21.5 user units / 0.0% repaired

download-sv_028-ellipse-002.stroke-widthpassed

expected 3 / produced 3

line-weight-delta: 0 <= 1.2 user units / 100.0% repaired

download-sv_028-polygon-002.opacitypassed

expected 0.7 / produced 0.6

opacity-delta: 0.1 <= 0.12 absolute opacity / 73.7% repaired

Why it did not pass

requested repairs incompleteunintended document changes

Unexpected changes

+download-sv_028-circle-013

Semantic preservation failures

None

Source fidelity diagnostics

None

prompt 4,215 tokoutput 4,908 tokreasoning 846 tokceiling 4,658 tokfinish stopopen weight

resolved qwen/qwen3.5-397b-a17b

response gen-1784545130-cKA7uS7DkrbaUn3P2LE3

expected diff5 edits

Part	Attr	Before	After
sv_028-extra-survey-marker	exists	true	false
download-sv_028-polygon-004	fill	#ec4899	#ff8c1a
download-sv_028-circle-001	cx	343	300
download-sv_028-ellipse-002	stroke-width	8	3
download-sv_028-polygon-002	opacity	0.32	0.7

partsscope

Target (5)

sv_028-extra-survey-marker

download-sv_028-polygon-004

download-sv_028-circle-001

download-sv_028-ellipse-002

download-sv_028-polygon-002

Should preserve (63)

download-sv_028-rect-001

download-sv_028-polygon-001

download-sv_028-polygon-003

download-sv_028-circle-002

download-sv_028-circle-003

download-sv_028-circle-004

download-sv_028-circle-005

download-sv_028-circle-006

download-sv_028-ellipse-001

download-sv_028-path-001

download-sv_028-path-002

download-sv_028-path-003

download-sv_028-path-004

download-sv_028-path-005

download-sv_028-ellipse-003

download-sv_028-circle-007

download-sv_028-path-006

download-sv_028-path-007

download-sv_028-path-008

download-sv_028-ellipse-004

download-sv_028-circle-008

download-sv_028-ellipse-005

download-sv_028-polygon-005

download-sv_028-circle-009

download-sv_028-ellipse-006

download-sv_028-polygon-006

download-sv_028-circle-010

download-sv_028-ellipse-007

download-sv_028-polygon-007

download-sv_028-ellipse-008

download-sv_028-polygon-008

download-sv_028-ellipse-009

download-sv_028-circle-011

download-sv_028-ellipse-010

download-sv_028-polygon-009

download-sv_028-rect-002

download-sv_028-rect-003

download-sv_028-circle-012

download-sv_028-path-009

download-sv_028-ellipse-011

download-sv_028-path-010

download-sv_028-path-011

download-sv_028-path-012

download-sv_028-path-013

download-sv_028-path-014

download-sv_028-path-015

download-sv_028-path-016

download-sv_028-ellipse-012

download-sv_028-ellipse-013

download-sv_028-ellipse-014

download-sv_028-path-017

download-sv_028-path-018

download-sv_028-ellipse-015

download-sv_028-line-001

download-sv_028-line-002

download-sv_028-line-003

download-sv_028-line-004

download-sv_028-line-005

download-sv_028-line-006

download-sv_028-line-007

download-sv_028-path-019

download-sv_028-path-020

download-sv_028-path-021

initial svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1600 1000">
  <defs>
    <linearGradient id="water08" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#7fe4f0" />
      <stop offset="0.4" stop-color="#33b8d8" />
      <stop offset="0.75" stop-color="#1a7aae" />
      <stop offset="1" stop-color="#0e4a78" />
    </linearGradient>
    <linearGradient id="ray08" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#eafdff" stop-opacity="0.6" /><stop offset="1" stop-color="#eafdff" stop-opacity="0" />
    </linearGradient>
    <linearGradient id="sand08" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#f0e2b0" /><stop offset="1" stop-color="#d8c488" />
    </linearGradient>
    <filter id="rayblur08" x="-60%" y="-60%" width="220%" height="220%"><feGaussianBlur stdDeviation="14" /></filter>
  </defs>

  <rect width="1600" height="1000" fill="url(#water08)" id="download-sv_028-rect-001" />

  
  <g filter="url(#rayblur08)">
    <polygon points="200,-40 280,-40 480,800 120,800" fill="url(#ray08)" id="download-sv_028-polygon-001" />
    <polygon points="640,-40 720,-40 860,800 540,800" fill="url(#ray08)" opacity="0.32" id="download-sv_028-polygon-002" />
    <polygon points="1120,-40 1220,-40 1320,800 1020,800" fill="url(#ray08)" opacity="0.6" id="download-sv_028-polygon-003" />
  </g>

  
  <g fill="none" stroke="#eafdff" stroke-width="1.5" opacity="0.6">
    <circle cx="343" cy="300" r="6" id="download-sv_028-circle-001" /><circle cx="320" cy="260" r="4" id="download-sv_028-circle-002" /><circle cx="312" cy="220" r="7" id="download-sv_028-circle-003" />
    <circle cx="1200" cy="360" r="5" id="download-sv_028-circle-004" /><circle cx="1216" cy="320" r="3" id="download-sv_028-circle-005" /><circle cx="1208" cy="280" r="6" id="download-sv_028-circle-006" />
  </g>

  
  <g transform="translate(560 360)">
    <ellipse cx="0" cy="0" rx="86" ry="64" fill="#3a6a4a" id="download-sv_028-ellipse-001" />
    <ellipse cx="0" cy="0" rx="86" ry="64" fill="none" stroke="#27543a" stroke-width="8" id="download-sv_028-ellipse-002" />
    <g stroke="#1f4630" stroke-width="2.5" fill="#4a7a56">
      <path d="M-30 -40 L-10 -10 L-40 0 Z" id="download-sv_028-path-001" /><path d="M30 -40 L10 -10 L40 0 Z" id="download-sv_028-path-002" />
      <path d="M-40 10 L-12 8 L-26 44 Z" id="download-sv_028-path-003" /><path d="M40 10 L12 8 L26 44 Z" id="download-sv_028-path-004" /><path d="M-12 -8 L12 -8 L8 24 L-8 24 Z" id="download-sv_028-path-005" />
    </g>
    <ellipse cx="-84" cy="-6" rx="26" ry="20" fill="#3a6a4a" id="download-sv_028-ellipse-003" />
    <circle cx="-96" cy="-10" r="4" fill="#1a2a1a" id="download-sv_028-circle-007" />
    <path d="M70 -40 Q110 -60 120 -30 Q90 -34 70 -20 Z" fill="#3a6a4a" id="download-sv_028-path-006" />
    <path d="M70 40 Q110 60 120 30 Q90 34 70 20 Z" fill="#3a6a4a" id="download-sv_028-path-007" />
    <path d="M-40 60 Q-30 96 -64 92 Q-52 70 -50 56 Z" fill="#3a6a4a" id="download-sv_028-path-008" />
  </g>

  
  <g fill="#ffb13c">
    <g transform="translate(960 260)"><ellipse rx="22" ry="11" id="download-sv_028-ellipse-004" /><polygon points="-20,0 -36,-10 -36,10" fill="#ec4899" id="download-sv_028-polygon-004" /><circle cx="12" cy="-2" r="2.5" fill="#1a2a1a" id="download-sv_028-circle-008" /></g>
    <g transform="translate(1010 290)"><ellipse rx="18" ry="9" id="download-sv_028-ellipse-005" /><polygon points="-16,0 -30,-8 -30,8" fill="#ff8c1a" id="download-sv_028-polygon-005" /><circle cx="10" cy="-2" r="2" fill="#1a2a1a" id="download-sv_028-circle-009" /></g>
    <g transform="translate(1050 250)"><ellipse rx="20" ry="10" id="download-sv_028-ellipse-006" /><polygon points="-18,0 -32,-9 -32,9" fill="#ff8c1a" id="download-sv_028-polygon-006" /><circle cx="11" cy="-2" r="2" fill="#1a2a1a" id="download-sv_028-circle-010" /></g>
    <g transform="translate(1000 340)"><ellipse rx="16" ry="8" id="download-sv_028-ellipse-007" /><polygon points="-14,0 -26,-7 -26,7" fill="#ff8c1a" id="download-sv_028-polygon-007" /></g>
  </g>
  
  <g transform="translate(420 540)">
    <ellipse rx="30" ry="20" fill="#2a7adf" id="download-sv_028-ellipse-008" /><polygon points="-28,0 -48,-16 -48,16" fill="#1a4aaf" id="download-sv_028-polygon-008" />
    <ellipse rx="30" ry="20" fill="none" stroke="#ffe14d" stroke-width="3" id="download-sv_028-ellipse-009" /><circle cx="16" cy="-4" r="3" fill="#1a2a1a" id="download-sv_028-circle-011" />
  </g>
  
  <g transform="translate(1180 700)">
    <ellipse rx="20" ry="12" fill="#ff7a1a" id="download-sv_028-ellipse-010" /><polygon points="-18,0 -32,-9 -32,9" fill="#e85a0a" id="download-sv_028-polygon-009" />
    <rect x="-4" y="-12" width="6" height="24" fill="#fff" id="download-sv_028-rect-002" /><rect x="-14" y="-10" width="5" height="20" fill="#fff" id="download-sv_028-rect-003" /><circle cx="11" cy="-2" r="2.5" fill="#1a2a1a" id="download-sv_028-circle-012" />
  </g>

  
  <path d="M0 1000 L0 820 Q300 780 600 820 Q900 860 1200 810 Q1400 780 1600 820 L1600 1000 Z" fill="url(#sand08)" id="download-sv_028-path-009" />

  
  <g>
    
    <ellipse cx="200" cy="830" rx="90" ry="50" fill="#e07a9a" id="download-sv_028-ellipse-011" />
    <g stroke="#b85578" stroke-width="3" fill="none"><path d="M140 830 Q200 800 260 830" id="download-sv_028-path-010" /><path d="M150 850 Q200 824 250 850" id="download-sv_028-path-011" /><path d="M160 815 Q200 800 240 815" id="download-sv_028-path-012" /></g>
    
    <g fill="#ff8c5a">
      <path d="M420 880 q-10 -90 10 -120 q10 30 0 120 Z" id="download-sv_028-path-013" />
      <path d="M440 880 q-4 -120 30 -150 q-20 50 -10 150 Z" id="download-sv_028-path-014" />
      <path d="M400 880 q-14 -80 -10 -110 q24 30 24 110 Z" id="download-sv_028-path-015" />
    </g>
    
    <g transform="translate(1080 840)" fill="none" stroke="#9a5ad0" stroke-width="4">
      <path d="M0 0 Q-40 -80 -20 -160 M0 0 Q-10 -90 10 -170 M0 0 Q40 -80 30 -160 M0 0 Q-60 -60 -70 -120 M0 0 Q60 -60 70 -120" id="download-sv_028-path-016" />
    </g>
    
    <g fill="#f0c84a"><ellipse cx="700" cy="850" rx="14" ry="44" id="download-sv_028-ellipse-012" /><ellipse cx="730" cy="858" rx="12" ry="36" id="download-sv_028-ellipse-013" /><ellipse cx="676" cy="858" rx="11" ry="32" id="download-sv_028-ellipse-014" /></g>
    
    <g fill="#3ac88a"><path d="M1350 880 q-12 -90 8 -130 q14 40 6 130 Z" id="download-sv_028-path-017" /><path d="M1380 880 q-6 -100 26 -130 q-14 50 -8 130 Z" id="download-sv_028-path-018" /></g>
  </g>

  
  <g transform="translate(1180 840)">
    <ellipse rx="40" ry="20" fill="#c84a7a" id="download-sv_028-ellipse-015" />
    <g stroke="#e07aa0" stroke-width="4" stroke-linecap="round">
      <line x1="-30" y1="0" x2="-36" y2="-30" id="download-sv_028-line-001" /><line x1="-15" y1="0" x2="-18" y2="-40" id="download-sv_028-line-002" /><line x1="0" y1="0" x2="0" y2="-44" id="download-sv_028-line-003" />
      <line x1="15" y1="0" x2="18" y2="-40" id="download-sv_028-line-004" /><line x1="30" y1="0" x2="36" y2="-30" id="download-sv_028-line-005" /><line x1="-22" y1="0" x2="-28" y2="-36" id="download-sv_028-line-006" /><line x1="22" y1="0" x2="28" y2="-36" id="download-sv_028-line-007" />
    </g>
  </g>

  
  <g stroke="#2a8a5a" stroke-width="6" fill="none" stroke-linecap="round" opacity="0.85">
    <path d="M900 880 Q880 800 910 740 Q940 680 920 620" id="download-sv_028-path-019" />
    <path d="M930 880 Q950 810 925 760 Q905 700 930 640" id="download-sv_028-path-020" />
  </g>

  
  <g transform="translate(640 900)" fill="#ff6a4a">
    <path d="M0 -26 L8 -8 L26 -8 L12 4 L18 24 L0 12 L-18 24 L-12 4 L-26 -8 L-8 -8 Z" id="download-sv_028-path-021" />
  </g>
<circle id="sv_028-extra-survey-marker" cx="1360" cy="234" r="41" fill="#ec4899" opacity="0.82" stroke="#111827" stroke-width="5" /></svg>

target svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1600 1000">
  <defs>
    <linearGradient id="water08" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#7fe4f0" />
      <stop offset="0.4" stop-color="#33b8d8" />
      <stop offset="0.75" stop-color="#1a7aae" />
      <stop offset="1" stop-color="#0e4a78" />
    </linearGradient>
    <linearGradient id="ray08" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#eafdff" stop-opacity="0.6" /><stop offset="1" stop-color="#eafdff" stop-opacity="0" />
    </linearGradient>
    <linearGradient id="sand08" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#f0e2b0" /><stop offset="1" stop-color="#d8c488" />
    </linearGradient>
    <filter id="rayblur08" x="-60%" y="-60%" width="220%" height="220%"><feGaussianBlur stdDeviation="14" /></filter>
  </defs>

  <rect width="1600" height="1000" fill="url(#water08)" id="download-sv_028-rect-001" />

  
  <g filter="url(#rayblur08)">
    <polygon points="200,-40 280,-40 480,800 120,800" fill="url(#ray08)" id="download-sv_028-polygon-001" />
    <polygon points="640,-40 720,-40 860,800 540,800" fill="url(#ray08)" opacity="0.7" id="download-sv_028-polygon-002" />
    <polygon points="1120,-40 1220,-40 1320,800 1020,800" fill="url(#ray08)" opacity="0.6" id="download-sv_028-polygon-003" />
  </g>

  
  <g fill="none" stroke="#eafdff" stroke-width="1.5" opacity="0.6">
    <circle cx="300" cy="300" r="6" id="download-sv_028-circle-001" /><circle cx="320" cy="260" r="4" id="download-sv_028-circle-002" /><circle cx="312" cy="220" r="7" id="download-sv_028-circle-003" />
    <circle cx="1200" cy="360" r="5" id="download-sv_028-circle-004" /><circle cx="1216" cy="320" r="3" id="download-sv_028-circle-005" /><circle cx="1208" cy="280" r="6" id="download-sv_028-circle-006" />
  </g>

  
  <g transform="translate(560 360)">
    <ellipse cx="0" cy="0" rx="86" ry="64" fill="#3a6a4a" id="download-sv_028-ellipse-001" />
    <ellipse cx="0" cy="0" rx="86" ry="64" fill="none" stroke="#27543a" stroke-width="3" id="download-sv_028-ellipse-002" />
    <g stroke="#1f4630" stroke-width="2.5" fill="#4a7a56">
      <path d="M-30 -40 L-10 -10 L-40 0 Z" id="download-sv_028-path-001" /><path d="M30 -40 L10 -10 L40 0 Z" id="download-sv_028-path-002" />
      <path d="M-40 10 L-12 8 L-26 44 Z" id="download-sv_028-path-003" /><path d="M40 10 L12 8 L26 44 Z" id="download-sv_028-path-004" /><path d="M-12 -8 L12 -8 L8 24 L-8 24 Z" id="download-sv_028-path-005" />
    </g>
    <ellipse cx="-84" cy="-6" rx="26" ry="20" fill="#3a6a4a" id="download-sv_028-ellipse-003" />
    <circle cx="-96" cy="-10" r="4" fill="#1a2a1a" id="download-sv_028-circle-007" />
    <path d="M70 -40 Q110 -60 120 -30 Q90 -34 70 -20 Z" fill="#3a6a4a" id="download-sv_028-path-006" />
    <path d="M70 40 Q110 60 120 30 Q90 34 70 20 Z" fill="#3a6a4a" id="download-sv_028-path-007" />
    <path d="M-40 60 Q-30 96 -64 92 Q-52 70 -50 56 Z" fill="#3a6a4a" id="download-sv_028-path-008" />
  </g>

  
  <g fill="#ffb13c">
    <g transform="translate(960 260)"><ellipse rx="22" ry="11" id="download-sv_028-ellipse-004" /><polygon points="-20,0 -36,-10 -36,10" fill="#ff8c1a" id="download-sv_028-polygon-004" /><circle cx="12" cy="-2" r="2.5" fill="#1a2a1a" id="download-sv_028-circle-008" /></g>
    <g transform="translate(1010 290)"><ellipse rx="18" ry="9" id="download-sv_028-ellipse-005" /><polygon points="-16,0 -30,-8 -30,8" fill="#ff8c1a" id="download-sv_028-polygon-005" /><circle cx="10" cy="-2" r="2" fill="#1a2a1a" id="download-sv_028-circle-009" /></g>
    <g transform="translate(1050 250)"><ellipse rx="20" ry="10" id="download-sv_028-ellipse-006" /><polygon points="-18,0 -32,-9 -32,9" fill="#ff8c1a" id="download-sv_028-polygon-006" /><circle cx="11" cy="-2" r="2" fill="#1a2a1a" id="download-sv_028-circle-010" /></g>
    <g transform="translate(1000 340)"><ellipse rx="16" ry="8" id="download-sv_028-ellipse-007" /><polygon points="-14,0 -26,-7 -26,7" fill="#ff8c1a" id="download-sv_028-polygon-007" /></g>
  </g>
  
  <g transform="translate(420 540)">
    <ellipse rx="30" ry="20" fill="#2a7adf" id="download-sv_028-ellipse-008" /><polygon points="-28,0 -48,-16 -48,16" fill="#1a4aaf" id="download-sv_028-polygon-008" />
    <ellipse rx="30" ry="20" fill="none" stroke="#ffe14d" stroke-width="3" id="download-sv_028-ellipse-009" /><circle cx="16" cy="-4" r="3" fill="#1a2a1a" id="download-sv_028-circle-011" />
  </g>
  
  <g transform="translate(1180 700)">
    <ellipse rx="20" ry="12" fill="#ff7a1a" id="download-sv_028-ellipse-010" /><polygon points="-18,0 -32,-9 -32,9" fill="#e85a0a" id="download-sv_028-polygon-009" />
    <rect x="-4" y="-12" width="6" height="24" fill="#fff" id="download-sv_028-rect-002" /><rect x="-14" y="-10" width="5" height="20" fill="#fff" id="download-sv_028-rect-003" /><circle cx="11" cy="-2" r="2.5" fill="#1a2a1a" id="download-sv_028-circle-012" />
  </g>

  
  <path d="M0 1000 L0 820 Q300 780 600 820 Q900 860 1200 810 Q1400 780 1600 820 L1600 1000 Z" fill="url(#sand08)" id="download-sv_028-path-009" />

  
  <g>
    
    <ellipse cx="200" cy="830" rx="90" ry="50" fill="#e07a9a" id="download-sv_028-ellipse-011" />
    <g stroke="#b85578" stroke-width="3" fill="none"><path d="M140 830 Q200 800 260 830" id="download-sv_028-path-010" /><path d="M150 850 Q200 824 250 850" id="download-sv_028-path-011" /><path d="M160 815 Q200 800 240 815" id="download-sv_028-path-012" /></g>
    
    <g fill="#ff8c5a">
      <path d="M420 880 q-10 -90 10 -120 q10 30 0 120 Z" id="download-sv_028-path-013" />
      <path d="M440 880 q-4 -120 30 -150 q-20 50 -10 150 Z" id="download-sv_028-path-014" />
      <path d="M400 880 q-14 -80 -10 -110 q24 30 24 110 Z" id="download-sv_028-path-015" />
    </g>
    
    <g transform="translate(1080 840)" fill="none" stroke="#9a5ad0" stroke-width="4">
      <path d="M0 0 Q-40 -80 -20 -160 M0 0 Q-10 -90 10 -170 M0 0 Q40 -80 30 -160 M0 0 Q-60 -60 -70 -120 M0 0 Q60 -60 70 -120" id="download-sv_028-path-016" />
    </g>
    
    <g fill="#f0c84a"><ellipse cx="700" cy="850" rx="14" ry="44" id="download-sv_028-ellipse-012" /><ellipse cx="730" cy="858" rx="12" ry="36" id="download-sv_028-ellipse-013" /><ellipse cx="676" cy="858" rx="11" ry="32" id="download-sv_028-ellipse-014" /></g>
    
    <g fill="#3ac88a"><path d="M1350 880 q-12 -90 8 -130 q14 40 6 130 Z" id="download-sv_028-path-017" /><path d="M1380 880 q-6 -100 26 -130 q-14 50 -8 130 Z" id="download-sv_028-path-018" /></g>
  </g>

  
  <g transform="translate(1180 840)">
    <ellipse rx="40" ry="20" fill="#c84a7a" id="download-sv_028-ellipse-015" />
    <g stroke="#e07aa0" stroke-width="4" stroke-linecap="round">
      <line x1="-30" y1="0" x2="-36" y2="-30" id="download-sv_028-line-001" /><line x1="-15" y1="0" x2="-18" y2="-40" id="download-sv_028-line-002" /><line x1="0" y1="0" x2="0" y2="-44" id="download-sv_028-line-003" />
      <line x1="15" y1="0" x2="18" y2="-40" id="download-sv_028-line-004" /><line x1="30" y1="0" x2="36" y2="-30" id="download-sv_028-line-005" /><line x1="-22" y1="0" x2="-28" y2="-36" id="download-sv_028-line-006" /><line x1="22" y1="0" x2="28" y2="-36" id="download-sv_028-line-007" />
    </g>
  </g>

  
  <g stroke="#2a8a5a" stroke-width="6" fill="none" stroke-linecap="round" opacity="0.85">
    <path d="M900 880 Q880 800 910 740 Q940 680 920 620" id="download-sv_028-path-019" />
    <path d="M930 880 Q950 810 925 760 Q905 700 930 640" id="download-sv_028-path-020" />
  </g>

  
  <g transform="translate(640 900)" fill="#ff6a4a">
    <path d="M0 -26 L8 -8 L26 -8 L12 4 L18 24 L0 12 L-18 24 L-12 4 L-26 -8 L-8 -8 Z" id="download-sv_028-path-021" />
  </g>
</svg>