Vector-Bench · a thetalab benchmark

theta labs / task

sv_034very_hard

downloaded_scenic_surgical

instructionnatural language

Remove the accidental marker from the jungle ruins. Shift the left canopy mass back toward the edge, restore the red body of the macaw, thin the smaller waterfall on the right, and make the central background canopy more visible. Keep the temple and foreground plants unchanged.

initial · brokeninput

target · expected fixcanonical

published model outputs

What each model produced.

34 endpoints

model result explorerGemini 3.1 Flash Lite

Gemini 3.1 Flash Lite

google/gemini-3.1-flash-lite-preview

TracePARTIAL

requested repairs

4/5

semantic preservation

changed

SVG validity

valid

source fidelity

different

target match

different

Full pass requires the first three gates. Source fidelity and canonical target match are diagnostics only.

outcome

partial

progress

69.3%

UCR

3.4%

preserve

96.6%

elapsed

3140.5s

cost

$0.0074

requested repairs4/5

sv_034-extra-survey-marker.existspassed

expected false / produced false

required presence matches

download-sv_034-ellipse-009.fillmissed

expected #e03a2a / produced #ef4444

color-lab: 9.862 <= 4.438 Delta E76 / 0.0% repaired

download-sv_034-ellipse-001.cxpassed

expected 200 / produced 203

numeric-delta: 3 <= 21.5 user units / 93.0% repaired

download-sv_034-line-006.stroke-widthpassed

expected 3 / produced 4

line-weight-delta: 1 <= 1.2 user units / 80.0% repaired

download-sv_034-ellipse-002.opacitypassed

expected 0.7 / produced 0.6

opacity-delta: 0.1 <= 0.12 absolute opacity / 73.7% repaired

Why it did not pass

requested repairs incompleteunintended document changes

Unexpected changes

download-sv_034-circle-006download-sv_034-rect-003

Semantic preservation failures

download-sv_034-rect-003download-sv_034-circle-006

Source fidelity diagnostics

download-sv_034-rect-003download-sv_034-circle-006

prompt 4,398 tokoutput 4,223 tokreasoning 0 tokceiling 4,660 tokfinish stopcheap control

resolved google/gemini-3.1-flash-lite-preview

response gen-1784548039-3A9QlmBY38Oij4xesLU2

expected diff5 edits

Part	Attr	Before	After
sv_034-extra-survey-marker	exists	true	false
download-sv_034-ellipse-009	fill	#ef4444	#e03a2a
download-sv_034-ellipse-001	cx	243	200
download-sv_034-line-006	stroke-width	8	3
download-sv_034-ellipse-002	opacity	0.32	0.7

partsscope

Target (5)

sv_034-extra-survey-marker

download-sv_034-ellipse-009

download-sv_034-ellipse-001

download-sv_034-line-006

download-sv_034-ellipse-002

Should preserve (59)

download-sv_034-rect-001

download-sv_034-ellipse-003

download-sv_034-path-001

download-sv_034-path-002

download-sv_034-path-003

download-sv_034-rect-002

download-sv_034-line-001

download-sv_034-line-002

download-sv_034-line-003

download-sv_034-line-004

download-sv_034-line-005

download-sv_034-rect-003

download-sv_034-polygon-001

download-sv_034-polygon-002

download-sv_034-polygon-003

download-sv_034-polygon-004

download-sv_034-rect-004

download-sv_034-polygon-005

download-sv_034-line-007

download-sv_034-line-008

download-sv_034-rect-005

download-sv_034-rect-006

download-sv_034-rect-007

download-sv_034-rect-008

download-sv_034-rect-009

download-sv_034-path-004

download-sv_034-path-005

download-sv_034-path-006

download-sv_034-path-007

download-sv_034-circle-001

download-sv_034-circle-002

download-sv_034-circle-003

download-sv_034-circle-004

download-sv_034-circle-005

download-sv_034-ellipse-004

download-sv_034-ellipse-005

download-sv_034-ellipse-006

download-sv_034-ellipse-007

download-sv_034-path-008

download-sv_034-path-009

download-sv_034-path-010

download-sv_034-line-009

download-sv_034-path-011

download-sv_034-path-012

download-sv_034-line-010

download-sv_034-path-013

download-sv_034-ellipse-008

download-sv_034-circle-006

download-sv_034-polygon-006

download-sv_034-polygon-007

download-sv_034-circle-007

download-sv_034-path-014

download-sv_034-ellipse-010

download-sv_034-circle-008

download-sv_034-polygon-008

download-sv_034-polygon-009

download-sv_034-circle-009

download-sv_034-path-015

download-sv_034-path-016

initial svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1600 1000">
  <defs>
    <linearGradient id="sky14" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#cfe8d0" /><stop offset="1" stop-color="#e8f0c8" />
    </linearGradient>
    <linearGradient id="fall14" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#eafdff" /><stop offset="0.6" stop-color="#bfe8f0" /><stop offset="1" stop-color="#8fd0e0" />
    </linearGradient>
    <linearGradient id="pool14" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#7fd0c8" /><stop offset="1" stop-color="#2a8a7a" />
    </linearGradient>
    <linearGradient id="rock14" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#7a8a6a" /><stop offset="1" stop-color="#4a5a3a" />
    </linearGradient>
    <linearGradient id="stone14" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#a8a89a" /><stop offset="1" stop-color="#787868" />
    </linearGradient>
    <filter id="mist14" x="-50%" y="-50%" width="200%" height="200%"><feGaussianBlur stdDeviation="20" /></filter>
  </defs>

  <rect width="1600" height="1000" fill="url(#sky14)" id="download-sv_034-rect-001" />

  
  <g filter="url(#mist14)">
    <ellipse cx="243" cy="160" rx="360" ry="180" fill="#6a9a5a" opacity="0.7" id="download-sv_034-ellipse-001" />
    <ellipse cx="800" cy="120" rx="420" ry="180" fill="#5a8a4a" opacity="0.32" id="download-sv_034-ellipse-002" />
    <ellipse cx="1400" cy="160" rx="360" ry="180" fill="#6a9a5a" opacity="0.7" id="download-sv_034-ellipse-003" />
  </g>

  
  <path d="M0 0 L0 720 Q120 700 240 720 L320 200 Q280 80 200 60 L0 40 Z" fill="url(#rock14)" id="download-sv_034-path-001" />
  <path d="M1600 0 L1600 720 Q1480 700 1360 720 L1280 200 Q1320 80 1400 60 L1600 40 Z" fill="url(#rock14)" id="download-sv_034-path-002" />
  
  <path d="M460 180 L1140 180 L1180 700 L420 700 Z" fill="#3a4a2e" id="download-sv_034-path-003" />

  
  <g>
    <rect x="660" y="180" width="280" height="560" fill="url(#fall14)" id="download-sv_034-rect-002" />
    <g stroke="#ffffff" stroke-width="4" opacity="0.6">
      <line x1="700" y1="200" x2="700" y2="720" id="download-sv_034-line-001" /><line x1="740" y1="200" x2="738" y2="720" id="download-sv_034-line-002" /><line x1="800" y1="200" x2="800" y2="720" id="download-sv_034-line-003" />
      <line x1="860" y1="200" x2="862" y2="720" id="download-sv_034-line-004" /><line x1="900" y1="200" x2="900" y2="720" id="download-sv_034-line-005" />
    </g>
    
    <rect x="980" y="280" width="80" height="420" fill="url(#fall14)" opacity="0.85" id="download-sv_034-rect-003" />
    <line x1="1020" y1="290" x2="1020" y2="680" stroke="#ffffff" stroke-width="8" opacity="0.6" id="download-sv_034-line-006" />
  </g>

  
  <g transform="translate(420 480)">
    
    <polygon points="0,200 360,200 320,150 40,150" fill="url(#stone14)" id="download-sv_034-polygon-001" />
    <polygon points="40,150 320,150 290,108 70,108" fill="#9a9a8a" id="download-sv_034-polygon-002" />
    <polygon points="70,108 290,108 264,70 96,70" fill="#8a8a7a" id="download-sv_034-polygon-003" />
    <polygon points="96,70 264,70 240,40 120,40" fill="#9a9a8a" id="download-sv_034-polygon-004" />
    
    <rect x="160" y="120" width="44" height="80" fill="#2a3022" id="download-sv_034-rect-004" />
    <polygon points="158,120 206,120 182,96" fill="#3a4030" id="download-sv_034-polygon-005" />
    
    <g stroke="#5a5a4a" stroke-width="2" opacity="0.5"><line x1="60" y1="160" x2="100" y2="190" id="download-sv_034-line-007" /><line x1="280" y1="120" x2="300" y2="150" id="download-sv_034-line-008" /></g>
  </g>
  
  <g fill="url(#stone14)">
    <rect x="300" y="640" width="40" height="120" id="download-sv_034-rect-005" /><rect x="296" y="632" width="48" height="12" id="download-sv_034-rect-006" />
    <rect x="1240" y="660" width="40" height="100" id="download-sv_034-rect-007" /><rect x="1236" y="652" width="48" height="12" id="download-sv_034-rect-008" />
    <rect x="1180" y="700" width="36" height="60" id="download-sv_034-rect-009" />
  </g>

  
  <g stroke="#3a6a2a" stroke-width="4" fill="none" opacity="0.85">
    <path d="M460 480 Q440 540 470 600 Q500 660 460 720" id="download-sv_034-path-004" />
    <path d="M700 480 Q720 540 690 600" id="download-sv_034-path-005" />
    <path d="M40 200 Q70 300 40 400 Q20 500 50 600" id="download-sv_034-path-006" />
    <path d="M1560 200 Q1530 300 1560 400 Q1580 500 1550 600" id="download-sv_034-path-007" />
  </g>
  <g fill="#4a8a3a" opacity="0.85">
    <circle cx="468" cy="560" r="10" id="download-sv_034-circle-001" /><circle cx="462" cy="700" r="9" id="download-sv_034-circle-002" /><circle cx="46" cy="380" r="10" id="download-sv_034-circle-003" /><circle cx="1556" cy="380" r="10" id="download-sv_034-circle-004" /><circle cx="692" cy="560" r="9" id="download-sv_034-circle-005" />
  </g>

  
  <ellipse cx="800" cy="820" rx="560" ry="120" fill="url(#pool14)" id="download-sv_034-ellipse-004" />
  <g fill="none" stroke="#cffff0" stroke-width="2.5" opacity="0.6">
    <ellipse cx="800" cy="760" rx="120" ry="22" id="download-sv_034-ellipse-005" /><ellipse cx="800" cy="800" rx="200" ry="34" id="download-sv_034-ellipse-006" /><ellipse cx="800" cy="850" rx="300" ry="44" id="download-sv_034-ellipse-007" />
  </g>
  
  <g fill="#2a5a22">
    <path d="M0 1000 L0 880 Q120 820 200 880 Q160 940 80 960 Q40 980 0 1000 Z" id="download-sv_034-path-008" />
    <path d="M1600 1000 L1600 880 Q1480 820 1400 880 Q1440 940 1520 960 Q1560 980 1600 1000 Z" id="download-sv_034-path-009" />
  </g>
  
  <g fill="#3a7a2a">
    <path d="M120 980 Q60 900 120 840 Q180 900 120 980 Z" id="download-sv_034-path-010" /><line x1="120" y1="980" x2="120" y2="850" stroke="#2a5a1a" stroke-width="2" id="download-sv_034-line-009" />
    <path d="M220 1000 Q160 920 220 860 Q280 920 220 1000 Z" id="download-sv_034-path-011" />
    <path d="M1480 980 Q1420 900 1480 840 Q1540 900 1480 980 Z" id="download-sv_034-path-012" /><line x1="1480" y1="980" x2="1480" y2="850" stroke="#2a5a1a" stroke-width="2" id="download-sv_034-line-010" />
    <path d="M1380 1000 Q1320 920 1380 860 Q1440 920 1380 1000 Z" id="download-sv_034-path-013" />
  </g>

  
  <ellipse cx="800" cy="730" rx="280" ry="70" fill="#ffffff" opacity="0.5" filter="url(#mist14)" id="download-sv_034-ellipse-008" />

  
  <g transform="translate(420 360)">
    <ellipse cx="0" cy="0" rx="22" ry="12" fill="#ef4444" id="download-sv_034-ellipse-009" /><circle cx="18" cy="-6" r="8" fill="#e03a2a" id="download-sv_034-circle-006" />
    <polygon points="-18,0 -56,-8 -40,4 -56,12" fill="#2a6ad0" id="download-sv_034-polygon-006" /><polygon points="-10,6 -30,28 -4,16" fill="#f0c83a" id="download-sv_034-polygon-007" />
    <circle cx="22" cy="-8" r="2" fill="#fff" id="download-sv_034-circle-007" /><path d="M26 -6 q8 0 8 6" stroke="#3a3a3a" stroke-width="3" fill="none" id="download-sv_034-path-014" />
  </g>
  <g transform="translate(1180 320) scale(0.8)">
    <ellipse cx="0" cy="0" rx="22" ry="12" fill="#f0c83a" id="download-sv_034-ellipse-010" /><circle cx="-18" cy="-6" r="8" fill="#f0c83a" id="download-sv_034-circle-008" />
    <polygon points="18,0 56,-8 40,4 56,12" fill="#2a9a4a" id="download-sv_034-polygon-008" /><polygon points="10,6 30,28 4,16" fill="#2a6ad0" id="download-sv_034-polygon-009" />
    <circle cx="-22" cy="-8" r="2" fill="#fff" id="download-sv_034-circle-009" />
  </g>
  
  <g stroke="#3a3a3a" stroke-width="3" fill="none" stroke-linecap="round" opacity="0.6">
    <path d="M760 240 q10 -8 20 0 q10 -8 20 0" id="download-sv_034-path-015" /><path d="M840 280 q9 -7 18 0 q9 -7 18 0" id="download-sv_034-path-016" />
  </g>
<circle id="sv_034-extra-survey-marker" cx="1200" cy="296" r="35" fill="#22c55e" opacity="0.82" stroke="#111827" stroke-width="5" /></svg>

target svgsource

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1600 1000">
  <defs>
    <linearGradient id="sky14" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#cfe8d0" /><stop offset="1" stop-color="#e8f0c8" />
    </linearGradient>
    <linearGradient id="fall14" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#eafdff" /><stop offset="0.6" stop-color="#bfe8f0" /><stop offset="1" stop-color="#8fd0e0" />
    </linearGradient>
    <linearGradient id="pool14" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#7fd0c8" /><stop offset="1" stop-color="#2a8a7a" />
    </linearGradient>
    <linearGradient id="rock14" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#7a8a6a" /><stop offset="1" stop-color="#4a5a3a" />
    </linearGradient>
    <linearGradient id="stone14" x1="0" y1="0" x2="0" y2="1">
      <stop offset="0" stop-color="#a8a89a" /><stop offset="1" stop-color="#787868" />
    </linearGradient>
    <filter id="mist14" x="-50%" y="-50%" width="200%" height="200%"><feGaussianBlur stdDeviation="20" /></filter>
  </defs>

  <rect width="1600" height="1000" fill="url(#sky14)" id="download-sv_034-rect-001" />

  
  <g filter="url(#mist14)">
    <ellipse cx="200" cy="160" rx="360" ry="180" fill="#6a9a5a" opacity="0.7" id="download-sv_034-ellipse-001" />
    <ellipse cx="800" cy="120" rx="420" ry="180" fill="#5a8a4a" opacity="0.7" id="download-sv_034-ellipse-002" />
    <ellipse cx="1400" cy="160" rx="360" ry="180" fill="#6a9a5a" opacity="0.7" id="download-sv_034-ellipse-003" />
  </g>

  
  <path d="M0 0 L0 720 Q120 700 240 720 L320 200 Q280 80 200 60 L0 40 Z" fill="url(#rock14)" id="download-sv_034-path-001" />
  <path d="M1600 0 L1600 720 Q1480 700 1360 720 L1280 200 Q1320 80 1400 60 L1600 40 Z" fill="url(#rock14)" id="download-sv_034-path-002" />
  
  <path d="M460 180 L1140 180 L1180 700 L420 700 Z" fill="#3a4a2e" id="download-sv_034-path-003" />

  
  <g>
    <rect x="660" y="180" width="280" height="560" fill="url(#fall14)" id="download-sv_034-rect-002" />
    <g stroke="#ffffff" stroke-width="4" opacity="0.6">
      <line x1="700" y1="200" x2="700" y2="720" id="download-sv_034-line-001" /><line x1="740" y1="200" x2="738" y2="720" id="download-sv_034-line-002" /><line x1="800" y1="200" x2="800" y2="720" id="download-sv_034-line-003" />
      <line x1="860" y1="200" x2="862" y2="720" id="download-sv_034-line-004" /><line x1="900" y1="200" x2="900" y2="720" id="download-sv_034-line-005" />
    </g>
    
    <rect x="980" y="280" width="80" height="420" fill="url(#fall14)" opacity="0.85" id="download-sv_034-rect-003" />
    <line x1="1020" y1="290" x2="1020" y2="680" stroke="#ffffff" stroke-width="3" opacity="0.6" id="download-sv_034-line-006" />
  </g>

  
  <g transform="translate(420 480)">
    
    <polygon points="0,200 360,200 320,150 40,150" fill="url(#stone14)" id="download-sv_034-polygon-001" />
    <polygon points="40,150 320,150 290,108 70,108" fill="#9a9a8a" id="download-sv_034-polygon-002" />
    <polygon points="70,108 290,108 264,70 96,70" fill="#8a8a7a" id="download-sv_034-polygon-003" />
    <polygon points="96,70 264,70 240,40 120,40" fill="#9a9a8a" id="download-sv_034-polygon-004" />
    
    <rect x="160" y="120" width="44" height="80" fill="#2a3022" id="download-sv_034-rect-004" />
    <polygon points="158,120 206,120 182,96" fill="#3a4030" id="download-sv_034-polygon-005" />
    
    <g stroke="#5a5a4a" stroke-width="2" opacity="0.5"><line x1="60" y1="160" x2="100" y2="190" id="download-sv_034-line-007" /><line x1="280" y1="120" x2="300" y2="150" id="download-sv_034-line-008" /></g>
  </g>
  
  <g fill="url(#stone14)">
    <rect x="300" y="640" width="40" height="120" id="download-sv_034-rect-005" /><rect x="296" y="632" width="48" height="12" id="download-sv_034-rect-006" />
    <rect x="1240" y="660" width="40" height="100" id="download-sv_034-rect-007" /><rect x="1236" y="652" width="48" height="12" id="download-sv_034-rect-008" />
    <rect x="1180" y="700" width="36" height="60" id="download-sv_034-rect-009" />
  </g>

  
  <g stroke="#3a6a2a" stroke-width="4" fill="none" opacity="0.85">
    <path d="M460 480 Q440 540 470 600 Q500 660 460 720" id="download-sv_034-path-004" />
    <path d="M700 480 Q720 540 690 600" id="download-sv_034-path-005" />
    <path d="M40 200 Q70 300 40 400 Q20 500 50 600" id="download-sv_034-path-006" />
    <path d="M1560 200 Q1530 300 1560 400 Q1580 500 1550 600" id="download-sv_034-path-007" />
  </g>
  <g fill="#4a8a3a" opacity="0.85">
    <circle cx="468" cy="560" r="10" id="download-sv_034-circle-001" /><circle cx="462" cy="700" r="9" id="download-sv_034-circle-002" /><circle cx="46" cy="380" r="10" id="download-sv_034-circle-003" /><circle cx="1556" cy="380" r="10" id="download-sv_034-circle-004" /><circle cx="692" cy="560" r="9" id="download-sv_034-circle-005" />
  </g>

  
  <ellipse cx="800" cy="820" rx="560" ry="120" fill="url(#pool14)" id="download-sv_034-ellipse-004" />
  <g fill="none" stroke="#cffff0" stroke-width="2.5" opacity="0.6">
    <ellipse cx="800" cy="760" rx="120" ry="22" id="download-sv_034-ellipse-005" /><ellipse cx="800" cy="800" rx="200" ry="34" id="download-sv_034-ellipse-006" /><ellipse cx="800" cy="850" rx="300" ry="44" id="download-sv_034-ellipse-007" />
  </g>
  
  <g fill="#2a5a22">
    <path d="M0 1000 L0 880 Q120 820 200 880 Q160 940 80 960 Q40 980 0 1000 Z" id="download-sv_034-path-008" />
    <path d="M1600 1000 L1600 880 Q1480 820 1400 880 Q1440 940 1520 960 Q1560 980 1600 1000 Z" id="download-sv_034-path-009" />
  </g>
  
  <g fill="#3a7a2a">
    <path d="M120 980 Q60 900 120 840 Q180 900 120 980 Z" id="download-sv_034-path-010" /><line x1="120" y1="980" x2="120" y2="850" stroke="#2a5a1a" stroke-width="2" id="download-sv_034-line-009" />
    <path d="M220 1000 Q160 920 220 860 Q280 920 220 1000 Z" id="download-sv_034-path-011" />
    <path d="M1480 980 Q1420 900 1480 840 Q1540 900 1480 980 Z" id="download-sv_034-path-012" /><line x1="1480" y1="980" x2="1480" y2="850" stroke="#2a5a1a" stroke-width="2" id="download-sv_034-line-010" />
    <path d="M1380 1000 Q1320 920 1380 860 Q1440 920 1380 1000 Z" id="download-sv_034-path-013" />
  </g>

  
  <ellipse cx="800" cy="730" rx="280" ry="70" fill="#ffffff" opacity="0.5" filter="url(#mist14)" id="download-sv_034-ellipse-008" />

  
  <g transform="translate(420 360)">
    <ellipse cx="0" cy="0" rx="22" ry="12" fill="#e03a2a" id="download-sv_034-ellipse-009" /><circle cx="18" cy="-6" r="8" fill="#e03a2a" id="download-sv_034-circle-006" />
    <polygon points="-18,0 -56,-8 -40,4 -56,12" fill="#2a6ad0" id="download-sv_034-polygon-006" /><polygon points="-10,6 -30,28 -4,16" fill="#f0c83a" id="download-sv_034-polygon-007" />
    <circle cx="22" cy="-8" r="2" fill="#fff" id="download-sv_034-circle-007" /><path d="M26 -6 q8 0 8 6" stroke="#3a3a3a" stroke-width="3" fill="none" id="download-sv_034-path-014" />
  </g>
  <g transform="translate(1180 320) scale(0.8)">
    <ellipse cx="0" cy="0" rx="22" ry="12" fill="#f0c83a" id="download-sv_034-ellipse-010" /><circle cx="-18" cy="-6" r="8" fill="#f0c83a" id="download-sv_034-circle-008" />
    <polygon points="18,0 56,-8 40,4 56,12" fill="#2a9a4a" id="download-sv_034-polygon-008" /><polygon points="10,6 30,28 4,16" fill="#2a6ad0" id="download-sv_034-polygon-009" />
    <circle cx="-22" cy="-8" r="2" fill="#fff" id="download-sv_034-circle-009" />
  </g>
  
  <g stroke="#3a3a3a" stroke-width="3" fill="none" stroke-linecap="round" opacity="0.6">
    <path d="M760 240 q10 -8 20 0 q10 -8 20 0" id="download-sv_034-path-015" /><path d="M840 280 q9 -7 18 0 q9 -7 18 0" id="download-sv_034-path-016" />
  </g>
</svg>