factoryze.Try a challenge

Open source · Early preview

Fix a real bug with AI. See how you did it.

A coding test where AI is allowed, and every prompt, approval and token goes on the record.

12:00

A passing fix proves less than it used to.

With AI, people understood less.

  • By hand67%
  • With AI50%

Quiz score · Anthropic 2026

They felt faster. They weren't.

  • Felt−20%
  • Measured+19%

Time per task · METR 2025

Better practice. Worse exam.

  • Practice, with AI+48%
  • Exam, without−17%

Maths scores · PNAS 2025

Junior jobs go first.

−16%

Jobs, ages 22–25 · Stanford 2025

Who was studied
  • Anthropic: randomised trial, 52 mostly junior engineers learning one Python library. People who asked the AI to explain scored 65% or more; people who handed the task over scored under 40%.
  • METR: 16 experienced open-source developers, 246 tasks, early-2025 tools. METR's 2026 update says newer tools are likely faster.
  • Bastani et al.: about 1,000 high-school maths students. A version that gave hints instead of answers avoided the drop.
  • Brynjolfsson, Chandar and Chen: US payroll records, relative change in the most AI-exposed occupations since late 2022.

Every “try again” has a cost.

Tokens run on power and water. Factoryze counts every one.

945TWhElectricity per year
1,200bn litresWater per year

Illustration · IEA 2025 estimates

Who it's for

Learners

Practise on seven real bugs.

Browse tasks
Instructors

Run one task with a whole class.

Ask about a pilot
Experienced developers

Turn a bug you remember into a task.

Contribute

Why contribute

That's why you contribute. A whole community learns from the bug you remember.

Turn the bug you'll never forget into a task. It takes a complaint, starter code and tests.

Questions

Is the browser challenge real?

It's a scripted demo: no code runs and no AI model is called. The desktop editor runs real code, tests and an agent.

What is a token?

The unit AI models read and write in, about four characters of English. It is what AI usage is billed by.

What does the report record?

Hidden-test results, your prompts, the edits and commands you approved, time spent and token counts. It is saved when you submit.

Does Factoryze measure environmental impact?

No. It counts tokens. It does not estimate electricity, water or carbon for a session.

What happens to a task I upload?

A maintainer reviews it privately. Your code isn't run or published automatically.