Do game-design skills make Claude Code better? We tested it
We tested our game-design skills against plain Claude Code. Fixing games was a tie, and diagnosing was worse, until we made Claude ask one question before it fixes anything.

Our game-design skills did not make Claude Code better at fixing games: on 20 broken games, a blind judge called it a tie. What did help was making Claude diagnose before it fixes: in a fair test it found the real cause of a complaint 78% of the time, against 42% for plain Claude Code.
We make the skills on this site: an MCP server and a /aigamedev plugin that give Claude Code game-design advice (jump feel, cameras, checkpoints and more) while it works. This is how we tested them, what went wrong, and what we changed.
Test 1: fix 20 broken games, judged blind
We built 20 small browser games, each with one planted problem: no checkpoints, a jerky camera, prices that make no sense, and so on. Claude Code got only the playtester complaint, such as “the game is too punishing”, and fixed each game twice: once with our plugin, once without.
Two checks followed. A script measured the planted problem in each fixed game. A blind Opus judge, who didn’t know which fix came from where, read both fixes, played them and picked the better one.
Result: a tie
| Plain Claude Code | With the plugin | |
|---|---|---|
| Judge picked it as better | 8 | 8 |
| Fully fixed the problem | 18 of 20 | 13 of 20 |
| Average quality, out of 5 | 3.80 | 3.75 |
| Fixes rated 5 out of 5 | 0 | 6 |
The judge called the other 4 games a tie. Plain Claude Code fully fixed more games, but when the plugin did fix one, it tended to go further: all six fixes rated 5 out of 5 came from the plugin.
The skills added precision, not ideas
Take “the game is too punishing” in a platformer. The planted cause: no checkpoints, so any death reloaded a 3-minute level. Both versions added checkpoints and kept the score on death, and the judge rated both as fixed.
The difference was in the details. With the plugin, Claude Code also added an instant-retry key and measured the result with a script that died at every hazard: the average replay after a death went from 99.5 seconds to 3.1. Plain Claude Code added something the skill doesn’t mention: about 1.5 seconds of protection after each respawn, so “a slime walking onto the checkpoint can’t kill you again right away.”
That was the pattern across the 20 games. Claude already knows the common fixes. The skills added numbers and a way to measure, not new ideas. Building and judging games is also slow and noisy, so for the next test we looked at the step where the ideas come from: the diagnosis.
Test 2: talk it through, then score the diagnosis
One complaint can have several causes. “The controls feel floaty” might be a jump that hangs, a character that slides, or physics tied to the frame rate. Each needs a different fix, so the first job is to find out which one it is.
We have a problem map: 100 playtester complaints across 8 genres, linked to 76 causes. From it we made 95 scenarios. In each one, a simulated developer knows the facts of their game and one hidden cause. They talk with Claude Code until it commits to a plan. A blind scorer with the answer key then checks the plan: did Claude find the real cause, did it give a concrete way to measure the fix, and did it ask a question that tells the causes apart? Last, we audited every reasoning step to see where each wrong answer started.
We tested three ways of working:
- Plain Claude Code.
- With the plugin. A router reads the complaint, picks one skill, and Claude follows it.
- Diagnose first. Claude lists the candidate causes from the map, asks one question that separates them, and only then loads a skill.
Result: diagnose first found the cause twice as often
| Found the real cause | When it was a less common cause | |
|---|---|---|
| Plain Claude Code | 42% | 30% |
| With the plugin | 39% | 18% |
| Diagnose first, full map | 88% | 84% |
| Diagnose first, real cause removed from the map | 78% | 70% |
| Same, plus a “name a new cause” rule | 82% | 74% |
| Same, and the map’s own questions removed too | 78% | 70% |
The plugin did help with one thing. It gave a concrete way to measure the fix 91% of the time, against 47% for plain Claude Code. Plain Claude asked a question that separates the causes in only 24% of conversations.
See it: one complaint, three first answers
Pick a complaint. Each column shows how that version opened the conversation. Then reveal the cause the simulated developer was hiding.
Why the plugin lost: it picked a skill before it knew the cause
The router picks a skill from the words of the complaint, and the words don’t tell you the cause. The audit of all 95 plugin conversations found three ways this went wrong:
- In 29%, Claude’s own one-line purpose for the router had already narrowed to the wrong cause before the router saw it.
- In 37%, the router’s pick was wrong for the real cause.
- In 38%, Claude went along with a wrong pick.
You can see it in the Ember Ward conversation. The real cause was enemies that are easy to exploit. Claude’s purpose line already said “break the dominant strategy”, the router returned the encounter-variety skill, and the reply opened with “Skill used: encounter-variety”. It never considered the enemies. Plain Claude Code came closer: it mentioned kiting in circles, but then planned around nerfing the dash and the burst. In Moss Hopper, the purpose line was already “make enemies readable against the background”, so the camera never came up.
Diagnose first: 88%, but that’s a best case
Diagnose first found the real cause 88% of the time. But its list of candidate causes comes from the same map the scorer used as its answer key, so 88% is a ceiling, not a fair score. The fair test is next.
The fair test: hide the real cause from the map
For the hold-out test, we removed the real cause from the map Claude sees. Diagnose first still found it 78% of the time. With one more rule, “if the answer fits none of the causes, say so and name a new one”, it found it 82% of the time.
Then we found a leak in our own test. The map also gives a ready-made question for each complaint, and some of those questions still offered the hidden cause as an answer. For “this level takes forever” one option was “I keep getting sent back”. So we ran it again with the real cause and every map question removed, and Claude had to write its own question. That is the number we stand by: 78%, and 70% for less common causes. The leak was worth about 4 points.
Seeing a list of options still made Claude think of causes that weren’t on it. With the cause and the questions removed, the real cause was among Claude’s candidates 73% of the time. In Lanternfall, from the earlier hold-out run, the map listed only one cause for “this level takes forever”. Claude listed five, including the hidden one (“Dying or failing sends you back too far”), and marked another as its own idea: “This one isn’t in my list, but it fits a dark, flooded tower.”
The map’s questions were worth something too. Without them, Claude asked a question that really separates the causes in 80% of conversations instead of 97%, and only 66% when the cause was a less common one.
| Plain Claude Code | Diagnose first, strict hold-out | |
|---|---|---|
| Found the real cause | 42% | 78% |
| …when it was a less common cause | 30% | 70% |
| Plan aimed at a wrong cause | 41% | 13% |
| Concrete way to measure the fix | 47% | 99% |
It costs more: about $0.90 a conversation instead of $0.18
Diagnosing means reading the map and one more round trip with you. It cost about $0.80 to $0.90 per conversation against $0.18 for plain Claude Code, and took about 2 replies instead of 1.3.
What we changed
- The MCP server has a new tool,
diagnose_problem, and it’s live. Give it a complaint and a genre, and it returns the candidate causes and a question that separates them. /aigamedevnow diagnoses before it fixes. It callsdiagnose_problem, asks you one question, and only then loads a skill.
We checked it live on 4 complaints. In every session, Claude called diagnose_problem and asked one question before suggesting any fix. For “hitting enemies feels weak”, it also added a cause the map lacks: “There’s a fourth I’d add: players can’t tell whether a hit landed”.
What this test can’t tell you
- The developers were simulated, not real people.
- Their facts were written to separate the map’s causes, so one good question usually settled it. Real playtest notes are messier.
- The first hold-out run leaked: the map’s questions could still offer the hidden cause. The strict run fixes that, and it’s the one we quote (78%).
- The real cause was always a cause from our own map, just hidden. Problems the map has never heard of may go worse.
- Each scenario ran once. A gap of a few points, like 42% against 39%, is noise.
- The built-game test had only 20 games.
Try it
Run these two lines once in your terminal:
claude plugin marketplace add https://aigamedev.ai/agents/marketplace.json
claude plugin install aigamedev@aigamedev
Then, in Claude Code, type /mcp, pick aigamedev and choose Authenticate. GitHub opens in your browser; click Authorize. Now give it a complaint from your playtest:
/aigamedev playtesters say the controls feel floaty
You should get one question back before any fix. The Claude app, Cursor and VS Code can use the same server: see Connect your AI agent.
Next: real developers
Simulated developers answer cleanly and know the facts that matter. Next we want to run diagnose first with real developers and their real playtest notes, and check whether the cause it names is the one that fixes the game. If you try it on your game, tell us whether it found the right cause.
Was this useful?
Thanks, we read every message.

