Change the wording, keep your experiment. Saved in this browser when storage is available.
Simone Systems Research / Search laboratory
Build a challenge for two AI strategies.
Give two search methods the same puzzle and trial budget. Then compare their chosen moves with the best possible move. Doing more searching is useful only if it leads to better decisions.Create a puzzle and give two search strategies equal work. Compare their moves against perfect play. This lets you test whether extra search improves a decision, rather than judging a method by how much work it appears to do.
Real classical AI search, running on your device. No account, model service or API key.
01 / Make a move
Can you beat perfect play?
Take either end. Your opponent replies. Each number scores points. Higher total wins.
Choose the left or right end. Hint: what can your opponent take next?
Make your own challenge
Change any of the six values, then apply. Each value must be a whole number from 1 to 9.
02 / Inspect the thinking
How each strategy spends its trials
These are actual sampled decisions, not scripted answers. Change the budget and rerun. A search can miss the best move.
Method 1
Spread the trials
Give each first move equal trials. Play the rest at random.
Method 2
Build a search tree
Spend more visits on promising branches. At opponent turns, search for their advantage.
Check against perfect play
More trials give a steadier picture of what happens against a player who moves randomly. A strong player can choose very differently. Better sampling does not fix that wrong opponent assumption.More random trials estimate the random opponent more steadily. That opponent may behave differently from a strong player.
Open one possible future
First 20 trials retained for inspection. Every trial still counts toward the decision.
03 / Check the claim
Compare strategies on matched challenges
Run 128 paired challenges at each budget. The two methods swap seats on the same board. Then each faces the same perfect opponent. Custom boards and discovered cases stay separate from this fixed test.
What makes this comparison fair?
Budgets: 8, 32 and 128 complete trials per non-forced decision. Each trial consumes one transition per remaining gem. Boards use values 1–3; tasks and random streams replay from the seed. Fixed stop: all 128 pairs at all three budgets. Tree bookkeeping adds overhead, so transitions are a specific work measure, not wall time, tokens or dollars.
The relative score uses win = 1, draw = ½, loss = 0, averaged within each seat pair. Final bands use a conservative Hoeffding bound across the three predeclared comparisons at 95% family coverage, under independent task sampling. A band crossing 50% is inconclusive. Fixed-opponent scores and move regret below are descriptive. Repeated runs do not create new independent confirmation.
Ready. Results are calculated here, when you run the experiment.
| Budget | Spread vs Tree | Final band | Against perfect play Spread / Tree | Move regret Spread / Tree | Measured transitions |
|---|---|---|---|---|---|
| No experiment run yet. | |||||
If the result is inconclusive, this test cannot tell us which method is better. That is different from showing they are equally good. W, D and L count Spread’s wins, draws and losses across both seats.Inconclusive means the test cannot establish a winner. It does not prove the strategies are equally good. W / D / L means wins, draws and losses for Spread across both seats.
Move regret measures how many points the opening choice gives up compared with the best opening, assuming perfect play afterward. Smaller is better. The exact solver checks the search methods; it does not help them choose.Move regret is the optimal opening margin minus the chosen move’s optimal margin, in points. Smaller is better. An oracle is a checking tool; the search methods cannot consult it.
04 / Make it yours
Save a playable experiment
Download a standalone playable lab with your starting board and search code. It runs offline. Recorded comparisons retain each strategy’s actual seed. Save the experiment receipt to inspect or reproduce the full test.
Would you trust this claim?
“At 128 trials, A beat B more often than at 8 trials.” Both methods received more trials. Does this prove A improved against a fixed opponent?
It does not establish that. B changed too. Compare A at both budgets against the same opponent, on matched tasks, with counted work and uncertainty. That is why this lab reports both comparisons.
How this works, credits and privacy
An original bounded game and deterministic browser simulation. The tree method uses adversarial UCT with c = √2 and chooses the most visited opening move. Spread chooses the highest sampled mean. Ties use a seeded choice. This is classical search, not a language model or a display of private reasoning.
Created by Jonathan Simone with AI assistance. Original diagrams and gem shapes. References: the original UCT paper, bounded-score statistics, Fact 1, Bret Victor’s exploration design, and Amit Patel’s visible search explanations. No private research records are included.
The activity uses page memory and makes no model or API calls. Downloads are created on your device. Refresh clears the activity. Usability and learning benefit have not been measured with visitors.