Machine learning and AI learning: 10 inspectable demonstrations across game theory, control, pixel RL, adversarial evaluation, and closed-loop gating

Starting from a uniform 50/50 policy across all 12 information states, tabular Counterfactual Regret Minimization (CFR) reduces exact best-response exploitability from 0.458333 to 0.000938 (-99.80%) in 1,000 updates (1.172 seconds). Without domain rules, CFR converges to the coupled analytical Nash family parameterized by alpha = 0.1940: bluffing Jack after a pass at 33.30% (exact Nash 1/3), calling with Queen facing a bet at 33.64% (exact Nash 1/3), slow-playing King 41.59% of the time (betting 58.41% vs 3*alpha = 58.20%), and calling with Queen after pass-bet at 53.05% (vs alpha + 1/3 = 52.73%).

Trainer PID 78400 | Evaluator PID 77484 | Initial exploitability 0.458333 | Final exploitability 0.000938 ( -99.7954%) | Reset check 0.458333

Trained from raw 4 x 84 x 84 grayscale frame stacks for 2,000,896 environment steps (31,264 optimizer updates across 1,954 rollouts in 1,300.08 seconds; trainer PID 48704, evaluator PID 7004). Initial parameters (SHA-256 6a5f2c086bb18025...) lose every game at -21.00 (0/20 wins). Final parameters (SHA-256 3b55949ee318620b...) reach +19.25 ( 20/20 wins, paired gain +40.25, 95% CI [+40.10, +40.45]) on held-out seeds 910001-910020, and +19.50 (20/20 wins) on independent review seeds 930001-930020. Resetting the policy returns immediately to -21.00.

In deterministic Atari (repeat_action_probability = 0.0), resetting the emulator across 20 seeds yields only 2 unique initial observations and 15 unique trajectories, allowing an open-loop action tape to score +21.00 (30/30 shutouts). Testing under the Machado et al. (2018) standard (repeat_action_probability = 0.25 , checkpoint SHA-256 59f2e93a4db15c2a...) forces 20/20 unique trajectories: a scripted RAM ball-Y tracker collapses to -15.00, while our pixel PPO policy wins 29/30 fresh episodes (mean +7.60, IQM +7.38).

Digest qualification: The historical record retains a 52-character value, not a complete SHA-256 and not an exact prefix. It is preserved for compatibility and is not usable for integrity verification. The full historical digest has not been established.

Reporting a single seed or cherry-picking the best checkpoint on evaluation seeds inflates RL claims. On CartPole, seed 0 reaches +457.9 +/- 76.2 while seed 1 collapses to +9.3 +/- 0.8. On Pong (`sticky = 0.25`), picking the best of 42 checkpoints on seeds 5000-5029 (`best_8750000`) yields +8.87, but shrinks by -1.14 points to +7.73 on fresh seeds 9000-9029 (indistinguishable from the unselected 10M checkpoint at +7.60). To prove LunarLander-v3 PPO under Agarwal et al. (2021), we trained 10 independent seeds (1,015,808 steps each) and evaluated across 200 held-out seeds per model (2,000 episodes):

In Super Mario Bros (NES), horizontal progress is stored across two 8-bit RAM bytes: screen page 0x006D and screen pixel offset 0x0086. When an agent steps 1 pixel left of spawn (x = 0), the 8-bit page byte underflows from 0x00 to 0xFF (255) with offset 0xFF (255). An unsigned decoder computes 256 * 255 + 255 = 65,535 (more than 20 times the World 1-1 flagpole at x = 3,161), rewarding the policy for walking backward into the left wall.

Before trusting any agent-submitted training run, we built 68 adversarial fake packages and 51 scorer tamper mutations. Across five evaluator revisions, false-negative slip-through dropped from 39/68 (`checklist_v1`), 24/68 (`v2`), 14/68 (`v3`), and 5/68 (`v4`) to 0/68 slipped through (`68/68` caught, `0` false positives) in checklist_v5 + precheck.py (SHA-256 71d1170b5c71732d...). Click any row below to inspect how v1 vs v5 handles the attack.

When adapting our Atari Pong policy from deterministic Regime A (`sticky = 0.0`) to stochastic Regime B (`sticky = 0.25`), continuing from trained weights beats matched 2M-step scratch training on B by +21.25 points (+10.60 vs -10.65, 95% CI [+18.750, +23.500]). However, doubling continuation training from +1M to +2M steps adds only +0.05 points on B while degrading Regime A retention from 19.15 to 17.55 (a -1.80 drop whose one-sided 95% lower bound -3.35 breaches our -3.0 retention gate).

Across 5,600 fresh Pokemon TCG games (1,002,605 verified actions), we wired a learned 88-feature Ridge scorer and a 90-feature relational card-copy scorer directly into live action selection. The learned weights genuinely changed 266, 812, and 868 live decisions, but all three pre-registered confirmation gates failed: Study 1 gained +5.625 pp but its 95% lower CI bound hit -0.003125 pp; Study 2 shrank to +1.00 pp [-2.125, +4.125] over 2,400 games; Study 3 scored -1.50 pp [-4.667, +1.667]. Root-cause diagnosis uncovered 58 feature-aliasing groups (236/511 rows) and proved that uniform 1/K exploration (mean menu K = 10.865, max weight 27.0) yields only 58/511 exact parent matches (11.35%) and an inverse-propensity Effective Sample Size of ESS = 48.06 (below the 100-match gate).

On identical state probes (SHA-256 f830f1d96ebbd2a8...), evaluating the same checkpoint on CPU vs RTX 5080 CUDA produces a maximum action-probability drift of 1.800060272216797e-5 (`0.0` drift across repeated CUDA reloads, with 8/8 probes identical across CPU and CUDA). In Stable-Baselines3 PPO, the rollout buffer holds n_steps * n_envs transitions. Raising n_envs from 8 to 20 with n_steps = 256 increases that buffer from 2,048 to 5,120 transitions, so complete rollout collection and training cycles occur 2.5 times less often per million transitions. This alone does not establish fewer optimizer steps: those also depend on minibatch size, optimization epochs and early stopping. The original training configuration has not been verified here, so its optimizer-step frequency remains unresolved. Furthermore, importing torch inside 20 Windows spawn workers reserves ~1.6 GB of CUDA DLL virtual memory per process (~32 GB pagefile). Isolating Ctypes/LaiNES workers in a torch-free module (mario_worker.py) eliminates the pagefile exhaustion.

Stable-Baselines3 PPO parameter definitions distinguish rollout size, minibatch size and optimization epochs.