TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
A project published on GitHub runs a neural-network game simulator and an agent on a Raspberry Pi Pico 2 W. The simulator predicts game frames from the current frame and an action, while the agent chooses actions from those generated frames; the project reports high exact-frame accuracy on several games, with weaker results on more complex scenes.
A developer has demonstrated two small neural networks running together on a Raspberry Pi Pico 2 W to simulate and play five grid-based games, with the simulator predicting each next frame from the current frame and the agent’s chosen action. The project, documented in the tinyworldformer GitHub repository, explores whether a memory-efficient, repeatedly applied transformer block can make a usable game world model small enough for a microcontroller.
The system has two parts. Its world model receives a frame and an action, then predicts the next frame and a reward. Its actor network sees the generated frame and selects an action. On the board, the two networks do not consult the underlying game code: the actor observes the model’s output, and the world model receives the actor’s actions. The project says both networks run on core 1 of the RP2350, while core 0 handles USB and the display.
The world model is a looped transformer: one transformer block, with shared weights, is applied between one and four times rather than using a stack of separate blocks. The developer says weight reuse was the reason to test this approach on a microcontroller. The world model ranges from 26,000 to 131,000 parameters, depending on the game, and uses quantized int8 weights; the actor is a smaller, float32 multilayer perceptron with 9,000 to 16,000 parameters. On the Pico, the project uses two transformer loops to balance prediction accuracy and speed.
The five games are Breakout, Doom-lite, Flappy Bird, Pong and Pac-Man lite. The report measures “exact frames” as the share of held-out transitions in which every predicted cell matches the real game’s frame; transitions immediately following random events are excluded. It reports 99.99% for Breakout, 100% for Flappy Bird, 99.45% for Pong, 83.6% for Doom-lite and 86.4% for Pac-Man lite at the selected two-loop setting. These are the project’s own evaluation results, not an independent assessment.
A World Model on a Microcontroller
The demonstration tests how far a compact device can go beyond running a fixed game or a single inference task. Here, the board runs a learned simulator and a decision-making agent in a feedback loop: the agent acts on a predicted world, and those actions shape the next prediction. That arrangement makes the project relevant to researchers and developers exploring on-device learning, simulated environments and small-scale AI systems.
The results also show a clear limit to what a single accuracy figure can convey. Exact-frame scores are high for simpler grid games, while the more visually detailed Doom-lite and Pac-Man-lite models score lower. The author notes that Doom-lite’s 768-cell 3D view counts a frame as wrong if even one shaded wall cell differs, although the underlying game state is usually right. The reported scores therefore describe a specific frame-matching test, not how enjoyable or reliable every game is in play.
The board can also run a streaming reinforcement-learning agent that learns on the device, according to the project. A player can take control by pressing a button. Those features make the work a practical demonstration of a small, interactive model, though the source does not establish performance on tasks beyond these five games.
Products Worth Considering
LG UltraGear 27-Inch QHD OLED Gaming Monitor, 240Hz, 0.03ms, G-SYNC Compatible, FreeSync Premium Pro, HDMI 2.1
Bose QuietComfort Wireless Bluetooth Headphones with Active Noise Cancellation, Mic, USB‑C Charging, Deep Bass, 24‑Hour Battery – White Smok
Samsung 27-Inch Odyssey QD-OLED G8 4K 240Hz Gaming Monitor
As an affiliate, we earn on qualifying purchases.
How the Pico Game Loop Works
The project chose the Raspberry Pi Pico 2 W because, its author says, it is inexpensive and has a floating-point unit. The games were written in NumPy so the developer could compare model-generated frames with the real game logic. Each game fits its state into a small visual grid; information that might otherwise be hidden has to appear in the frame. For example, Breakout encodes the ball’s heading in its token, and Doom-lite includes a minimap.
Before prediction, each frame is divided into patches: the patch size varies by game, producing 36 to 80 tokens. The model combines learned cell and position embeddings with an action vector. It then applies the same attention-and-MLP block repeatedly, adding the input back before each pass. Training supervises the output after each pass, allowing the number of loops at runtime to trade computation for accuracy.
Converting model scores back into a playable frame also involves rules. Background cells are chosen independently, while unique objects such as a ball or player are placed through a single choice across possible positions. Following random events, the decoder samples among near-best choices to represent events such as a new serve or respawn. The source provides code and instructions for reproducing the project, as well as a way to play the real games in a terminal.
“I wanted to see if a looped world model could be small enough to run on a microcontroller.”
— The tinyworldformer project author
Limits of the Reported Accuracy
The source is a developer’s project report, and the supplied material does not include an independent replication or a separate review of its measurements. It also does not provide a measured frame rate, power consumption, memory use during operation, or a consistent latency figure. Those details would help readers judge how responsive and resource-efficient the system is on the board.
The exact-frame metric has defined exclusions and differs in meaning across games. Frames immediately after random events are left out because the model samples those outcomes, and a one-cell visual mismatch counts as a failed frame even when most of the state is correct. The report says Doom-lite’s game state is almost always right despite its lower exact-frame score, but the supplied material does not give a separate numerical measure for that claim. It is also unclear how performance changes across training runs or game configurations.
The project describes an on-device streaming agent that can learn from scratch, but the provided figures concern world-model frame prediction and do not quantify that agent’s learning speed or final playing performance. The source does not specify a publication date, so the project’s current status and any later revisions are not established here.
Reproduction and Further Testing
The repository provides source code and reproduction information, including firmware-related instructions and ways to run the games outside the board. Readers can inspect the model, compare generated frames with the NumPy game implementations and test the reported setup on compatible hardware. The author also describes swapping a pretrained actor for a streaming agent that learns on the Pico.
No follow-up release, independent benchmark or planned next milestone is identified in the supplied report. The next useful evidence would include reproducible measurements of runtime speed and resource use, alongside tests of the learning agent and the model’s performance across repeated runs. Until then, the reported results establish a developer demonstration on five small games, rather than a general measure of what world models can do on microcontrollers.
Key Questions
What does the tiny world model do?
It predicts a game’s next frame and reward from the current frame and an action. A separate actor network selects actions using the generated frames.
Which games does the Pico project run?
The report covers Breakout, Doom-lite, Flappy Bird, Pong and Pac-Man lite, each implemented as a small grid-based game.
How accurate are the predictions?
The project reports exact-frame scores ranging from 83.6% for Doom-lite to 100% for Flappy Bird at its two-loop setting. The metric requires every cell to match and excludes transitions immediately after random events.
Does the system learn while running on the board?
The author says a streaming reinforcement-learning agent can learn on the Pico. The supplied report does not quantify its learning speed or final game-playing performance.
Are the results independently verified?
The figures are reported by the project’s author on GitHub. The supplied source material does not identify an independent replication or evaluation.
Source: github.com
