
miHoYo's AI can now play Honkai: Star Rail on its own: it reads only the screen, never the game's memory, and clears 49.7% of 40 minigame tasks it had never seen before.
What the agent actually did in-game
The footage is deliberately unglamorous. No stunt clips: Gamer pushed main story content in Jarilo-VI, the Xianzhou Luofu and Penacony, following NPCs, clearing mine areas and fighting boss encounters.
The evaluation design matters more than the footage. The team used two side quests and seven minigames in Honkai: Star Rail as test environments, including spatial puzzles such as "Abacus Circuitry", "Aurumaton Gatekeeper"-style navigation tasks and "Dream Ticker". None of it, the team says, was in the training set.
The task is finishing the job, not winning a fight
Gamer is not optimised to win a single battle. It is optimised to complete a goal that may run twenty minutes and dozens of interactions, where one bad decision poisons everything after it. That is precisely where LLM agents usually fall apart.
The official framing is careful
The first reaction online was "cheats are coming". The team pre-empted it: Gamer is academic research into general-purpose agents for interactive environments, with potential uses in game QA, accessibility and human-computer interaction research. They explicitly do not endorse cheating, unauthorised access or exploit use.
Five frames in, thirty actions out
The technical core is simple to state and hard to build. Gamer does not read game memory and does not call any game-specific API. Its only input is a screen capture at 5 Hz. Its output is native keyboard state and mouse movement at an effective 30 Hz.
To the game client, it looks like a person at a desk.
Vision-only is why it generalises
Memory-reading agents score better in a lab and die outside it: a new patch, a new resolution or a shifted UI breaks them. Vision-only is clumsy and slow, but it travels. According to GameSpark, the base model is Alibaba's open-source Qwen3-VL (8B).
The team continued pre-training on more than 10,000 hours of cross-game keyboard and mouse trajectories, mapping heterogeneous control schemes into a single keyboard-and-mouse representation, then applied sparse-thinking instruction tuning to cut redundant computation, and finally a temporally aware reinforcement learning stage for long-horizon play. The point was never to teach the model more about games. It was to teach it where to put its hands.
From 1,306 ms to 65 ms
The bottleneck in real-time play is not model size, it is inference latency. The team built an engine called Streaming-vLLM and, on NVIDIA H200 hardware, cut end-to-end request latency from 1,306 ms to 65.12 ms — roughly 20x.
For context, professional esports players react in about 150 ms; casual players sit above 250 ms. On latency alone, the agent is faster than a human. But an H200 is a datacentre card serving many simulated environments at once. The "real time" in this paper is a server rack's real time, not your desktop's.
The 49.7% is the real story
On 40 unseen minigame tasks, the reinforcement-learned Gamer-RL averaged a 49.7% success rate, better than the un-tuned generalist model. Most coverage skips this figure. It is the most honest line in the paper.
Half-failure means the agent can follow an arrow, take a prompt and press the button. The moment a task demands genuine spatial reasoning, multi-step trial and error, or remembering a clue it saw three minutes ago, it starts dropping the ball.
Main quests pass, minigames do not
Main quests are heavily guided: NPCs wait, markers point the way. Minigames offer none of that. The pattern says the current generation of agents is good at complying and bad at understanding. At 49.7%, Gamer cannot even be trusted with daily chores — set it on your Spiral Abyss runs and you will be watching it retry twice.
QA teams should worry, not cheat sellers
The cheat industry will not touch this. It is expensive, latency-bound to a datacentre, and its behavioural signature is easy for anti-cheat to flag. Using a vision-language-action model to cheat is a terrible trade.
The people being cornered are QA engineers. A live-service mobile game needs dozens to hundreds of flows regression-tested every patch; the work is expensive, repetitive and easy to miss. An agent that can finish a main quest line and reproduce a crash path is worth far more to a studio than to a player.
Games are becoming data factories
There are not many places on earth that reliably produce high-quality long-horizon interaction data. Games are the cheapest one. Use the game as a training ground, feed the agent back into QA and the next product — that chain is worth more than "AI plays your dailies for you".
My read
The takeaway is not 49.7%. It is that the "5 Hz vision + 30 Hz keyboard and mouse + no memory access" route now works: a generalist agent can walk into any 3D world without the developer's cooperation. That is an opportunity for games and a warning for every other software category.
Will players be replaced? Not soon. The risk lands in two or three years, if the capability is distilled to the edge and the cost becomes tolerable. The first thing to change will not be how games feel. It will be how "account behaviour" is judged.
Can Gamer actually run my dailies
A: No. Its success rate on 40 unseen minigame tasks is 49.7%, and main quests require heavy guidance; for repeatable chores such as daily commissions or Spiral Abyss, retry cost exceeds doing it by hand.
Will this be used for cheating
A: Unlikely in the near term. It needs H200-class compute, cloud inference and low latency, and its input/output signature differs visibly from a human's. The team also explicitly opposes cheating and unauthorised access.
When can regular players use it
A: There is no timeline. Only early pre-training files were released; data and weights remain closed under a non-commercial CC BY-NC-SA 4.0 licence, which makes this a capability test rather than a product tease.
Sources were verified against the arXiv listing and the outlets cited below. More industry briefings are published on GameHub's news desk at https://www.gghubs.com/zh/news.
Would you hand your daily stamina grind to a 49.7% agent, or would you rather grind it yourself?