Cheap Chinese Coding Models: What to Choose in September 2026
A developer of a web project asked a simple question: which inexpensive model to hook up for programming with pay-per-token billing that also handles the job well. We collected prices and independent measurements for Chinese models as of September 14, 2026, and then tested nine models on the same tasks with hidden tests — from version sorting to fixing a small repository with an agent.
Short answer. Among the cheap ones, the first to try are DeepSeek V4.1-Flash and GLM-5.3-Flash: both cost around $0.15 per million input tokens and $0.50—0.60 per million output tokens, both sit near the top of independent rankings, and both connect to Claude Code, Cline and OpenCode out of the box. If the tasks are heavy and the budget allows paying several times more, GLM-5.3 is noticeably stronger. The surprise is Hy3 from Tencent: with reasoning on our tasks it matched GLM-5.3-Flash.
Our own measurement added to the rankings the main thing — how the models behave on the same tasks. Nine models solved ten single-answer tasks in Python, PHP and JavaScript and fixed two small repositories with an agent. GLM-5.3 did best — 20 out of 20 on the first try, but its task costs around eight cents. Among the cheap ones, GLM-5.3-Flash leads (19 out of 20 for 0.4 cents) and Tencent's Hy3 with reasoning (19 out of 20 for 1.2 cents), while DeepSeek V4.1-Flash without reasoning solves 18 out of 20 in two seconds and three hundredths of a cent — faster and cheaper than all. Xiaomi's models, the most popular on OpenRouter, turned out weaker than the rest on our tasks.
How much it costs
Prices per million tokens from official pages. For programming, the third column matters most: an agent rereads the same files dozens of times, and most of the input is repeats that the provider takes from cache.
| Model | Input | Cached input | Output | Open weights |
|---|---|---|---|---|
| DeepSeek V4.1-Flash | $0.15 | $0.003 | $0.60 | yes, MIT |
| GLM-5.3-Flash (Z.ai) | $0.15 | $0.03 | $0.50 | yes, MIT |
| Qwen3.8-Flash (Alibaba) | $0.15 | cache discount available | $0.47 | yes |
| MiMo-V2.5 (Xiaomi) | $0.14 | $0.0028 | $0.28 | yes |
| MiMo-V2.5-Pro (Xiaomi) | $0.435 | $0.0036 | $0.87 | yes, MIT |
| Hy3 (Tencent) | $0.132 | $0.033 | $0.528 | yes, Apache 2.0 |
| MiniMax-M3 | $0.30 | $0.06 | $1.20 | yes |
| GLM-5.3 (Z.ai) | $1.40 | $0.26 | $4.40 | yes |
| Kimi K3 (Moonshot) | $3.00 | $0.30 | $15.00 | yes |
DeepSeek has a quirk: the listed prices apply off-peak, while on weekdays from 01:00 to 04:00 and from 06:00 to 10:00 UTC everything costs twice as much. In Moscow time that is 4:00—7:00 and 9:00—13:00, meaning the peak falls right in the morning of the workday.
What independent measurements show
Manufacturers publish impressive numbers, but those are their own runs. The gap with independent ones can be huge: according to Alibaba, Qwen3.6-27B solves 77.2% of SWE-bench Verified tasks, but on fresh SWE-rebench tasks, which the model could not have seen during training, it solves 31.2%. That is why only third-party measurements are given below.
Artificial Analysis — an overall index across a set of tests, including programming and terminal work (version 4.3):
| Model | Index | Speed, tokens/s |
|---|---|---|
| GLM-5.3 | 45 | 72 |
| Kimi K3 | 44 | 37 |
| GLM-5.3-Flash | 42 | 109 |
| DeepSeek V4.1-Flash | 40 | 212 |
| Qwen3.8-Flash | 40 | 51 |
| MiniMax-M3 | 30 | 107 |
| MiMo-V2.5-Pro | 26 | 43 |
| Hy3 | 26 | 91 |
| for comparison: Claude Sonnet 5 | 38 | 76 |
| for comparison: GPT-5.6 Luna | 38 | 120 |
Three cheap "flash" models rank above Claude Sonnet 5 in the index, and DeepSeek V4.1-Flash responds faster than anything else in the table — twice as fast as GLM-5.3-Flash and four times as fast as Qwen3.8-Flash. For an agent that takes dozens of steps in a row, speed translates directly into minutes of waiting.
SWE-rebench — fixing real bugs in real repositories, 111 tasks that appeared between May 15 and July 1, 2026. It has something other rankings lack: how much money goes into one task.
| Model | Solved | $ per task | $ per solved |
|---|---|---|---|
| GLM-5.2 | 62.9% | 1.40 | 2.23 |
| MiniMax M3 | 47.2% | 0.95 | 2.01 |
| MiMo-V2.5-Pro | 46.5% | 0.10 | 0.22 |
| DeepSeek V4-Pro | 40.2% | 0.15 | 0.37 |
| Qwen3.6-27B | 31.2% | 0.62 | 1.99 |
| for comparison: Claude Opus 5 | 63.4% | 3.47 | 5.47 |
| for comparison: Claude Sonnet 5 | 56.8% | 1.43 | 2.52 |
The last column is our own calculation: the price of a task divided by the share solved. The fresh models (DeepSeek V4.1-Flash, GLM-5.3, Qwen3.8) are not in this window yet; their tasks will land in the next one. But the picture is visible: the senior GLM line nearly catches Claude Opus 5 in share solved at a price two and a half times lower, while MiMo-V2.5-Pro solves less, but its solved task costs a little over twenty cents. MiniMax M3 is cheap per token but burns a lot of them — 13.9 million per task versus 4—5.5 million for its neighbors in the table, so in the end it comes out expensive.
Arena WebDev — ratings from people who compare web interfaces built by models blind (data as of September 11). Among Chinese models, Qwen3.8-Max ranks highest (4th place) and Kimi K3 (5th), Qwen3.8-Flash is 9th, GLM-5.3 and DeepSeek V4.1-Flash share 15th—16th, GLM-5.3-Flash is 17th. For comparison, Claude Opus 5 is third.
What people actually use. By token share in the "programming" category on OpenRouter in September, the top spots go to MiMo-V2.5 (15.4%), GPT-5.6 Luna (14.4%) and GLM-5.3-Flash (12.8%); DeepSeek V4-Flash and V4.1-Flash together add another 12.1%. Developers vote with their money for exactly the cheap models.
Our measurement: nine models on the same tasks
All models received identical tasks: ten single-answer tasks with hidden tests and two agentic assignments where the model fixes a small repository on its own, reading files and running tests. Each task was solved twice; if the tests failed, the model saw their output and could correct itself once. DeepSeek was connected directly, the other models through OpenRouter. How the tasks and checks are structured is described in detail at the end of the article, along with full tables for each task.
| Model and mode | Connection | First try | With test hint | Time per task | Tokens in response | Task price | Agentic tasks | Agentic run price | Responses from 4-bit builds |
|---|---|---|---|---|---|---|---|---|---|
| DeepSeek V4.1-Flash, no reasoning | DeepSeek API | 18 of 20 | 19 of 20 | 1.9 s | 446 | $0.0003 | 4 of 4 | $0.0022 | — |
| DeepSeek V4.1-Flash, reasoning high | DeepSeek API | 18 of 20 | 19 of 20 | 58 s | 18,348 | $0.011 | 4 of 4 | $0.0025 | — |
| DeepSeek V4.1-Flash, reasoning max | DeepSeek API | 18 of 20 | 20 of 20 | 83 s | 24,524 | $0.015 | 4 of 4 | $0.0032 | — |
| DeepSeek V4.1-Flash, high, 32K cap | DeepSeek API | 17 of 20 | 20 of 20 | 40 s | 11,829 | $0.0075 | 4 of 4 | $0.0027 | — |
| DeepSeek V4.1-Flash, max, 32K cap | DeepSeek API | 18 of 20 | 20 of 20 | 57 s | 15,572 | $0.010 | 4 of 4 | $0.0035 | — |
| DeepSeek V4-Pro, high, 32K cap | DeepSeek API | 16 of 20 | 19 of 20 | 232 s | 16,881 | $0.043 | 4 of 4 | $0.012 | — |
| DeepSeek V4.1-Flash, no reasoning | OpenRouter | 17 of 20 | 18 of 20 | 8.3 s | 461 | $0.0007 | 4 of 4 | $0.0059 | 6% |
| DeepSeek V4.1-Flash, with reasoning | OpenRouter | 19 of 20 | 19 of 20 | 63 s | 6,280 | $0.0078 | 4 of 4 | $0.0063 | 13% |
| GLM-5.3-Flash, with reasoning | OpenRouter | 19 of 20 | 20 of 20 | 246 s | 8,971 | $0.004 | 4 of 4 | $0.0045 | 6% |
| GLM-5.3, with reasoning | OpenRouter | 20 of 20 | 20 of 20 | 228 s | 19,045 | $0.077 | 4 of 4 | $0.037 | 2% |
| Qwen3.8-Flash, no reasoning | OpenRouter | 16 of 20 | 19 of 20 | 9.5 s | 815 | $0.0006 | 4 of 4 | $0.0028 | 32% |
| Qwen3.8-Flash, with reasoning | OpenRouter | 18 of 20 | 19 of 20 | 448 s | 27,211 | $0.013 | 4 of 4 | $0.0039 | 44% |
| MiMo-V2.5, no reasoning | OpenRouter | 10 of 20 | 16 of 20 | 21 s | 952 | $0.0005 | 4 of 4 | $0.0015 | 0% |
| MiMo-V2.5, with reasoning | OpenRouter | 15 of 20 | 17 of 20 | 220 s | 9,055 | $0.0034 | 4 of 4 | $0.0031 | 0% |
| MiMo-V2.5-Pro, with reasoning | OpenRouter | 13 of 20 | 18 of 20 | 177 s | 7,269 | $0.01 | 4 of 4 | $0.011 | 0% |
| Hy3, no reasoning | OpenRouter | 13 of 20 | 16 of 20 | 7.0 s | 524 | $0.0006 | 4 of 4 | $0.0033 | 0% |
| Hy3, with reasoning | OpenRouter | 19 of 20 | 20 of 20 | 304 s | 20,156 | $0.012 | 4 of 4 | $0.0063 | 0% |
| MiniMax-M3, with reasoning | OpenRouter | 15 of 20 | 18 of 20 | 122 s | 20,296 | $0.027 | 4 of 4 | $0.018 | 13% |
"Task price" is the average actual cost of a task including the correction attempt, "time" is the average time to the first response. The time of models through OpenRouter depends on whose server answered, and with slow resellers it is many times greater than with the developer itself. Comparing models by it should be done cautiously: the same DeepSeek through OpenRouter responded four times slower than directly.
What the table shows.
- GLM-5.3 — the only model without a single error: 20 out of 20 on the first try. But also the most expensive in the measurement: about eight cents per task, twenty times more than GLM-5.3-Flash.
- GLM-5.3-Flash nearly catches the senior model — 19 out of 20 on the first try and all 20 with the test hint — at a price of 0.4 cents per task. One downside: its reasoning cannot be turned off, and through OpenRouter the response took an average of four minutes.
- Hy3 from Tencent — the surprise of the measurement: with reasoning, 19 out of 20 on the first try for 1.2 cents. Without reasoning, the same model solves only 13 out of 20 — unlike DeepSeek, the reasoning mode makes a big difference for it.
- DeepSeek V4.1-Flash — the fastest and cheapest: without reasoning 18 out of 20 in two seconds, with reasoning through OpenRouter — 19 out of 20.
- Qwen3.8-Flash — the middle: 18 out of 20 with reasoning and 16 out of 20 without. With reasoning the response took the longest of all, seven and a half minutes, and from a third to nearly half of the responses came from a reseller's 4-bit build.
- MiniMax-M3 — 15 out of 20 on the first try at a price of 2.7 cents per task: expensive for such a result.
- MiMo-V2.5 and MiMo-V2.5-Pro from Xiaomi — the weakest: from 10 to 15 out of 20 on the first try. This diverges both from their popularity on OpenRouter and from their good price per solved task on SWE-rebench: on our tasks they more often than others made mistakes in the details of the condition.
Everyone solved the agentic tasks. Every model in every mode fixed both repositories in both runs. These tasks could not separate the models, but they did show the difference in price: a run cost from 0.15 cents for MiMo-V2.5 to 3.7 cents for GLM-5.3. And one more detail: through OpenRouter only 37—78% of the input came from cache versus 82—86% with a direct connection to DeepSeek. A single agent's requests land with different providers, and the cache is lost, and over a long agent run that is noticeable money.
DeepSeek in more detail: which mode to enable
Other people's rankings answer the question "which model is stronger" but not "which mode to enable and how much it will cost." DeepSeek, which we use ourselves, we ran in all modes, including the outgoing V4-Pro, directly through its API and on the same tasks as the other models. The tasks and conditions are described at the end of the article.
| Mode | First try | With test hint | Time per task | Tokens in response | Task price |
|---|---|---|---|---|---|
| Flash, no reasoning | 18 of 20 | 19 of 20 | 1.9 s | 446 | $0.0003 |
| Flash, reasoning high | 18 of 20 | 19 of 20 | 58 s | 18,348 | $0.011 |
| Flash, reasoning max | 18 of 20 | 20 of 20 | 83 s | 24,524 | $0.015 |
| Flash, high, 32K response cap | 17 of 20 | 20 of 20 | 40 s | 11,829 | $0.0075 |
| Flash, max, 32K response cap | 18 of 20 | 20 of 20 | 57 s | 15,572 | $0.010 |
| V4-Pro, high, 32K response cap | 16 of 20 | 19 of 20 | 232 s | 16,881 | $0.043 |
Without reasoning, the model handled almost everything, and that is the main takeaway for everyday work. It only stumbled on version sorting: it checked the string with a regular expression with $ at the end, and in Python such a pattern lets through a string with a trailing newline, so "1.2.3\n" was recognized as a valid version. In one run the model corrected itself after seeing the failing test, in the other it repeated the same mistake. The error is classic, and a human reviewer often misses it too.
Reasoning costs 35—50 times more and is 30—45 times slower, and solves no more. In the first run we set a response cap of 32,768 tokens — similar limits are set by default in many agents — and four of the five failed first attempts in high and max modes turned out not to be code errors at all: the model cut off mid-reasoning without writing a single line. If you enable reasoning, raise the cap: DeepSeek allows up to 384 thousand tokens.
In the second run the cap was 131 thousand tokens. The cutoffs almost disappeared, but the result did not improve either: on the first try the same 18 out of 20 as without reasoning. The high mode wrote an average of 18 thousand tokens per task, max — 24 thousand, and once high exhausted all 131 thousand over five minutes on a thirty-line function and produced no answer. Both modes also stumbled on the PHP task, which the model solved immediately without reasoning: the filter by Russian names returned an empty array for them. On this number of tasks, a difference of one or two attempts is within the range of randomness, but the difference in price and time is not.
All modes handled the agentic tasks: both repositories fixed in each of the four runs, without a single erroneous tool call and without attempts to fudge the tests instead of the code.
| Mode | Solved | Steps | Run time | Cached input | Run price |
|---|---|---|---|---|---|
| Flash, no reasoning | 4 of 4 | 8 | 15 s | 86% | $0.0022 |
| Flash, reasoning high | 4 of 4 | 5 | 16 s | 82% | $0.0027 |
| Flash, reasoning max | 4 of 4 | 5 | 20 s | 82% | $0.0035 |
| V4-Pro, reasoning high | 4 of 4 | 5 | 57 s | 84% | $0.0117 |
Without reasoning, the model acts differently: it takes eight steps instead of five and makes more tool calls, trying and correcting based on test output — and still fits into the same time and comes out the cheapest of all. And the penny-level prices are explained by the cache: the agent rereads 82—86% of the input tokens, and DeepSeek charges $0.003 per million for them.
We managed to test V4-Pro on its last day. DeepSeek announced that as of September 14 requests to it are redirected to V4.1-Flash, but during the measurement Pro was still answering with its own model — this is visible from the service fingerprint in the response. It solved slightly less — 19 tasks out of 20 versus 20 for Flash in the same mode — and a task took it an average of 232 seconds versus 40 and cost $0.043 at Pro prices versus $0.0075. All five of its failed attempts were the same cutoff at 32 thousand tokens, and each had to be waited on for more than seven minutes: Pro writes its response noticeably slower. It solved the agentic tasks the same as Flash, but three times longer and four times more expensive. So DeepSeek's move to a single model for programming takes nothing away.
Subscription or pay-per-token
Almost all manufacturers except DeepSeek have "coding plans" with a monthly fee and a request limit per five-hour window: GLM Coding Plan from $18 per month, Token Plan from Xiaomi from $6, from MiniMax from $22, from Alibaba only Pro remains at $50 and without Qwen3.8 in the model list. The savings are real if you sit in an agent every day. But subscriptions have two limitations: they may only be used from supported tools (Claude Code, Cline, OpenCode and the like), not from your own scripts, and the exact limits are often not published. For irregular work and for your own code, pay-per-token is simpler.
How to connect to Claude Code
All three top candidates have an Anthropic API-compatible endpoint, so Claude Code works with them without proxies — the environment variables ANTHROPIC_BASE_URL and ANTHROPIC_AUTH_TOKEN are enough:
- DeepSeek:
https://api.deepseek.com/anthropic, modeldeepseek-flash[1m](with a million-token context). DeepSeek's official instructions setCLAUDE_CODE_EFFORT_LEVEL=max. - Z.ai (GLM):
https://api.z.ai/api/anthropic. By default all roles get GLM-5.3-Flash; the senior model must be specified manually. - Xiaomi (MiMo):
https://api.xiaomimimo.com/anthropic.
For Cline, Roo and other tools with an OpenAI-compatible API, the same providers have regular endpoints.
About paying from Russia
According to user reports, Russian cards are accepted by neither DeepSeek, nor Z.ai, nor Alibaba Cloud, and LongCat from Meituan takes only Alipay and WeChat Pay. The usual routes are a foreign card or OpenRouter: it lets you in from Russia, accepts cryptocurrency with a fee of about 5%, and sells access to most of the models in the table. With OpenRouter, watch which provider is serving the model: the cheap ones sometimes have cut-down builds and tool calling does not work, while DeepSeek's price through its own node matches the official one.
And on your own hardware?
All the models in the table are released with open weights, but not all can realistically be run at home. DeepSeek V4.1-Flash is 552 billion parameters, GLM-5.3-Flash is 320 billion; they need a server. Of the fresh ones, Qwen3.8-27B under the Apache 2.0 license is suitable for a single GPU: in the 4-bit Q4_K_M build it weighs 16.5 GB and fits into a 24 GB GPU along with context. On Arena WebDev it ranks 18th, right behind GLM-5.3-Flash — very decent for a local model, though it is far from the cloud "flashes" in agentic work.
Bottom line
- DeepSeek V4.1-Flash through the DeepSeek API — the default choice: the fastest and cheapest, on our tasks without reasoning it solved 18 out of 20 on the first try. Reasoning is worth enabling only on tricky algorithms and with a raised response cap. Heavy sessions are more profitable to run off-peak — on weekdays after 13:00 Moscow time.
- GLM-5.3-Flash — the best result among the cheap ones: 19 out of 20 on the first try at a price of 0.4 cents per task. Worth taking if quality matters more than speed. Through OpenRouter the response took minutes; with Z.ai itself, according to Artificial Analysis measurements, the model produces around 110 tokens per second. There is also a subscription from $18 per month.
- GLM-5.3 — when maximum reliability is needed: the only model without errors, but twenty times more expensive than the junior one.
- Hy3 from Tencent — an interesting alternative: with reasoning it is no worse than GLM-5.3-Flash at 1.2 cents per task, without reasoning it is no good.
- Qwen3.8-Flash — a workable option, but paying for Alibaba Cloud from Russia is difficult, and through OpenRouter a noticeable share of responses comes from cut-down builds.
- MiniMax-M3 and Xiaomi's models fell behind the rest on our tasks. Kimi K3 at $15 per million output tokens is not among the cheap ones and was not included in the measurement.
The limitations of the test are worth keeping in mind. Twenty attempts per model is a small sample, and a difference of one or two tasks is within the range of randomness. Through OpenRouter it was impossible to choose the provider, so some models answered from cut-down builds. The agentic tasks turned out easy for everyone. On a real project, the context of each agent step is dozens of times larger, so cache price and response speed weigh more there than in our tables.
How the measurement was conducted: conditions, tasks, full results
This section is for those who want to check the conclusions or repeat the experiment. All requests were sent on September 14, 2026.
How the models were connected. DeepSeek — directly through the DeepSeek API, outside peak hours. The other models — through OpenRouter with a fal.ai key: OpenRouter itself does not open from Russia, but fal passes requests through to it. In no-reasoning mode the temperature was 0, in reasoning modes — the model defaults. The response cap without reasoning was 8 thousand tokens for DeepSeek and 16 thousand for models through OpenRouter, with reasoning — 131 thousand for DeepSeek and 96 thousand for the rest. The first DeepSeek run went with a cap of 32,768 tokens — that is exactly where reasoning cut off, and it is listed separately in the summary tables.
How success was counted. The model receives the task condition and a system prompt: "answer with a complete solution in a single code block, standard library only, no tests or explanations." The code from the response is run together with hidden tests from a separate unprivileged user with a 60-second limit. If the tests fail, the model sees their output and gets one attempt to correct. "First try" and "with test hint" in the tables are the share of solved attempts. Each task was solved twice in each configuration. Before the measurement, all tasks were verified with reference solutions: the reference must pass the tests, and the buggy source code for the fix tasks must fail.
How the price was counted. For DeepSeek — by the tokens from the API response and official off-peak prices, separately for cache hits. For models through OpenRouter — the actual cost that OpenRouter returns in each response. fal charged $3.75 to the account while the cost of the requests according to OpenRouter data was $3.56 — about 5% on top for intermediation.
Who answered the requests. On OpenRouter, different companies serve the same model, and through fal you cannot choose the provider: fal rejects a request with such a specification. So responses came sometimes from the model developer's servers, sometimes from resellers, including 4-bit builds that can be weaker than the original. What share of responses came from such builds is shown in the results table: where it is large, a direct connection to the developer may show the model in a better light.
Ten single-answer tasks
| Task | Language | What it checks |
|---|---|---|
| Parsing durations like "1h30m" | Python | strict grammar: unit order, no space between number and unit, exact decimal arithmetic — "0.1h" must give exactly 360 seconds, while with binary fractions you get 360.00000000000006 |
| Sorting versions by SemVer 2.0.0 | Python | literal adherence to the spec: numeric and alphanumeric prerelease identifiers, build metadata, no leading zeros, a string with a trailing newline |
| SQL: top two buyers in each country | Python and SQLite | window functions, the calendar year boundary, excluding refunds, buyers without a country, and exactly two rows per country when sums are equal — ROW_NUMBER, not RANK |
| Fix working-day counting | Python | three bugs in ready-made code (sign, range boundary, "or" instead of "and") and a speed requirement: a range of thousands of years must be computed in under a second |
| JSON Patch per RFC 6902 | Python | all six operations, escaping in JSON Pointer, atomicity (on error the document does not change), comparison by JSON rules where true is not equal to 1 |
| Splitting a sum into shares | Python | largest remainder method, negative sums, precision on numbers up to 1030 without floating point |
| Async execution with a limit | Python, asyncio | start order, no more than the given number of tasks at once, exceptions instead of a result, and correct cancellation of the whole run |
| Token bucket for rate limiting | Python | refill via a fake clock, rejection without deduction, and behavior when the clock goes backward |
| Fix a grouping function | PHP 8.3 | typical PHP traps: array_merge loses numeric keys, strpos returns 0 on a match at the start, strtolower does not understand UTF-8, array_filter by default receives values, not keys |
| LRU cache with a time to live | JavaScript, Node.js 22 | eviction order, has() which must not refresh an entry, removing expired entries before live ones, NaN keys and objects |
Two agentic tasks
The model receives a repository description and four tools: list files, read a file, write a whole file, and run tests. Test files are write-protected, and there are no more than 30 steps. When the model finishes, the tests are run again, and the test files are verified by checksum.
- Online store shopping cart, 9 tests. VAT is rounded by banker's rules instead of ordinary ones, the promo code is case-sensitive and does not apply on the last day, re-adding an item overwrites the quantity, an empty cart returns 0 instead of 0.00, the default date is not substituted — and on top of that you need to write a receipt printout from scratch that is consistent with the total.
- Parsing nginx logs, 7 tests. The time zone is lost, the query string ends up in the path, a size of "-" breaks line parsing, the percentile is computed incorrectly and crashes on an empty list, the 5xx error share is counted from 501 and divides by zero, when paths are equal the order is broken — and you need to write a summary report broken down by UTC hour.
Full results by task
Each cell shows how many of two attempts were solved on the first try and how many with the test hint ("1/2" — one immediately, the second after the hint). For agentic tasks — how many runs ended with passing tests.
| Model and mode | duration | SemVer | SQL | working days | JSON Patch | sum shares | asyncio | token bucket | PHP | JS LRU | cart | nginx logs |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek V4.1-Flash, no reasoning | 2/2 | 0/1 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2 of 2 | 2 of 2 |
| DeepSeek V4.1-Flash, reasoning high | 1/1 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 1/2 | 2/2 | 2 of 2 | 2 of 2 |
| DeepSeek V4.1-Flash, reasoning max | 2/2 | 1/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 1/2 | 2/2 | 2 of 2 | 2 of 2 |
| DeepSeek V4.1-Flash, high, 32K cap | 1/2 | 1/2 | 2/2 | 2/2 | 1/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2 of 2 | 2 of 2 |
| DeepSeek V4.1-Flash, max, 32K cap | 2/2 | 2/2 | 2/2 | 2/2 | 0/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2 of 2 | 2 of 2 |
| DeepSeek V4-Pro, high, 32K cap | 2/2 | 2/2 | 2/2 | 2/2 | 0/1 | 2/2 | 0/2 | 2/2 | 2/2 | 2/2 | 2 of 2 | 2 of 2 |
| DeepSeek V4.1-Flash, no reasoning (OpenRouter) | 1/2 | 0/0 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2 of 2 | 2 of 2 |
| DeepSeek V4.1-Flash, with reasoning (OpenRouter) | 1/1 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2 of 2 | 2 of 2 |
| GLM-5.3-Flash, with reasoning (OpenRouter) | 2/2 | 2/2 | 2/2 | 1/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2 of 2 | 2 of 2 |
| GLM-5.3, with reasoning (OpenRouter) | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2 of 2 | 2 of 2 |
| Qwen3.8-Flash, no reasoning (OpenRouter) | 2/2 | 0/2 | 1/1 | 2/2 | 1/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2 of 2 | 2 of 2 |
| Qwen3.8-Flash, with reasoning (OpenRouter) | 2/2 | 1/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 1/1 | 2 of 2 | 2 of 2 |
| MiMo-V2.5, no reasoning (OpenRouter) | 2/2 | 0/0 | 0/2 | 2/2 | 0/1 | 2/2 | 2/2 | 2/2 | 0/1 | 0/2 | 2 of 2 | 2 of 2 |
| MiMo-V2.5, with reasoning (OpenRouter) | 2/2 | 0/0 | 2/2 | 2/2 | 1/1 | 2/2 | 2/2 | 2/2 | 0/2 | 2/2 | 2 of 2 | 2 of 2 |
| MiMo-V2.5-Pro, with reasoning (OpenRouter) | 2/2 | 0/1 | 2/2 | 2/2 | 1/2 | 2/2 | 1/2 | 2/2 | 1/2 | 0/1 | 2 of 2 | 2 of 2 |
| Hy3, no reasoning (OpenRouter) | 0/1 | 1/1 | 2/2 | 2/2 | 0/1 | 2/2 | 2/2 | 2/2 | 0/1 | 2/2 | 2 of 2 | 2 of 2 |
| Hy3, with reasoning (OpenRouter) | 2/2 | 2/2 | 2/2 | 2/2 | 1/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2/2 | 2 of 2 | 2 of 2 |
| MiniMax-M3, with reasoning (OpenRouter) | 1/1 | 2/2 | 2/2 | 2/2 | 1/1 | 2/2 | 1/2 | 1/2 | 1/2 | 2/2 | 2 of 2 | 2 of 2 |
Why tasks remained unsolved
"Cutoff" — the model hit the response cap and did not finish writing the code, "code error" — the code was written but does not pass the tests. Also shown — how many steps the agent took and what share of the input came from cache.
| Model and mode | Failed attempts | of which cutoff at response cap | code error | Agentic: average steps | Agentic: cached input |
|---|---|---|---|---|---|
| DeepSeek V4.1-Flash, no reasoning | 3 | 0 | 3 | 8.0 | 86% |
| DeepSeek V4.1-Flash, reasoning high | 3 | 1 | 2 | 5.0 | 83% |
| DeepSeek V4.1-Flash, reasoning max | 2 | 0 | 2 | 5.2 | 83% |
| DeepSeek V4.1-Flash, high, 32K cap | 3 | 2 | 1 | 5.0 | 82% |
| DeepSeek V4.1-Flash, max, 32K cap | 2 | 2 | 0 | 5.0 | 82% |
| DeepSeek V4-Pro, high, 32K cap | 5 | 5 | 0 | 5.2 | 84% |
| DeepSeek V4.1-Flash, no reasoning (OpenRouter) | 5 | 0 | 5 | 6.0 | 37% |
| DeepSeek V4.1-Flash, with reasoning (OpenRouter) | 2 | 1 | 1 | 6.2 | 51% |
| GLM-5.3-Flash, with reasoning (OpenRouter) | 1 | 0 | 1 | 6.5 | 50% |
| GLM-5.3, with reasoning (OpenRouter) | 0 | 0 | 0 | 7.5 | 45% |
| Qwen3.8-Flash, no reasoning (OpenRouter) | 5 | 0 | 5 | 8.2 | 70% |
| Qwen3.8-Flash, with reasoning (OpenRouter) | 3 | 0 | 3 | 7.5 | 64% |
| MiMo-V2.5, no reasoning (OpenRouter) | 14 | 0 | 14 | 7.0 | 71% |
| MiMo-V2.5, with reasoning (OpenRouter) | 8 | 0 | 8 | 7.8 | 72% |
| MiMo-V2.5-Pro, with reasoning (OpenRouter) | 9 | 0 | 9 | 8.0 | 76% |
| Hy3, no reasoning (OpenRouter) | 11 | 0 | 11 | 8.2 | 78% |
| Hy3, with reasoning (OpenRouter) | 1 | 0 | 1 | 7.5 | 77% |
| MiniMax-M3, with reasoning (OpenRouter) | 7 | 2 | 5 | 9.5 | 63% |
Who actually answered through OpenRouter
The number of responses from each provider and the quantization of its build according to OpenRouter data ("unknown" — the provider did not specify).
- DeepSeek V4.1-Flash, no reasoning: Alibaba (unknown) — 14, GMICloud (fp8) — 10, DeepInfra (fp8) — 6, DeepSeek (own server) — 5, Morph (fp8) — 4, Relace (fp4) — 3, Wafer (unknown) — 2, Modal (unknown) — 2, Venice (fp8) — 1
- DeepSeek V4.1-Flash, with reasoning: Alibaba (unknown) — 16, GMICloud (fp8) — 9, Relace (fp4) — 6, Venice (fp8) — 4, Morph (fp8) — 4, DeepInfra (fp8) — 3, Wafer (unknown) — 2, Parasail (fp8) — 1, DeepSeek (own server) — 1
- GLM-5.3-Flash, with reasoning: StreamLake (fp8) — 21, unnamed (vLLM) — 5, Relace (unknown) — 4, Z.AI (fp8) — 4, Crusoe (fp4) — 2, Wafer (unknown) — 2, GMICloud (fp8) — 2, DeepInfra (fp4) — 1, SiliconFlow (fp8) — 1, Parasail (fp8) — 1, Phala (fp8) — 1, Sail Research (fp8) — 1, NextBit (fp8) — 1, Venice (unknown) — 1
- GLM-5.3, with reasoning: Morph (fp8) — 29, DigitalOcean (unknown) — 3, Reka (fp8) — 3, Venice (unknown) — 2, unnamed (vLLM) — 2, Wafer (unknown) — 2, SiliconFlow (fp8) — 2, Modal (unknown) — 2, Novita (fp8) — 1, AkashML (fp8) — 1, GMICloud (fp8) — 1, AtlasCloud (fp8) — 1, Makora (fp4) — 1
- Qwen3.8-Flash, no reasoning: Alibaba (unknown) — 39, Makora (fp4) — 18
- Qwen3.8-Flash, with reasoning: Alibaba (unknown) — 29, Makora (fp4) — 23
- MiMo-V2.5, no reasoning: Xiaomi (fp8) — 36, DeepInfra (fp8) — 16, StreamLake (unknown) — 4, Novita (fp8) — 2
- MiMo-V2.5, with reasoning: Xiaomi (fp8) — 30, DeepInfra (fp8) — 13, StreamLake (unknown) — 12, Novita (fp8) — 1
- MiMo-V2.5-Pro, with reasoning: DigitalOcean (unknown) — 34, DeepInfra (fp8) — 6, Xiaomi (fp8) — 5, GMICloud (bf16) — 5, Novita (unknown) — 5, StreamLake (unknown) — 3, AtlasCloud (fp8) — 1
- Hy3, no reasoning: Tencent (fp8) — 32, DeepInfra (fp8) — 9, GMICloud (bf16) — 8, Novita (unknown) — 4, Phala (unknown) — 4, AtlasCloud (fp8) — 3
- Hy3, with reasoning: Tencent (fp8) — 32, DeepInfra (fp8) — 10, GMICloud (bf16) — 4, Phala (unknown) — 2, AtlasCloud (fp8) — 2, Novita (unknown) — 1
- MiniMax-M3, with reasoning: Novita (fp8) — 28, DeepInfra (fp8) — 11, CoreWeave (fp4) — 7, Minimax (fp8) — 6, AtlasCloud (fp8) — 4, StreamLake (fp8) — 3, GMICloud (fp8) — 2, ModelRun (fp4) — 1, unnamed (vLLM) — 1
Sources
Prices are from the official pages of DeepSeek, Z.ai, Alibaba Cloud Model Studio, Xiaomi MiMo, Tencent Cloud, MiniMax and Moonshot as of September 14, 2026. Independent measurements: Artificial Analysis, SWE-rebench, Arena WebDev, OpenRouter, "programming" category. The results that manufacturers publish about their own models were deliberately not included in the comparisons — one such number is given only as an example of the discrepancy. Our own measurement: DeepSeek — through its API, the other models — through OpenRouter with a fal.ai key.
