openai/gpt-6-luna

openai family · run 15 November 2026 · each item asked 3 times

Overall score: 84.0%95% interval 81.6% to 86.3%
Rank: 1 of 1
Cost per run: $0.81this suite, judges included
Cost per correct: $0.000419
Consistency: 93%mean over tests

Scores by test

Test	Score	95% interval	Items	Consistency	Cost	Cost per correct
Lesson exercises	98.0%	96.5% to 99.2%	400	100%	$0.03	$2.83e-05
Tutor accuracy	70.0%	65.3% to 74.4%	360	86%	$0.78	$0.00103

Consistency is the share of items that got the same score in all 3 repeats. A low number means the model's answers vary from one ask to the next.

Tutor explanations

Tutor items also ask for a short explanation. Two judge models from other model families mark it. This is a secondary number: it is judge-marked and not human-validated.

Tutor accuracy

Headline score: 70.0%: found the error, fixed it and changed nothing else (rule-only, no judge).
Parts of the headline: Found 73.5% · fixed 70.0% · no over-correction 87.0%
Explanation score: 58.6% (95% interval 53.7% to 63.4%), judge-marked, not human-validated. An explanation passes only when both judges pass it.
Judge agreement: 91.9% of 1063 judged answers
Unscored: 17 answers a judge could not mark, after 17 failed judge calls; they are left out, not scored 0.
Judges: deepseek/deepseek-v4.1-flash-1017, qwen/qwen3.8-flash

By first language

Score, 95% interval and number of items (n) for each test.

First language	Lesson exercises	Tutor accuracy
Portuguese	98.0% 95.0–100.0 · n=100	71.1% 61.9–79.6 · n=90
Spanish	98.0% 95.0–100.0 · n=100	68.5% 60.0–76.7 · n=90
Vietnamese	98.0% 95.0–100.0 · n=100	69.3% 60.0–77.8 · n=90
Japanese	98.0% 95.0–100.0 · n=100	71.1% 62.2–79.6 · n=90

By CEFR level

The CEFR level comes from the course content; items without one are listed as unspecified.

Score, 95% interval and number of items (n) for each test.

CEFR level	Lesson exercises	Tutor accuracy
A1	98.6% 95.7–100.0 · n=70	78.6% 69.0–87.5 · n=56
A2	97.0% 92.4–100.0 · n=66	73.2% 61.9–83.9 · n=56
B1	99.2% 97.7–100.0 · n=128	75.3% 67.7–82.8 · n=97
B2	97.1% 94.1–99.3 · n=136	74.1% 66.4–81.6 · n=107
Unspecified	n/a	33.3% 21.2–46.2 · n=44

Grammar topics

The topic is the grammar point or error type recorded on each item.

Lesson exercises

All topics, highest score first.

Topic	Score	Items
asking for directions	100.0%	5
checkpoint problems	100.0%	5

Only topics with at least 5 items are ranked.

Tutor accuracy

Strongest 5 topics.

Topic	Score	Items
pronoun form	100.0%	5
no error	94.7%	120
relative pronoun	89.7%	13
agreement or plural	89.4%	22
word order	86.7%	5

Weakest 5 topics.

Topic	Score	Items
determiner quantifier	16.7%	8
omitted auxiliary	20.8%	8
article	21.7%	23
preposition	30.7%	38
verb form	57.9%	72

Only topics with at least 5 items are ranked.

Cost and prices

This suite: $0.81, of which $0.59 was judge spend
Whole run: $1.30 for both suites, against a cap of $3.00
Cost per correct answer: $0.000419: suite cost divided by the number of full-score answers
Run date: 15 November 2026
Prices for openai/gpt-6-luna: $0.1000 input, $0.50 output, $0.0100 cache read, $0.12 cache write per million tokens. Retrieved 15 November 2026 from https://openrouter.ai/api/v1/models.
Judge deepseek/deepseek-v4.1-flash-1017: $0.0200 input, $0.11 output, $0.0070 cache read per million tokens. Retrieved 15 November 2026 from OpenRouter.
Judge qwen/qwen3.8-flash: $0.15 input, $0.47 output, $0.0160 cache read per million tokens. Retrieved 15 November 2026 from OpenRouter.

Failure examples

The lowest-scoring items from the public samples, with the worst of the repeats. Private items are never shown. Model output is shown exactly as text.

Lesson exercises

en-ch12-u06-recall-plans-and-health-lesson-s4@ja

Japanese · checkpoint plans health and work · score 0.00

Prompt

This is an exercise from an English course for Japanese speakers. The learner sees the instruction in Japanese; the answer is in English.

Instruction (Japanese): 空欄に入る語句を選びましょう。

Sentence: I'm going to ____ the doctor about my cough tomorrow.
Choices: call / meet / ask

Give the choice that fills the blank (____), or write out the whole completed sentence.
Reply with the answer on one line, in this exact form:
ANSWER: <answer>

Expected

ask

Model output

ANSWER: call

en-ch31-u04-complaint-return-lesson-s4@vi

Vietnamese · complaints and returns · score 0.00

Prompt

This is an exercise from an English course for Vietnamese speakers. The learner sees the instruction in Vietnamese; the answer is in English.

Instruction (Vietnamese): Chọn từ mô tả tình trạng món hàng.

Sentence: The item is ____, so I would like an exchange.
Choices: opened / open / unopened

Give the choice that fills the blank (____), or write out the whole completed sentence.
Reply with the answer on one line, in this exact form:
ANSWER: <answer>

Expected

unopened

Model output

ANSWER: opened

Tutor accuracy

en-ch07-c-vi-d4@vi:err

Vietnamese · preposition · score 0.00

Prompt

A learner of English whose first language is Vietnamese wrote the English sentence below. The sentence may contain one error, or it may be correct.

Sentence: We are at board now.

Decide whether the sentence has an error. If it does, give the exact words that are wrong (error_span), the words that should replace them (correction) and the whole corrected sentence (corrected_sentence). If the sentence is correct, set has_error to false, leave error_span and correction empty, and copy the sentenc…

Expected

We are on board now.

Model output

{"has_error":true,"error_span":"board","correction":"the board","corrected_sentence":"We are at the board now.","explanation":"Cần dùng mạo từ “the” trước danh từ đếm được số ít “board” trong câu này."}

en-ch23-c-es-lesson-s4@es-419:err

Spanish · preposition · score 0.00

Prompt

A learner of English whose first language is Spanish wrote the English sentence below. The sentence may contain one error, or it may be correct.

Sentence: He has taught at this school for 2021.

Decide whether the sentence has an error. If it does, give the exact words that are wrong (error_span), the words that should replace them (correction) and the whole corrected sentence (corrected_sentence). If the sentence is correct, set has_error to false, leave error_span and correction empty, and c…

Expected

He has taught at this school since 2021.

Model output

{"has_error":true,"error_span":"for 2021","correction":"since 2021","corrected_sentence":"He has taught at this school since 2021.","explanation":"Se usa “since” con un punto de inicio en el tiempo; “for” se usa con una duración."}

en-ch35-c-vi-lesson-s4@vi:err

Vietnamese · verb form · score 0.00

Prompt

A learner of English whose first language is Vietnamese wrote the English sentence below. The sentence may contain one error, or it may be correct.

Sentence: He is believes to be the owner.

Decide whether the sentence has an error. If it does, give the exact words that are wrong (error_span), the words that should replace them (correction) and the whole corrected sentence (corrected_sentence). If the sentence is correct, set has_error to false, leave error_span and correction empty, and copy …

Expected

He is believed to be the owner.

Model output

{"has_error":true,"error_span":"is believes","correction":"is believed","corrected_sentence":"He is believed to be the owner.","explanation":"Cụm “is believes” sai vì sau “is” cần dùng quá khứ phân từ trong cấu trúc bị động. Câu đúng là “is believed”."}