How we test

Every page on AI First Look mixes three kinds of evidence: what the maker says, what independent testers found, and what we measured ourselves. We keep them apart and label each one, so you can decide how much to trust it.

1. Our own hands-on tests

Language models. We give every model the same 6 small tasks through OpenRouter:

1. Follow a JSON format (including working out a relative date like "next Tuesday")
2. A logic puzzle we wrote ourselves, so it cannot be memorized
3. Write working code, which we run in a locked-down sandbox against 9 test cases
4. Find one fact hidden in a log of about 30,000 tokens
5. Call a tool correctly for two cities in one reply
6. Write a product blurb that keeps 5 rules at once

Answers are checked automatically by fixed rules, not by our opinion. We record the time, tokens and cost of every task, and you can open each one to see the exact task, how we checked it and the model's full answer. Each task runs once, at the model's default settings, so treat the result as a quick check rather than a benchmark: the same model can pass a task one day and miss it the next.

Image models. We run every image model through the same 6 prompts: English text, text in several languages, a portrait, a photo-real scene, an illustration and a product photo. We publish the first result for each prompt, unedited, with its cost and time, and note obvious flaws such as garbled background text.

2. What independent testers found

• We collect 15 to 30 hands-on reports per model from X, YouTube, Reddit and evaluation groups such as Artificial Analysis, LMArena and Vals AI, and feature 6 to 10 of them. Summaries by area (cost, speed, coding and so on) are based on everything we collected, and the page says how many reports that is.
• We prefer reports with something you could check: numbers, side-by-side comparisons, code or outputs. Pure hype and pure complaints without details are left out, and so are posts by the maker itself.
• Good and bad reports keep their real proportions. We do not hunt for criticism to look balanced, and we do not hide it either: if there is a credible negative report, at least one is shown.
• Every card is our own one-sentence summary with a link to the original. Posts from X, Reddit and YouTube only load from those sites when you click, so nothing tracks you before that.
• For YouTube reviews we use an AI tool to watch the video and list what was tested, then we check at least one result against the video ourselves before it goes on the page.

3. Official scores vs independent scores

Makers publish their own benchmark scores. Where someone else has run the same test, we put the two side by side:

• Holds up: the independent result is within about 2 points of the official one.
• Lower or higher than claimed: the gap is bigger than that.
• Not checked yet: no independent result we could find. A score the maker credits to a third party only counts as checked once we see it on that third party's own site.

4. Leaks, rumors and release dates

Before a model is announced, every claim carries one of these labels:

• Official: from the maker's own announcement, docs or accounts.
• Official teaser: the maker hinted at it without details.
• Reliable leak: several independent sources report the same thing, with evidence such as screenshots or code.
• Speculation: a single claim we could not verify. We say so, including when we looked for the source and could not find it.

Expected release dates come from the maker when there is one, and otherwise from leaks or prediction markets such as Polymarket, which we label as traders' bets rather than facts.

5. Updates and corrections

New models change fast, so pages are updated as new results appear. Each page shows when it was last updated and keeps an update log at the bottom. We use AI tools to help with research and drafting; every fact on a page links to its source, and a person reviews every page.

Found a mistake, or have a test result we should include? Tell us.

6. Independence

AI First Look is not affiliated with any AI company we cover, and nobody pays to be covered or to change a result. We pay for our own API usage. The site may show ads; advertisers have no say in what we write.