Raw Aus Pol

← Media bias

Do the headlines match the stories?

A second check: once a story runs, is it told straight? Each outlet ranked by the share of its headlines that are fair to the story and not framed for or against a party.

Method, in short: each outlet is ranked by its straight headline rate: the share of its headlines we rated fair (the story backs it) and neutral (not framed for or against any party). Ties go to the outlet with fewer misleading headlines. Leans shows which way its slanted headlines tip. Where an outlet blocks our tools we can only see headlines, so it can't be checked for fit and is ranked separately on framing alone. That's an easier test, not a better score. Full rules are under How we rate.

How reliable are these ratings? Less strict than we'd like, especially for the punchier outlets.

Each outlet's stories were rated by a different AI agent working to the same written rules. To check they applied the rules evenly, a separate rater re-rated 120 stories from the 2021 to 2025 samples, 20 picked at random from each outlet we could read in full, without seeing the first ratings.

OutletSame ratingRated fair first timeRated fair by the checker
ABC News17 of 201916
SBS News17 of 201918
Guardian Australia17 of 202017
news.com.au12 of 201810
Michael West Media11 of 201811
Independent Australia10 of 20167

Almost every disagreement was a headline first rated fair that the checker called spin, usually over loaded words in the outlet's own voice ("slams", "brutal", "Big Lie") or vague teasers ("Shock development"). So the straight headline rates above are too generous for every outlet, and most of all for news.com.au, Michael West Media and Independent Australia. The broad order holds, with ABC, SBS and the Guardian well ahead of news.com.au, Independent Australia and Michael West Media, but the real gap between them is wider than shown. Twenty stories each is a small check, so treat small differences within those groups as noise. For long stories the raters mostly judged from the opening paragraphs and the end, reading every word only where something looked off. The Age isn't in the check: we only see its free preview, which flatters its rate in a different way.

Outlets

The articles

Loading…

How we rate

Headline compares the headline with the article. Fair: it sums up the story's main point. Spin: the story backs it, but it picks the harshest or kindest angle, uses loaded words ("slammed", "chaos", "backflip"), or leads with a side point. Misleading: it claims or implies something the story doesn't support. Unrated: we could only see the headline, so we don't judge it.

Framing is whose favour the headline runs in, judged from the headline alone. Bad news reported straight is neutral. Framing is about how it's told, not whether the news is good or bad. The score on each outlet counts headlines framed for Labor, the Greens or against the Coalition as towards the left, and the reverse as towards the right.

By party uses the same test, one party at a time: does the headline cast that party in a good light, a bad one, or neither? A headline can be good for one party and bad for another. Teals means community independents.

Paywalls: we don't get around them. For paywalled stories we rate from the headline, standfirst and whatever preview the outlet shows the public, and say so on each article.

Limits: up to 40 stories an outlet per period is a sample, not a verdict. Picks were taken in the order we found them, not hunted for the worst examples. Opinion pieces are allowed to argue: we only check that the headline matches the argument. Raw Aus Pol is openly pro-Labor, so read our ratings with that in mind, and click through.