Verbatim Pundits

How intellectually honest political commentators are in what they actually say on the record.

Scores describe argumentative behaviour observed in sampled recordings. They are not fact-checks, and they are not judgements of a person's sincerity or character.

Pilot board. This is an early run, published so the method can be checked in the open rather than because the evidence is settled. It ranks 10 of 39 people on the roster, each on 5 to 12 recordings. Blinding removes the name from the transcript and does not make the speaker unknown: the judges still identified them on 100% of blinded recordings, so read the blinded score as name-removed rather than anonymous. Scores carry no correction for the format of the appearance. It was fitted and then withheld, because the corpus does not yet support it (venue 'debate' seen 4 times, under the minimum of 8). A reaction stream and a sit-down interview are not equally demanding, and that difference sits inside these scores.

Steel-manning and charity — 35% of the overall score. Does the speaker represent the other side fairly, at its strongest?
Epistemic rigor and calibration — 35% of the overall score. Does the speaker's confidence match the support they give?
Good faith and consistency — 30% of the overall score. Does the speaker argue in good faith and hold one standard?

Rank	Person	Blinded overall	Steel-manning and charity	Epistemic rigor and calibration	Good faith and consistency	Open overall	Halo	Recordings	Confidence
1	Ezra Kleincolumnist and podcast host	73.5 [67.1, 79.1]	71.7	73.8	75.4	71.8	-1.7	7	high
2	Coleman Hugheswriter and podcast host	65.0 [62.4, 67.8]	61.5	66.0	67.9	64.9	-0.1	7	high
3	Steven Bonnellstreamer and political commentator	57.8 [50.8, 65.0]	57.2	59.9	55.9	58.1	+0.4	6	high
4	Sam Sederpodcast and YouTube show host	51.6 [48.3, 55.5]	47.6	54.1	53.5	53.3	+1.4	7	high
5	Ana Kasparianonline news show host and commentator	48.0 [41.8, 53.6]	40.2	50.6	54.2	48.5	+0.3	6	high
6	Ben Shapiropodcast and radio host	42.4 [36.6, 48.8]	36.1	49.1	41.9	41.6	-0.9	12	high
7	Hasan Pikerstreamer and political commentator	34.6 [27.6, 42.1]	32.0	36.8	35.1	34.6	-0.1	8	high
8	Zack Hoytstreamer and political commentator	34.2 [26.6, 43.2]	30.6	35.1	37.4	35.7	+1.4	11	high
9	Charlie Kirkpolitical activist and radio/podcast host	32.5 [29.8, 35.1]	33.8	30.9	33.0	32.3	-0.3	5	high
10	Matt Walshpodcast host and author	31.3 [26.8, 35.3]	26.3	33.6	34.4	32.0	+0.8	8	high

What the scores mean

In the blinded score the judges read the transcript with the speaker's name and show removed. That does not make the speaker anonymous: a well-known commentator is often recognisable from their positions, and one of the two judges can search the web, which is measured and disclosed rather than blocked. The open score gives the judges the name; the difference between the two is shown as the halo.

Steel-manning and charity (35%)

Does the speaker represent the other side fairly, at its strongest? Credit for stating an opposing view accurately when that view appears in the recording, for answering its strongest form rather than a weaker one, and for naming where the real disagreement lies. The speaker loses credit for claiming bad motives without evidence given in the recording.

Epistemic rigor and calibration (35%)

Does the speaker's confidence match the support they give? Credit for backing load-bearing claims with stated evidence, a source or a mechanism, for separating fact from interpretation and speculation, for handling numbers consistently, and for saying what would change their mind. This checks the reasoning shown, not whether the claims are true.

Good faith and consistency (30%)

Does the speaker argue in good faith and hold one standard? Credit for applying the same standard to cases set side by side, for conceding a good point, for answering the question actually asked, for avoiding insults and insinuation, and for departing from their own side with reasons when the recording shows what that side is.

Method

Each recording is transcribed, the speaker's name and show are removed for the blinded score, and two AI judges score it against a fixed rubric: Claude Fable and Gemini Flash. Each judge's scale is calibrated onto a shared one, a person's score is the mean over their recordings, and the interval is a bootstrap over those recordings. A dimension a recording gives no real chance to show is marked not observed and does not count.