Any Judge That Cannot Explain Its Verdict Carries No Authority

Imagine walking into a courtroom. You've been accused of something. The judge looks down, reads a single number -- "78" -- and returns to silence. No reasoning. No cited law. No explanation of which evidence mattered or why. Just a number, and a sentence that follows from it.

You'd call that absurd. You'd call it unjust. You'd be right.

Now consider what happens every day across thousands of universities, newsrooms, and freelance contracts: an AI detection tool returns a score -- "78% AI-generated" -- and a consequence follows. A grade is withheld. A payment is frozen. A student is hauled before an academic integrity board. And in most cases, the accused never learns why the tool reached its conclusion. Just the number. Just the verdict.

This isn't a strained analogy. It's a precise description of a system that has adopted the authority of judgment without any of the obligations that make judgment legitimate.

The Foundations of Legitimate Judgment

Every major system of adjudication humans have developed shares a common feature: the requirement to show your work.

In the American legal system, due process isn't a nicety -- it's a constitutional guarantee under the Fifth and Fourteenth Amendments. Due process requires, at minimum, that a person facing adverse action receives notice of the basis for that action and a meaningful opportunity to respond. A number with no explanation provides neither.

The Daubert standard, established by the Supreme Court in Daubert v. Merrell Dow Pharmaceuticals (1993), governs the admissibility of expert testimony in federal courts. Under Daubert, scientific evidence must meet specific criteria: the methodology must be testable, it must have known error rates, it must have been subjected to peer review, and it must be generally accepted in the relevant scientific community. These aren't arbitrary requirements. They exist because the courts recognized that scientific-sounding claims can be deeply misleading if you can't examine how they were produced.

Now apply those criteria to a typical AI detection tool. Is the methodology testable by outside parties? Usually not -- it's proprietary. Are the error rates known and documented? Sometimes partially, but rarely for specific populations or writing types. Has it been subjected to independent peer review? Almost never with the rigor Daubert requires. Is it generally accepted in the relevant scientific community? Computational linguists and AI researchers have expressed significant reservations.

Under the Daubert standard, most AI detection tools wouldn't be admissible as expert evidence in a federal courtroom. Yet we're comfortable using them to fail students.

The Federal Rules of Evidence (particularly Rules 702 and 703) reinforce this: expert testimony must be based on sufficient facts, reliable principles, and a reliable application of those principles to the case. An opaque score from a black-box algorithm, applied uniformly without case-specific analysis, struggles to meet any of those requirements.

Transparency Requirements in Other Domains

The principle that consequential decisions require explanation isn't unique to courtrooms. It runs through every domain where algorithms affect people's lives, and in most of those domains, we've already recognized the need for transparency.

Credit scoring. Under the Fair Credit Reporting Act (FCRA) and the Equal Credit Opportunity Act (ECOA), if you're denied credit, the lender must provide specific reasons. Not "your score was too low." Specific, actionable reasons: "too many recent inquiries," "high revolving utilization," "insufficient credit history." This requirement exists because Congress recognized that unexplained numerical judgments are both unfair and useless -- unfair because you can't challenge what you don't understand, and useless because you can't improve what you can't identify.

Medical diagnosis. When a doctor diagnoses a condition, they explain their reasoning. They describe the symptoms they observed, the tests they ran, the differential diagnoses they considered and ruled out, and why they landed where they did. Imagine a doctor handing you a piece of paper that said "78% likelihood of disease X" with no further explanation. You'd find a different doctor. But we accept this exact format from AI detection tools that affect academic careers.

Algorithmic sentencing. The use of risk assessment tools in criminal sentencing has generated enormous controversy, and for good reason. The COMPAS system, used in multiple U.S. states, was found by ProPublica in 2016 to produce racially biased risk scores. But the controversy wasn't just about bias -- it was about opacity. Defendants couldn't examine how their scores were calculated. They couldn't challenge specific inputs. The company claimed the algorithm was proprietary. The parallels to AI detection should be obvious and uncomfortable.

In State v. Loomis (2016), the Wisconsin Supreme Court upheld the use of COMPAS scores but imposed critical conditions: the scores couldn't be the sole basis for sentencing decisions, and courts had to be informed of the tool's limitations. Even in the context of criminal sentencing -- where the institutional power imbalance is enormous -- the court recognized that unexplained algorithmic outputs need guardrails.

Most AI detection policies in academia have no such guardrails.

The EU AI Act: A Regulatory Wake-Up Call

The European Union's AI Act, which began its phased implementation in 2024, represents the most comprehensive attempt so far to regulate AI systems by risk level. Systems used in education and employment are classified as "high-risk" and subject to stringent requirements.

Article 13 mandates that high-risk AI systems "shall be designed and developed in such a way as to ensure that their operation is sufficiently transparent to enable deployers to interpret a system's output and use it appropriately." Article 14 requires meaningful human oversight. Article 86 gives affected individuals the right to an explanation of individual decisions made by high-risk AI systems.

These aren't aspirational principles. They're legal requirements with enforcement mechanisms. An AI detection tool deployed in a European university that outputs a score with no explanation, no documented methodology, and no mechanism for challenge would likely violate multiple provisions of the Act.

Even outside the EU, the Act is setting a global baseline. Institutions that adopt detection tools without considering explainability requirements aren't just making an ethical mistake -- they're exposing themselves to legal risk that's increasing, not decreasing.

What "Algorithmic Justice" Actually Requires

There's a growing field of scholarship around what researchers call "algorithmic justice" -- the application of justice principles to automated decision-making systems. The core insight is straightforward: if a system has the power to harm people, it inherits the obligations that come with that power.

Those obligations aren't exotic. They're the same ones we expect from any adjudicative process:

The right to be heard. Before a consequential decision is made, the affected person should have an opportunity to present their side. In the context of AI detection, that means the score shouldn't be the end of the conversation -- it should be the beginning. The writer should be able to present counter-evidence: drafts, process documentation, voice analysis, testimony from collaborators. Many institutional policies technically allow for this, but in practice, the psychological weight of a high detection score creates a presumption of guilt that's difficult to overcome.

The right to understand the basis of the decision. You can't meaningfully challenge what you can't see. If a detector flags your work, you should be able to examine which specific features triggered the flag, what thresholds were applied, and how confident the system is in its assessment. "78% AI-generated" tells you nothing you can work with.

The right to appeal to a human decision-maker. Algorithms can inform decisions. They shouldn't make them unilaterally. Every consequential output from a detection system should pass through a human who has the authority, the expertise, and the institutional support to override the machine when the machine is wrong.

The obligation to document and disclose limitations. Any honest assessment tool has boundaries. It works better on some types of text than others. It has higher error rates for certain populations. Its confidence varies by context. These limitations should be documented, disclosed, and actively communicated to decision-makers -- not buried in a terms-of-service document that nobody reads.

When Detection Scores Have Been Challenged

The fragility of unexplained detection scores becomes apparent when someone actually pushes back with rigor.

In several documented cases at U.S. and UK universities, students who were flagged by AI detectors successfully challenged the findings by presenting process evidence: timestamped drafts showing the evolution of their work, research notes demonstrating intellectual engagement with sources, and in some cases, stylometric analysis showing consistency with their established writing patterns. In virtually every case I'm aware of where substantive counter-evidence was presented, the detection-based accusation was withdrawn or overturned.

That's telling. It suggests that the institutions themselves don't fully trust the detection scores -- they just don't have a better process in place. When confronted with actual evidence of authorship, the algorithmic verdict crumbles. But most students don't know they can challenge the score. Most don't know what counter-evidence to collect. And some institutions have processes so opaque that challenging a finding feels impossible even when it's technically permitted.

A few cases have reached more formal proceedings. In 2024, a graduate student at a large public university in the U.S. retained legal counsel after being expelled based primarily on a Turnitin AI detection score. The student's attorney challenged the reliability of the tool under the institution's own due process standards and presented expert testimony from a computational linguist. The expulsion was reversed. But the student had spent months under suspension, missed a semester, and incurred thousands of dollars in legal fees -- all because of an unexplained score.

What Actual Explainability Looks Like

Saying that systems should be explainable is easy. Defining what that means in practice is harder but not impossibly so. In my experience, a minimally acceptable standard for any tool used in high-stakes evaluation includes:

Localized evidence. Not just "this document is 78% AI-generated," but "these specific passages, in these specific sections, exhibited these specific patterns." The assessment should point to where in the text the concerns arise, not just deliver a global score.

Feature transparency. What did the system actually measure? Perplexity? Burstiness? Token probability distributions? Syntactic uniformity? The user -- and the person being evaluated -- should be able to see which features were analyzed and how each contributed to the overall assessment.

Confidence intervals. No measurement is perfectly precise. A system should communicate its uncertainty: "This score falls within a range of X to Y with Z confidence." A score presented without a confidence interval implies a precision that doesn't exist.

Known limitations. Prominently disclosed, not buried. "This tool has higher false positive rates for formal academic writing, technical documentation, and non-native English speakers. It has not been independently validated for the following use cases."

A methodology document that is detailed enough for an outside expert to evaluate and, ideally, replicate. Proprietary algorithms that can't be examined shouldn't be used to make consequential decisions about people's lives. There's no asterisk on that.

These aren't unreasonable demands. They're the same standards we apply to any form of evidence that carries consequences. The fact that they sound demanding in the context of AI detection says more about the current state of the field than about the standards themselves.

The Principle, Restated

I'll end where I started. Any judge that cannot explain its verdict carries no authority. This isn't a new idea -- it's one of the oldest principles in jurisprudence, in science, in any domain where one party's assessment affects another party's life.

AI detection tools have assumed the role of judge in contexts that matter enormously: academic careers, professional reputations, creative livelihoods. They've assumed this role without meeting any of the standards that make judgment legitimate. They provide numbers without reasoning. They render verdicts without evidence. They affect lives without accountability.

That's not a technology problem. It's a justice problem. And the solution isn't better algorithms -- it's the same solution it's always been. Show your work. Explain your reasoning. Accept scrutiny. If you can't do those things, you haven't earned the authority to judge.

Learn more about legally defensible metrics and voice fingerprint analysis.