The Algorithm as Judge: Article 22 GDPR, the SCHUFA Ruling, and the Right to a Human Decision

The Hook: A Number Nobody Chose

Imagine you want to finance a car, sign up for a mobile phone plan, or rent an apartment. You fill in a form, hit "submit," and seconds later the answer arrives: rejected. No clerk read your application, no human being thought about you. Between your request and the refusal stood a single number — a probability value, computed by a credit agency you never contacted, based on data whose origin you do not know, according to a formula guarded as a trade secret.

The bank tells you: "We didn't decide, it was the score." The agency tells you: "We didn't decide, we only supply a number. The bank decided." Between these two statements yawns a gap in which you, the affected person, simply vanish. Nobody will admit to having made the decision, and yet the decision was made. It was precisely into this gap that the Court of Justice of the European Union reached on 7 December 2023 — with a judgment that reopens the question of who actually decides when a machine does the arithmetic.

For someone like Sven — a Senior AI Engineer working across cloud architecture, software design, and IT security — this is not a matter for the consumer-protection department. It is an architecture question. Any system that automatically evaluates, filters, prioritizes, or rejects people — credit scoring, fraud detection, applicant pre-screening, dynamic pricing, risk classification — potentially falls within the scope of Article 22 of the General Data Protection Regulation. And ever since the Luxembourg rulings of 2023 and 2025, that scope is considerably wider than most engineers assume. This article takes you along the whole route: from the quiet, often-overlooked prohibition in Article 22, through the three SCHUFA judgments and the right to an explanation, to what all of this concretely means for building automated systems — and how it dovetails with the EU AI Act.

Part 1: The Quiet Prohibition — What Article 22 GDPR Actually Says

A norm that works differently from the rest of the GDPR

Most provisions of the General Data Protection Regulation (GDPR, Regulation (EU) 2016/679) follow a permission principle: processing is forbidden unless a legal basis applies (consent, contract, legitimate interest, and so on, Article 6). Article 22 is built differently. Its first paragraph reads:

"The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her."

At first glance this sounds like a right of objection that the data subject would have to invoke actively. But the European Data Protection Board had already clarified in its guidelines on automated decision-making (WP251, adopted in 2018), and the CJEU confirmed it in 2023, that Article 22(1) is not a mere right of objection but a general prohibition. A decision based solely on automated processing with significant effects is, in principle, impermissible — unless one of the narrow exceptions in paragraph 2 applies. The data subject need not first say "no"; the processing is forbidden from the outset. This is a legally momentous difference, because it reverses the burden: the human need not defend himself against the machine; rather, the operator must justify why the machine was allowed to decide alone at all.

The four conditions of the rule

For the prohibition to bite, four conditions must come together. There must be (1) a decision that is based (2) solely on (3) automated processing, including profiling, and that (4) produces legal effects or similarly significantly affects the person. Each of these four elements is a doorway for legal dispute — and it was exactly at these points that the credit agencies pushed to escape the prohibition.

The word "solely" means there must be no meaningful human involvement. A person who merely rubber-stamps an algorithmically prepared decision pro forma — without the capacity and authority to actually change it — does not remove the "solely." The human involvement must be substantive: a genuine review by someone who has the competence and the authority to override the result.

"Legal effects" means an interference with the person's legal position — the cancellation of a contract, the denial of a social benefit, the refusal of an entry or citizenship application. "Similarly significantly affects" extends this to factual disadvantages of comparable weight: the refusal of a loan, the screening-out of an application, exclusion from a service. Not every automated trifle qualifies — but anything that governs a person's access to essential goods and opportunities certainly does.

The three exceptions and the safeguards

Paragraph 2 names three constellations in which an automated decision is, exceptionally, permissible after all: where it is necessary for entering into or performing a contract, where it is authorized by Union or Member State law, or where it is based on the person's explicit consent. Yet even then the operator is not free: paragraph 3 requires "suitable measures" to safeguard the person's rights and freedoms — at a minimum the right to obtain human intervention, the right to express one's point of view, and the right to contest the decision. And for special categories of data (health, ethnic origin, political opinion, and so on, Article 9) the hurdles are higher still.

Article 22 can therefore be read as a three-tiered structure: a general prohibition, three narrow doors, and behind each door a catalogue of protective duties. The entire dispute over SCHUFA scoring turned on whether the credit agencies were even standing in front of the building — or whether, as they claimed, they did not "decide" at all and the norm therefore did not concern them.

Part 2: The Backstory — Scoring, § 31 BDSG, and the Court in Wiesbaden

What a score is — and what it conceals

A credit score is a probability value. It expresses the statistical likelihood that a person will meet their payment obligations, condensed into a single number or a letter grade. Germany's SCHUFA (Schutzgemeinschaft für allgemeine Kreditsicherung) is the best known of several credit reference agencies that compute such values for nearly the entire adult population and supply them to banks, landlords, mobile operators, and mail-order retailers. The underlying models are proprietary; which features enter with which weight is treated as a trade secret.

Germany had created its own scoring provision in § 31 of the Federal Data Protection Act (Bundesdatenschutzgesetz, BDSG), laying down conditions for the permissible use of score values. That provision in a sense assumed that scoring was permitted and merely needed regulating — a perspective in tension with the strict prohibition approach of Article 22. It was precisely this tension that the CJEU was to resolve.

The case of OQ and the reference from VG Wiesbaden

A citizen — anonymized in the proceedings as OQ — had applied to a bank for a loan and been refused. The basis was an unfavorable score value supplied by SCHUFA. OQ demanded from SCHUFA both disclosure of the data taken into account and the erasure of an entry she regarded as incorrect. The competent Hessian data protection authority saw no need to act; OQ brought a claim against that finding before the Administrative Court (Verwaltungsgericht, VG) of Wiesbaden.

The VG Wiesbaden stayed the proceedings and, on 1 October 2021, referred the decisive question to the CJEU (Case C-634/21): does the mere generation of a score value by a credit agency already constitute an "automated individual decision" within the meaning of Article 22(1) — with the consequence that the agency itself is subject to the prohibition? And if not: is § 31 BDSG compatible with the GDPR at all? It was the first opportunity for the Court to determine the scope of Article 22 in principle.

Part 3: The Judgment C-634/21 — When Preparing Is Already Deciding

The agency's defense

SCHUFA argued in seemingly convincing terms: it made no decision. It merely computed a score and transmitted it to its contractual partners. Whether a loan was granted was decided solely by the bank. The score was a preparatory act, not a decision; consequently Article 22 did not apply to SCHUFA but at most to the bank. Formally, there was a certain logic to this: the final, legally effective act — the refusal of the loan — was indeed carried out by the bank.

The Court's answer

On 7 December 2023 the CJEU rejected this argument. The Court held that the generation of the score value by the credit agency is itself an "automated individual decision" within the meaning of Article 22(1) — provided that the third party's decision (for example the bank's) on establishing, implementing, or terminating a contractual relationship draws strongly on that score value. The decisive word is "strongly": if in practice the bank attaches such determinative importance to the score that it effectively dictates the outcome, then the real decision shifts to whoever produces the score.

The Court reasoned teleologically — from the protective purpose. If the determinative assessment lay outside the scope of Article 22 and only the formal final sign-off lay within it, a protection gap would arise: the data subject could assert her rights to information, explanation, and contestation against the bank, which does not even know the logic of the score; and against the agency, which does know the logic, she would have no rights, because it supposedly does not "decide." The CJEU closed this gap by placing substance over form: whoever carries out the assessment that is in fact determinative is subject to the prohibition — regardless of who formally presses the final button.

The consequence for § 31 BDSG

With that, the second referred question was effectively answered. If SCHUFA scoring falls under Article 22, it is in principle prohibited and requires an opening clause under Article 22(2)(b) — that is, a national legal provision that carries the exception and contains the prescribed safeguards. The CJEU left it to the VG Wiesbaden to examine whether § 31 BDSG meets these requirements, but formulated standards so strict that serious doubts about the German provision's conformity with Union law remained. The German legislature was thereby put on notice to recalibrate its scoring law — a task that reaches far beyond the individual case.

Part 4: The Sibling Rulings C-26/22 and C-64/22 — the Right to a Fresh Start

On the same day the CJEU decided two joined cases concerning a different, equally existential question: how long may a private credit agency store adverse data?

The two claimants, UF and AB, had gone through insolvency proceedings and had been granted early discharge from their remaining debts by court decisions (of 17 December 2020 and 23 March 2021 respectively). This discharge of residual debt is a deliberate fresh start: it is meant to allow over-indebted people to re-enter economic life. In the public insolvency register the notice was erased after just six months (§ 3 InsBekV). SCHUFA, however, retained the information — relying on a code of conduct for credit agencies approved by the supervisory authority — for three years.

The CJEU found this incompatible with the GDPR. It was contradictory for a private body to hold data longer than the public register from which it originated. The discharge of residual debt is of existential importance to the person concerned; its purpose — the return to economic life — would be frustrated if credit agencies were allowed to carry the overcome past forward for years. At the same time the Court strengthened the procedural rights of those affected: data protection authorities must investigate a complaint, and their decisions are subject to full judicial review, not merely a formal check of whether the authority took action.

The three judgments of 7 December 2023 together form a pincer movement: one shifts the attribution of the decision toward the credit agency; the other two limit the time during which adverse data may have any effect at all. In both cases the affected person regains ground — against an infrastructure that previously she could barely grasp.

Part 5: Dun & Bradstreet C-203/22 — the Right to an Explanation Versus the Trade Secret

The heart of the conflict

The SCHUFA judgment answered the whether: credit agencies fall under Article 22. But it did not answer how deep the transparency obligation runs. This is exactly where the next case began. A woman in Austria — called CK in the proceedings — was denied a mobile phone contract on the basis of an automated creditworthiness assessment by Dun & Bradstreet Austria GmbH. CK demanded to learn how that assessment came about. The agency invoked its trade secret: the precise logic of the model was protected business knowledge.

Two fundamental interests thus faced one another: the data subject's right to "meaningful information about the logic involved" (Article 15(1)(h) in conjunction with Article 22) and the protection of trade secrets (Directive (EU) 2016/943). How much insight does an operator owe — and where does the duty end at the business secret?

The answer of 27 February 2025

The CJEU decided on 27 February 2025 (Case C-203/22), effectively confirming a right to an explanation. "Meaningful information about the logic involved," according to the judgment, means: the data subject must be informed, in a concise, transparent, intelligible, and easily accessible form, about the procedure and principles actually applied in using her personal data by automated means to arrive at a particular result — such as a credit profile. She must be able to understand the significance and the envisaged consequences of the processing, ideally by means of concrete, tangible examples: which data were drawn on? How would the result have changed had those data been different?

Importantly, note what the Court does not require: disclosure of the algorithm itself, the source code, or the complete mathematical formula. Simply handing over a complex computational procedure would in any case not be "intelligible" within the meaning of the norm. What is required is an explanation of the logic, not a copy of the model.

The balance with the trade secret

For the tension with the trade secret the CJEU chose a graduated path. The protection of trade secrets is recognized, but it does not automatically override the fundamental rights of the data subject. If an operator invokes a trade secret, it may not simply refuse the information; it must submit the — in its view protected — information to the competent supervisory authority or court, which then weigh the conflicting interests and determine the scope of the right of access. Wherever possible, means should be chosen that preserve the person's rights without infringing the rights of others. The trade secret thus becomes the object of a balancing exercise — not a blank check for opacity.

Part 6: The Interplay with the EU AI Act — Two Regimes, One Assessment

Two legal frameworks stacked on top of each other

Credit scoring and similar assessment systems are today regulated twice over. The GDPR, through Article 22, addresses the decision about a person: who may decide by purely automated means, and what rights does the affected person have? The EU AI Act (Regulation (EU) 2024/1689) addresses the system itself: it classifies AI systems that assess the creditworthiness of natural persons or establish their credit score in Annex III (point 5(b)) as high-risk systems — with the exception of systems used to detect financial fraud. For high-risk systems the AI Act demands a whole bundle of product-style obligations: risk management, data governance, technical documentation, logging, human oversight, and a conformity assessment.

The two regimes stack — they do not replace one another. Meeting the requirements of the AI Act does not automatically satisfy Article 22 GDPR, and vice versa. The GDPR asks about the legal basis, the data-subject rights, and the safeguards of the individual decision; the AI Act asks about the quality, traceability, and controllability of the system across its entire life cycle.

Convergence on the right to an explanation

It is striking that both regimes carry the same idea at different points: the explainability of a machine decision. Article 22, as interpreted in the Dun & Bradstreet judgment, gives the data subject a claim to an intelligible explanation of the logic. The AI Act contains, in Article 86, a right to an explanation of individual decision-making: a person affected by a decision taken on the basis of a high-risk Annex III system — which includes creditworthiness assessment — is entitled to a clear, meaningful explanation. Two legal texts, different origins, the same aim: the human should not merely endure the machine but be able to understand and challenge it.

Part 7: What This Means for Building Systems

A framework for practice

For engineers who design automated assessment systems, the legal position can be translated into a series of concrete review and design questions. The following table summarizes the elements of Article 22 and the construction decisions that follow from them.

From the right to an explanation (Dun & Bradstreet) follows a further, technically demanding requirement: the system must be explainable without disclosing the trade secret. This calls for a separation between the model (which may remain protected) and an explanation layer that can say, in concrete cases, which features contributed to the result and in which direction, and how a change in those features would have shifted the outcome. I am of the opinion that this is exactly where the methods of explainable AI — such as feature-attribution techniques and counterfactual explanations — move from a pleasant extra to a regulatory duty: they supply the vocabulary in which a model can account for itself, in a legally compliant way, to the person affected.

Three design principles

First: human oversight must be genuine. A sign-off button that an overworked clerk presses every few seconds does not remove the "solely." Anyone wishing to rely on human involvement must give the human time, information, and authority to truly review the machine result.

Second: data minimization and storage limitation are part of the architecture. The sibling rulings show that not only the decision but also the durability of the data is justiciable. Deletion deadlines, tied to public references, belong in the data model, not in a forgotten operating instruction.

Third: explainability is an interface, not an afterthought. If a data subject can demand an intelligible explanation, the system must be able to produce that explanation — reproducibly, case-specifically, and in a form a non-expert can follow. This can scarcely be bolted on afterwards; it must be designed in from the start.

The Central Takeaway

The thread running through all three Luxembourg judgments is a single idea: substance before form. Whoever carries out the assessment that is in fact determinative decides — no matter who formally presses the final button. Whoever holds adverse data must let it go once its purpose is fulfilled. And whoever passes judgment on a person by machine owes that person an intelligible explanation, behind which it cannot hide by pointing to a trade secret.

For everyday engineering work this means: the question "Does my system fall under Article 22?" should stand at the beginning of a design, not at its end. For the answer determines not only the legal basis but the architecture — the existence of a genuine human review authority, the deletion logic, the explanation layer. Retrofitting these requirements is expensive and often only partially possible. Designed in from the start, they become what they are meant to be: not a shackle on the technology, but the very condition for an automated decision to be just at all.

The practitioner's prompt, then, is this: take your most heavily automated assessment system and put three questions to it. Does its result in fact determine a decision about a person? Could a human realistically override that result? And could you explain to an affected person, in intelligible words, why she received this result and no other? Wherever one of these answers comes out uncertain lies your next task.

A Question to Reflect On

The CJEU held that the one who "decides" is the one whose assessment is in fact determinative — even where a human still formally stands in between. Yet the better machine recommendations become, the harder it grows for any human to override them: who bears responsibility when they set themselves against a statistically superior machine and err? Where does the line run between a human oversight that offers real protection and one that merely upholds the illusion of control while the machine has long since decided?

Cross-References in the Vault

This article connects to several earlier texts in the folder. The EU AI Act, whose high-risk classification and right to an explanation play a central role here, is treated in depth in The Pyramid of Risk: How the EU AI Act Tames Artificial Intelligence – and Why It Concerns the Whole World. The related question of how personal data stay protected across borders, and how the CJEU acts as guardian of fundamental rights, is developed in The Man Who Toppled Three Agreements: Max Schrems, Transatlantic Data Transfers, and the Puzzle of Adequacy. The broader compliance context includes NIS2 and What It Really Means for Mid-Market IT Consulting Firms in Germany, Security by Default: The EU Cyber Resilience Act and the End of the Insecure Product, and When IT Is Not Allowed to Fail: DORA and Digital Operational Resilience in Finance. For readers who want to understand how modern AI systems learn to align with human specifications in the first place, the technical background is in Praise and Blame for the Machine: RLHF and the Art of Teaching AI What We Want.

Sources

• CJEU, judgment of 7 Dec 2023, C-634/21 (SCHUFA Holding, Scoring) — summary and analysis: Bird & Bird, A&O Shearman, CMS
• CJEU, judgments of 7 Dec 2023, C-26/22 and C-64/22 (SCHUFA Holding, Discharge from remaining debts) — GDPRhub, PwC Legal
• CJEU, judgment of 27 Feb 2025, C-203/22 (CK v Dun & Bradstreet Austria) — Stibbe, Inside Privacy (Covington), Bird & Bird
• Overview of the CJEU's case law on automated decision-making — IAPP
• EU AI Act, Annex III (high-risk systems) and Article 86 (right to explanation) — artificialintelligenceact.eu, Annex III; classification of credit scoring: regulatoryai.eu
• Background on § 31 BDSG, the VG Wiesbaden reference, and the German context — Verfassungsblog, LfD Niedersachsen

← All articles