The Enterprise AI Failure Casebook

A delivery platform's algorithms ranked and blocked riders with no right to a human review

Italy's data protection authority fined Glovo's Italian company 5 million euros after finding that its platform made automated decisions about more than 35,000 riders without the right to a human review.

Foodinho s.r.l. (Glovo group) · November 13, 2024 · Updated October 31, 2026

On October 1, 2022, a rider died in a road accident in Florence while making a delivery for Foodinho, the Italian company of the Glovo group. According to Foodinho, its head of external relations received a news agency report of the crash at 5:57 the next evening. At 9:01 the following morning, the manager of the operations team told an operator to disable the rider's account. The Garante's decision quotes the instruction, translated here from the Italian: "I'd say do as for the others. We set it to DISABLED and leave the note in the description." The decision also cites the company's Slack messages for it. The operator told inspectors he had received the instruction verbally.

The operator did it by hand. The platform's list of reasons for disabling an account had no entry for death, so "other" was entered. The platform then did what it did on every change of status. It sent the rider's account a standard message. Translated from the Italian, it said the account "has been deactivated for failure to comply with Terms and Conditions" and that, if payments were still due at the end of the next billing period, details of the last order would follow. The rider's family made the message public through the press.

Italy's data protection authority, the Garante, opened an inquiry on its own initiative, after press reports of the deactivation and a report from a group of IT experts. Inspectors from the Guardia di Finanza's privacy unit visited Foodinho in December 2022 and in February and March 2023, and again in July 2023 after a report about the rider app. Foodinho managed its riders on the Glovo platform, owned by the group's Spanish company, GlovoApp23. Foodinho said the message was a human error. The Garante disagreed. It found the message was sent because the system was set to send it automatically at every change of status, whatever the reason, and that this applied to every rider.

The inspectors then looked at the platform itself. An "excellence score", built from four measures over the last 28 days, decided which riders could book shifts first when slots opened twice a week. In Rome and Milan some slots filled every day, so riders with lower scores could not get them. A second algorithm assigned orders within a shift. Accounts were blocked automatically when set conditions were met. In 2022, 6,369 rider accounts in Italy were deactivated and 53,861 were blocked.

Foodinho said it ran no fully automated decision-making. It offered an expert report on human review. The Garante found that the report's database query selected only records tied to an operator's email address, which left out deactivations the system made itself. It also found that Foodinho had used facial recognition to check riders' identity from November 2020, paused it in July 2022 after bugs that could block riders' profiles without cause, and began testing it again in some cities in February 2023, all with no lawful basis.

In a decision dated November 13, 2024, the Garante fined Foodinho 5 million euros and banned the facial recognition processing. It ordered Foodinho to give riders the right to human intervention, to state their view and to contest automated decisions, with trained staff who can set aside the algorithm's output. It also ordered periodic checks of the algorithms' accuracy. It counted a 2021 decision against the same company as an aggravating factor. In December 2024, Foodinho told the Garante it would not appeal and would pay the reduced amount, half the fine, that the decision allowed.

What failed

Foodinho let software decide which riders got shifts first, which got orders and whose accounts were blocked. The Garante found decisions based solely on automated processing, with significant effects on more than 35,000 riders, and none of the safeguards the GDPR requires for them: human intervention, the chance to give a view, and the right to contest. The company's own evidence of human review measured only the decisions a person had touched. The same set-up told a dead rider his account was closed for breaking the rules, because it sent one standard message at every change of status, whatever the reason. Foodinho told the Garante it had not known about some of the processing on its platform, including data sent to third parties, and the Garante treated that as gross negligence.

Sources

1. Garante per la protezione dei dati personali, Decision no. 675 of November 13, 2024 against Foodinho s.r.l. (in Italian), Sections 1, 2, 4.1, 4.5.1, 4.5.3, 4.5.4, 4.7, 5, 6 and the operative part
2. Garante per la protezione dei dati personali, press release of November 22, 2024, "Rider, Garante privacy: no all'algoritmo incontestabile dai lavoratori" (in Italian), Paragraphs 1 to 4
3. Garante per la protezione dei dati personali, Decision no. 818 of December 19, 2024 extending Foodinho's deadlines (in Italian), Recitals on Foodinho's note of December 16, 2024; operative part