The Enterprise AI Failure Casebook

New York City's mayor said its business chatbot was "wrong in some areas"

New York City launched an AI chatbot to answer business owners' questions about city rules, and five and a half months later, asked about a news story on its wrong answers, the mayor said it was "wrong in some areas."

City of New York · April 2, 2024 · Updated October 31, 2026

On October 16, 2023, Mayor Eric Adams and Chief Technology Officer Matthew Fraser released the New York City Artificial Intelligence Action Plan. The plan promised a framework for city agencies to evaluate AI tools and their risks. The city's announcement named one of those risks: AI can provide inaccurate or unintended results.

The same announcement launched MyCity Business, with Small Business Services Commissioner Kevin D. Kim. The site carried a pilot of the city's first citywide AI chatbot. The city said the chatbot would save business owners time and money by instantly giving them actionable and trusted information from more than 2,000 NYC Business web pages and articles. The topics it named were compliance with codes and regulations, available business incentives and how to avoid violations and fines. The site and the chatbot were in beta.

On April 2, 2024, at the mayor's weekly in-person media availability, a reporter raised a story about the chatbot published the week before. The reporter asked whether the chatbot should be taken down until it stopped giving wrong answers, noting that it had been up ten minutes earlier.

The mayor objected to the reporter's phrase "so wrong," which he said meant the entire process was broken. Then he said: "It's wrong in some areas, and we've got to fix it." He said any technology has to be put into the real environment to iron out the kinks, because "you can't stay in the lab forever." What a city learns there, he said, may be enough to take a tool down altogether, or enough to fix it. Of the chatbot, he said: "We're identifying what the problems are. We're going to fix them."

Deputy Mayor Maria Torres-Springer said the city had been "very clear from the beginning that this is a pilot." She said updates were already planned to reduce incorrect answers. In the meantime, she said, a disclaimer on the site told New Yorkers not to rely on it for legal advice, and the city wanted users to send feedback so the tool could improve. Deputy Mayor Fabien Levy said the chatbot was created for small business purposes and that some questions in the story were unrelated.

The reporter asked whether a question about informing staff of schedule changes was a small business question. The mayor answered: "Yes, that's a small business question, in my opinion." He said outside testing was what the city wanted: "We want people to kick the tires." He said the city would not have found those answers if it had stayed in a laboratory, and that the whole story had been given to the team to fix.

As of October 31, 2026, the chatbot's address redirects to a nyc.gov page that reads: "The Chatbot beta test has ended." The page gives no date.

What failed

The city chose, as its first citywide AI chatbot, a service it said would give business owners information on compliance with codes and regulations and on avoiding violations and fines. That is the use where a wrong answer can lead a business owner to break the law. The launch announcement named inaccurate results as a known risk of AI. Five and a half months later, the mayor said the chatbot was wrong in some areas. The safeguards officials named were a disclaimer against relying on it for legal advice and a request for user feedback. The problems the mayor said the city would fix were raised by a news story, not by the city, and he said outside testing of that kind was what the city wanted. Officials said updates were planned and did not say the chatbot would come down. The city's record does not list the wrong answers, say what testing the city ran before launch, or say why the beta ended.

Sources

1. NYC Office of the Mayor, transcript of Mayor Adams's in-person media availability, April 2, 2024, Question-and-answer exchange on the AI chatbot: the reporter's questions; answers by Mayor Adams, Deputy Mayor Torres-Springer and Deputy Mayor Levy
2. NYC Office of the Mayor, Mayor Adams Releases First-of-Its-Kind Plan for Responsible Artificial Intelligence Use in NYC Government, October 16, 2023, Subheads; paragraphs 1 and 2; the paragraph beginning "AI captures"; the MyCity Business paragraphs, including "save business owners and entrepreneurs time and money" and "Still in its beta phase"
3. City of New York, nyc.gov page shown at the chatbot's address (chat.nyc.gov), opened October 31, 2026, Page heading: "The Chatbot beta test has ended."