EvenUp, a legal tech startup that recently achieved unicorn status with a $1 billion valuation, is facing serious allegations from former employees who claim the company has misled investors about the capabilities of its AI technology. The startup, which promises to automate personal-injury demand letters using artificial intelligence, has reportedly relied heavily on human workers to complete tasks that were supposed to be handled by AI.
The company experienced explosive growth, jumping from an $85 million valuation at the start of 2024 to over $1 billion in an October funding round. EvenUp’s pitch centers on using AI to analyze medical records and case files, extracting key details to determine appropriate compensation for accident victims. Investors have praised its “AI-based approach” as a “quantum leap forward” in legal technology.
However, eight former employees who spoke to Business Insider paint a different picture. They describe numerous problems with EvenUp’s AI system, including missed injuries, hallucinated medical conditions, incorrectly recorded doctor visits, and mixed-up accident details. Some former employees said their supervisors explicitly told them not to use the AI because it was “unreliable and created too many errors.”
The technical issues are significant: The AI has reportedly hallucinated doctor visits that never occurred, confused injury locations (reporting shoulder injuries when legs were hurt), and mixed up crucial details like the direction cars were traveling in accidents. It has also missed critical information about whether doctors attributed injuries to specific accidents—details that can substantially impact settlement amounts.
Former employees expected AI would streamline their work, but instead found themselves working long hours, sometimes until 3 a.m. and on holidays, manually completing or correcting tasks. Simple cases might take two hours, but complex ones could require eight hours of work, leading to 16-hour workdays.
EvenUp CEO Rami Karabibar defended the approach, stating that combining humans and AI “ensures maximum accuracy and the highest quality.” The company claims its AI is “improving every day” and that employees now spend 20% less time writing demand letters than at the beginning of the year. The startup also notes that 72% of demand letter content now starts from an AI draft, up from 63% in June 2023.
The situation highlights broader concerns about the gap between AI promises and practical reality as generative AI technology sweeps across industries.
Key Quotes
They claim during the interview process and training that the AI is a tool to help the work go faster and that you can get a lot more done because of the AI. In practice, once you start with the company, my experience was that my managers told me not even to use the AI. They said it was unreliable and created too many errors.
A former EvenUp employee who left earlier this year described the disconnect between the company’s promises about AI capabilities and the reality employees faced, where supervisors discouraged using the AI system due to reliability concerns.
The combined approach ensures maximum accuracy and the highest quality. Some demands are generated and finalized using mostly AI, with a small amount of human input needed, while other more complicated demands require extensive human input but time is still saved by using the AI.
EvenUp cofounder and CEO Rami Karabibar defended the company’s human-AI hybrid approach, framing the extensive human involvement as a feature rather than a limitation of the technology.
When you scan the data, it gets scrambled a lot, and having AI read the scrambled data is not helpful.
Abdi Aidid, an assistant professor of law at the University of Toronto who has built machine-learning tools, explained why AI struggles with medical records that feature inconsistent formatting and handwritten notes—a key challenge for EvenUp’s technology.
They [the managers] acted like it should be really quick because the AI did everything. But it didn’t.
A former employee described the pressure to complete work quickly based on assumptions about AI automation, while the reality required extensive manual labor, including working on holidays to meet deadlines.
Our Take
EvenUp’s situation exemplifies a critical challenge facing the AI industry: the tension between venture capital expectations and technological maturity. The company’s rapid ascent to unicorn status was built on promises of AI automation, yet the reality appears to be a labor-intensive operation with AI as a supplementary tool rather than the primary engine.
What’s particularly concerning is the potential for AI hallucinations and errors in high-stakes legal contexts. Missing injuries or inventing medical conditions could directly impact compensation for accident victims—the very people EvenUp claims to help. This raises important questions about AI deployment in professional services where accuracy is paramount.
The broader lesson here is about transparency in AI capabilities. While human-in-the-loop systems are legitimate and often necessary, investors and customers deserve clarity about the actual state of automation versus the aspirational vision. As AI continues its rapid expansion, cases like EvenUp’s may prompt more rigorous due diligence and honest assessment of what AI can and cannot reliably accomplish today.
Why This Matters
This story reveals critical tensions in the AI startup ecosystem between investor expectations and technological reality. As companies race to capitalize on the generative AI boom, EvenUp’s situation demonstrates how the gap between promised automation and actual capabilities can create significant operational and ethical challenges.
The legal industry represents a massive opportunity for AI disruption, with personal-injury law involving time-consuming document review and analysis. If AI cannot reliably handle these tasks—especially with sensitive medical records and high-stakes financial outcomes—it raises questions about AI readiness in other professional services sectors.
The allegations also highlight the “human-in-the-loop” reality that many AI companies face but may not fully disclose to investors. While using humans to train and refine AI systems is standard practice, the extent of human involvement at EvenUp suggests the technology may be further from true automation than marketed. This matters for investors evaluating AI companies, customers relying on AI accuracy, and workers whose roles may be misrepresented.
For the broader AI industry, this case could increase scrutiny on claims about AI capabilities and prompt more transparency requirements around the actual versus promised performance of AI systems, particularly in high-stakes applications like legal and medical fields.
Recommended Reading
For those interested in learning more about artificial intelligence, machine learning, and effective AI communication, here are some excellent resources:
Recommended Reading
Related Stories
- The AI Hype Cycle: Reality Check and Future Expectations
- Wall Street Asks Big Tech: Will AI Ever Make Money?
- OpenAI’s Valuation Soars as AI Race Heats Up
- Perplexity CEO Predicts AI Will Automate Two White-Collar Roles by 2025
Source: https://www.businessinsider.com/evenup-ai-errors-hallucinations-former-employees-2024-11