Arena, the well-known crowdsourced AI leaderboard that started as a research project at UC Berkeley in 2023, has reached a remarkable milestone: $100 million in annualized revenue just eight months after launching its first commercial product. The platform has become the go-to place for comparing the performance of large language models, letting users anonymously pit two AI responses against each other and vote on which is better. Over 10 million such evaluations have been submitted to date.
The revenue surge comes from AI Evaluations, a paid service introduced in September 2024 that provides model developers and enterprises with detailed performance analytics drawn from Arena’s community of users. By December 2024, the service had already reached $30 million in annualized revenue. It has since more than tripled to hit $100 million, reflecting the booming demand for rigorous, real-world AI assessment tools.
However, CEO Anastasios Angelopoulos clarified to TechCrunch that the $100 million figure carries a caveat. While Arena describes it as annualized revenue (ARR), customers pay based on consumption rather than traditional SaaS subscriptions. “A lot of people don’t even understand that our business is making any money at all, they still see us as like an open-source project,” Angelopoulos said. This distinction means the revenue is not recurring in the conventional sense, but it nonetheless demonstrates the immense value the platform provides to the AI industry.
Arena’s rise comes amid a rapidly expanding market for AI evaluation and data labeling. The company faces no direct competitor in the crowdsourced model comparison space after Yupp, the only other startup of its kind, shut down in March 2025 after raising $33 million from a16z crypto’s Chris Dixon. Instead, Arena competes “for the same dollar” as human labeling firms such as Mercor, Surge, and Scale AI, all of which help AI developers refine their models during post-training. This market is growing at an extraordinary pace: Handshake, a data vendor, saw its annualized revenue from AI training nearly double from $550 million in January 2025 to almost $1 billion by April, according to The Information. Mercor also claimed to have surpassed $1 billion in annualized revenue earlier this year, though a supply chain breach has since complicated its relationship with key clients including Meta.
The need for robust evaluation stems from the sheer pace of AI model releases. Frontier labs like OpenAI, Anthropic, and Google routinely cite Arena’s rankings in their own launch announcements, treating the leaderboard as a de facto industry benchmark. Arena now ranks models across text, coding, vision, and image generation, as well as complex agent workflows through a recently introduced Agent Mode. This comprehensive coverage ensures that developers can assess performance on multiple dimensions, from simple chat responses to sophisticated multi-step tasks.
The founding team brings together deep academic and entrepreneurial expertise. Angelopoulos and Wei-Lin Chiang are both postdoctoral researchers at UC Berkeley, while Ion Stoica, a UC Berkeley professor and co-founder of Databricks, served as an advisor before the project was incorporated in April 2025. The company raised $150 million in a Series A round in January 2025 at a valuation of nearly $2 billion, bringing total funding to $250 million. Investors include Felicis, Andreessen Horowitz, Kleiner Perkins, and Lightspeed.
To understand why Arena’s business model works, it helps to look at the broader landscape of AI evaluation. Traditional benchmarks like MMLU, GSM8K, and HumanEval are widely used but have limitations—they can be gamed, and their static nature doesn’t capture the nuances of real-world usage. Arena’s crowdsourced approach offers a dynamic, human-centric alternative. By showing two anonymous model outputs (one from each competing AI) and asking users to choose the better response, the platform aggregates millions of judgments that reflect actual user preferences. This methodology, inspired by Elo rating systems from chess, produces rankings that are both transparent and adaptive to improvements in model quality.
The paid service AI Evaluations takes this a step further. Instead of just seeing overall rankings, customers receive granular analytics—how their model performs on specific topics, languages, or difficulty levels, and even breakdowns by user demographics. This data is invaluable for companies that need to identify weaknesses in their models before deployment. For example, a lab building a medical chatbot might discover through Arena’s evaluations that its model performs poorly on rare disease queries, prompting a targeted fine-tuning effort.
The economics of AI evaluation are also becoming more visible. While training a frontier model like GPT-5 can cost hundreds of millions of dollars, the post-training phase—where models are aligned, safety-tested, and optimized for specific tasks—can account for another 10-20% of total costs. Human labeling firms have historically dominated this space, but automated and semi-automated evaluation tools like Arena are increasingly seen as faster and more scalable. The fact that Arena reached $100 million ARR in under a year suggests that the market is willing to pay for quality assessment services, even if the revenue model is consumption-based.
The competitive dynamics are shifting too. Yupp’s shutdown left Arena as the only crowdsourced leaderboard, but the company still faces indirect competition from synthetic data generators and automated evaluation pipelines. For instance, platforms like Scale AI offer human feedback as a service, while newer entrants like Patronus AI focus on safety-specific testing. Arena’s differentiation lies in its massive scale—millions of human judgments—and its established brand trust among the research community and major labs.
Angelopoulos has hinted that Arena’s next steps include expanding its evaluation capabilities for multimodal models and real-time agent interactions. The company is also exploring ways to make its data more accessible to smaller startups that might not afford the full paid service. Whether Arena can sustain its growth trajectory depends on continued adoption by the AI industry and its ability to stay ahead of evolving model architectures. For now, the platform’s rapid monetization confirms that evaluating AI may be nearly as lucrative as building it.