Recent AI advancements show surprising progress in legal and corporate analysis tasks
San Francisco: Last month, I found that AI systems were not very good at professional tasks like law and business analysis. The scores were low, with major tech companies scoring under 25%. We thought lawyers were safe from being replaced by computers for now.
But AI can improve quickly. This week, Anthropic released their new Opus 4.6 system, which scored much higher on these tasks. It reached just under 30% on a single try and averaged 45% when given multiple attempts. The model includes new features like “agent swarms” that help solve complex problems.
The new score is a big jump from before. Mercor’s CEO Brendan Foody was amazed, saying this progress was “insane.” While 30% is far from perfect, it shows technology is getting better faster than expected. Lawyers should realize machines are becoming more capable, even if they don’t need to worry about being replaced next week.