Vals.ai’s study shows Vecflow’s legal platform–Oliver–eclipses human lawyers in key legal tasks and is the “sole offering capable of nearing human-level performance” for complex research tasks.

Released this morning, Vals.ai’s study compares Oliver to four other established offerings: Thomson Reuters, LexisNexis, Harvey, and vLex. The study confirms that Oliver–with only 6 months on the market and far fewer resources than its peers– "often performed on par with or outperformed more established offerings." This includes companies valued in the billions.

We at Vecflow are extremely proud of our results. We think the study is an excellent step towards fully transparent benchmarking. Notably, however, several vendors opted to drop out after seeing their results. We believe that allowing companies to withdraw from specific categories, especially after seeing results, is hard to reconcile with the concept of a fair and independent study. 

Below is a table of the results, including areas where companies withdrew.

The evaluation tested AI legal assistants against human lawyers across seven legal tasks. Notably, Oliver outperforms the human lawyer in Document Q&A and Document Summarization. Following Harvey’s drop-out, Oliver is also the sole competitor in SEC EDGAR Research–the only task involving multi-stage, complex reasoning.

"Intelligent AI workflows represent the future of legal work." said Vecflow CTO, Joe Parker. "The evaluation highlights three critical findings:

1. AI assistants are already surpassing lawyers in several critical areas

2. Oliver's research capabilities stand unmatched in the legal tech sector

3. AI performs best when complementing a lawyer's existing workflow"

Vals.ai specifically noted that "in the challenging EDGAR research task, Oliver, through its coordination of specialized AI agents, stood out as the sole offering capable of nearing human-level performance." Research over EDGAR means multiple levels of iteration and research, currently unrivaled by the study’s other providers.

"We've already seen our customers able to handle more cases in less time. And there's evidence they can do this at even higher quality," said Thomas Bueler-Faudree, CPO of Vecflow, in an interview with Vals.ai. "You're going to see smaller firms that are technologically forward being able to rapidly do ten, 100 times more material in cases than they used to. I think you're going to see increasing competition in law, and hopefully you'll see one of the big four law firms appear in the US.”

Since the benchmark was conducted six months ago, we have made significant improvements to Oliver based on customer feedback. Oliver's capabilities have advanced substantially beyond the already impressive results shown in the study, and we are excited to continue innovating for the firms who trust Oliver every day.

Read more at - https://www.vals.ai/vlair