TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Mercor’s analysis estimates that 90% of American working hours involve tasks with little or no coverage in its initial index of AI benchmarks. The company mapped 29 benchmarks to tasks across 923 O*NET occupations and says the results are provisional, not a complete measure of all available benchmarks.
Mercor estimates that 90% of U.S. working hours are spent on tasks with little or no coverage in its initial index of AI benchmarks, raising questions about how well current evaluations measure AI performance on work people actually do. The company mapped 29 benchmarks to tasks across 923 occupations, but says the findings are provisional and depend on the benchmarks it selected.
Mercor’s Benchmark Map breaks the U.S. workforce into 923 occupations and 18,797 tasks, using the Department of Labor’s O*NET taxonomy. The company reports that only 208 occupations have even one task linked to at least five prompts in its benchmark index. It also says six of O*NET’s 22 job families have no occupation meeting that threshold.
For this analysis, Mercor selected 29 benchmarks based on public availability, agent-like tasks, economic relevance, breadth across job families and difficulty. A task counted as covered when the company’s language-model judge linked it to at least five benchmark prompts. Mercor says it checked the judge’s tagging against human expert tags on a subset of its APEX benchmarks.
To estimate the share of work time represented, Mercor combined its task-coverage estimates with separate estimates of the working-time share associated with O*NET tasks. It says benchmark coverage is uneven: its index heavily covers some AI-exposed tasks for lawyers, for example, while covering few tasks for legal secretaries and court clerks. The report describes smaller, middle-income knowledge-work occupations as underrepresented.
What the Coverage Gap Means for AI Evaluation
The result highlights a distinction between model performance on benchmark tests and performance across the tasks that make up a broad range of jobs. If evaluations concentrate on a limited set of occupations or activities, a strong score may provide little evidence about how a model handles tasks outside that set. Mercor argues that this makes it difficult to judge how capable AI systems are at work Americans actually perform.
For employers, workers and policymakers, the map is a warning against treating benchmark results as a complete account of workplace readiness. A benchmark can test a narrow activity well without covering all the judgment, communication or practical steps involved in a job. The report does not establish how AI performs on the uncovered tasks; it identifies a measurement gap, not proof that models succeed or fail at them.
The finding may also shape where evaluation work is directed. Mercor’s analysis suggests that benchmark attention is not distributed evenly across the labor market, with some prominent occupations receiving more coverage than comparable roles. Broader and more representative testing could give decision-makers a clearer basis for comparing model capabilities across different kinds of work.
As an affiliate, we earn on qualifying purchases.
How Mercor Built the Benchmark Map
The map uses O*NET’s occupation and task framework, which organizes the U.S. workforce into occupations and lists tasks associated with each. Mercor chose benchmarks it considered economically relevant and agentic, meaning prompts asked models to carry out tasks rather than simply answer questions or advise a user. It limited the number of benchmarks per job family to reduce the influence of areas such as software engineering and research.
Mercor’s coverage threshold is specific: an O*NET task had to be associated by its LLM judge with at least five prompts from the selected benchmarks. The company then combined that coverage assessment with estimates from Hatgis-Kessell and colleagues of the share of working time associated with each task. Its analysis of how coverage varies by occupation also uses wage-bill estimates calculated from Bureau of Labor Statistics mean wages and employment counts, assuming a 40-hour work week.
These choices matter when interpreting the headline estimate. The 90% figure describes estimated working time spent on tasks with little or no coverage under Mercor’s method and initial benchmark list. It does not mean that AI is incapable of doing 90% of work, nor does it describe the share of work that AI has automated.
“Our findings suggest that 90% of American working hours are spent on tasks with little or no coverage from existing benchmarks.”
— Mercor, in its Benchmark Map findings
AI productivity tools for knowledge workers
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Limits of the 90% Estimate
The 90% estimate is not a census of every AI benchmark. Mercor says it used an initial list of 29 and that the list is not a complete or perfectly representative sample of unsaturated, public, agentic economic benchmarks. The company says an incomplete list will push measured coverage downward, but that the size of this bias is difficult to bound.
The estimate also depends on mapping benchmark prompts to O*NET tasks with an LLM judge and on separate estimates of how workers divide their time among tasks. Mercor reports human validation for a subset of its benchmarks, but the supplied material does not provide a full account of mapping accuracy across all 29 benchmarks. It is also unclear how the estimate would change with a larger or differently selected benchmark set.
Mercor’s findings concern benchmark representation, not the actual performance of AI systems on every uncovered task. The source does not establish that models cannot perform those tasks, that they perform them well, or that workers spend 90% of their time on tasks impossible to automate. Those questions require evidence beyond the coverage map.
As an affiliate, we earn on qualifying purchases.
Mercor Plans to Expand Coverage
Mercor says it plans to expand the benchmark index and update its estimates as coverage grows. Future versions could show whether adding benchmarks changes the headline estimate or fills gaps across job families and occupations. The company says the map is intended both to reveal missing coverage and to help address those gaps.
For now, readers should treat the figure as an estimate tied to a defined set of benchmarks and a specific task-mapping method. Mercor has not provided a timetable in the supplied material for the next update. Further details on the expanded benchmark list and revised coverage measurements will help show how much the current finding reflects the broader evaluation landscape.
As an affiliate, we earn on qualifying purchases.
Key Questions
What does Mercor mean by 90% of working hours?
It is Mercor’s estimate of the share of American working time associated with O*NET tasks that have little or no coverage in its initial index of 29 benchmarks. It is not a claim that 90% of work is impossible to automate.
How did Mercor decide whether a task was covered?
Mercor used an LLM judge to link benchmark prompts to O*NET tasks. A task counted as covered if it was linked to at least five prompts from the selected benchmarks.
Does the report show that AI cannot do the uncovered tasks?
No. The analysis measures whether tasks are represented in the selected benchmarks. It does not directly test model performance on every task that lacks coverage.
Why does Mercor call the estimate provisional?
The company says its 29-benchmark list is incomplete and may not represent the full range of relevant benchmarks. Mercor says it will update the analysis as the index expands.
Source: hn
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
