📊 Full opportunity report: Minerva. The opposite path. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
Italy’s Minerva was built as a European sovereign language model from scratch, trained on 2.5 trillion tokens with half Italian content. Despite impressive technical achievements, it scored only 4.9% on Italian academic tests, highlighting challenges in scaling language models for country-specific knowledge.
Italy’s Minerva project, a large-scale European sovereign language model trained from scratch on 2.5 trillion tokens, scored just 4.9% on the INVALSI Italian school-exam benchmark, despite significant institutional investment and technical achievement. This result raises questions about the effectiveness of scale in producing country-specific knowledge.
The Minerva project, led by Sapienza University of Rome and funded through Italy’s National AI strategy, trained models ranging from 350 million to 7 billion parameters using approximately 50% Italian data. It utilized Italy’s CINECA supercomputing infrastructure and published its weights, data, and code openly from the outset.
While Minerva outperforms comparable multilingual models on Italian benchmarks at a technical level, its 3B parameter version scored only 4.9% on the INVALSI Italian school exams—a near chance level—despite being trained on 660 billion tokens with half Italian content. Researchers note that larger dataset size and more parameters are crucial for handling complex language tasks, which Minerva’s results suggest are not sufficient at current scales.
This empirical finding contrasts with Italy’s substantial investment and the model’s technical achievements, indicating that simply scaling up may not be enough to produce deep country-specific knowledge in language models. The broader implication is that European sovereign-LLM efforts may need to reconsider scale and resource commitments.
Minerva.
The opposite
path.
Italy spent years building a European sovereign LLM from scratch. Then Minerva-3B scored 4.9% on the INVALSI Italian school exam.
Where AMÁLIA layered Portuguese specialization onto a multilingual foundation, Minerva trained from scratch on 2.5 trillion tokens with approximately 50% Italian content. Where AMÁLIA’s weights are not yet public, Minerva published weights, training data, and code as truly-open from day one. By every institutional measure, the Italian approach worked. But the empirical results contain a finding the press coverage has been quiet about — and it has implications that extend well beyond Italy.
Same problem. Opposite path.
European sovereign-LLM development has two primary architectural approaches. Italy chose from scratch with substantial native-language foundation. Portugal chose continuation pre-training of a multilingual model. The structural comparison surfaces what each commitment actually requires operationally.
The comparison is not “Italy did it better than Portugal.” Both projects respond to the same structural problem with different architectural strategies under different institutional and economic constraints. Italy’s national-AI investment is structurally larger by an order of magnitude — and Minerva is the visible artifact of that scale.
Rebooting the Machines: A New Human Vision for Artificial Intelligence
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
4.9% on INVALSI. The bitter lesson surfaces.
In June 2024, researchers evaluated Minerva-3B on the Italian school-exam benchmark. The result was unambiguous. This is not a critique of Minerva — it is a critique of the public discourse around what Minerva’s empirical results actually demonstrate.
ASUS Ascent GX10 AI Supercomputer, DGX Spark, NVIDIA GB10 Superchip, 128GB LPDDR5x, 1TB PCIe Gen4 NVMe SSD, Wi-Fi 7 & BT5.4, Agentic AI Ready, Supports OpenClaw, NemoClaw, Stackable Chassis
Extreme AI Performance: Powered by NVIDIA GB10 Grace Blackwell Superchip delivering 1 petaFLOP of AI performance and 128GB…
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
350M to 7B. Four parameter scales, one architecture.
The Minerva model family covers four parameter tiers, each with specific training corpora. Each scale level reveals what the from-scratch path actually requires at different operating points.
Italian + English
100B English
~50% English
+ 200B code
Mastering Small Language Models: A Practical Guide to Building Lightweight NLP Systems with Python, Transformers, and Quantization Techniques
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three answers. Same question.
Minerva, AMÁLIA, and OpenEuroLLM represent the three operational answers to the European sovereign-LLM question. Each makes different architectural and institutional bets. The strategic discourse benefits from treating all three as data points in the same empirical experiment.
Assessing English Language Learners: Bridges From Language Proficiency to Academic Achievement
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three standards the movement should adopt.
The structural critique generalizes beyond Minerva. The European sovereign-LLM movement benefits from internalizing these lessons across every subsequent national project. Italy modeled the openness standard; the movement should adopt it as norm.
Minerva is one valid answer to the European sovereign-LLM question. AMÁLIA is another. OpenEuroLLM is potentially a third. The strategic discourse benefits from treating all three as data points in the same empirical experiment rather than as competing national-prestige projects. More analysis like this is needed. Not less.
Implications for European Sovereign-LLM Strategies
The low academic-test score of Minerva highlights a potential gap between technical progress and real-world language understanding. It suggests that European countries investing heavily in sovereign language models may need to rethink scale and resource allocation to achieve meaningful country-specific expertise. This finding questions assumptions that larger models automatically translate into deeper national knowledge, which has implications for future AI policy and investment strategies across Europe.
European Sovereign-LLMs: From Ambition to Empirical Reality
The Minerva project exemplifies Europe’s approach of building sovereign language models from scratch, emphasizing transparency and national investment. Italy’s effort involved 2.5 trillion tokens, half Italian content, and a dedicated research team operating on Italy’s CINECA supercomputers, with funding from national AI initiatives.
Prior to Minerva, debates centered on whether continuation training or training from scratch was more effective. Italy’s choice to train from scratch aimed to produce a model deeply rooted in Italian language and knowledge, contrasting with models like Portugal’s AMÁLIA, which layered specialization onto multilingual foundations.
Despite technical successes, Minerva’s low performance on academic benchmarks exposes a persistent challenge: scale alone may not suffice for complex language understanding, raising questions about the strategic value of such investments.
“Italy’s Minerva demonstrates that massive investment and open research do not automatically translate into deep country-specific language understanding.”
— Thorsten Meyer, AI researcher
Unresolved Questions About Scale and Effectiveness
It remains unclear whether further scaling of Minerva or similar models will improve their performance on complex, country-specific tasks. The ongoing iterations and future evaluations will clarify if the low benchmark score is a temporary artifact or indicative of a fundamental limit at current parameter scales.
Additionally, the precise impact of dataset composition versus size and parameters on model knowledge depth continues to be debated among researchers.
Next Steps for European Sovereign Language Models
The Minerva team plans to continue refining their models, including 2025 continual training experiments. Future evaluations on more complex and diverse benchmarks are expected to determine if increased scale or different training strategies can bridge the knowledge gap.
Policy discussions across Europe are likely to incorporate these empirical findings, potentially leading to adjustments in funding and strategic priorities for sovereign AI initiatives.
Key Questions
Why did Minerva score so low on Italian academic tests?
The low score suggests that, despite large-scale training, the model lacks the depth of country-specific knowledge needed for complex academic tasks. It highlights the importance of scale, data quality, and targeted training for such benchmarks.
Does this mean European sovereign LLMs are ineffective?
Not necessarily. It indicates that current approaches may need to be scaled further or combined with other strategies to achieve desired knowledge depth. The results are a critical empirical data point to inform future efforts.
Will increasing the size of Minerva improve its performance?
This remains an open question. Researchers plan to explore larger models and different training approaches, but current evidence suggests that scale alone may not guarantee better results for complex tasks.
How does Minerva compare to models like AMÁLIA?
Minerva was trained from scratch with a focus on Italian content and is openly available, whereas AMÁLIA layered specialization onto a multilingual foundation. Despite these differences, both face similar challenges regarding the relationship between scale and knowledge depth.
Source: ThorstenMeyerAI.com
