Norway's 2 petabytes of Huawei flash storage and LLM training

Norway's National Library is developing a sovereign Norwegian-language large language model using 2 petabytes of Huawei flash storage to process the country's largest digital collection of cultural content, believing that countries without LLMs trained in their native language are at a disadvantage. The project, tasked by Norway's Ministry of Culture, faces key challenges including data quality and pipeline throughput rather than compute power, and must bridge the gap between archive storage optimized for preservation and high-speed storage needed for AI training. The team is also addressing novel problems around evaluation standards for a Norwegian LLM, governance questions about access and use, and orchestration across multiple storage and computing systems.