“7x Speed, Same Performance”… NOTA Proves World’s 3rd Ranked LLM Optimization Technology at ICML

The competition in generative AI is expanding from model development to ‘inference optimization’ technology that enhances actual service performance. We are now in an era where corporate competitiveness is determined by how quickly and efficiently the same AI model can be executed with minimal computing resources. Nota demonstrated its open-source large-scale language model (LLM) optimization capabilities by ranking among the top contenders in the AI optimization challenge held at the world-renowned machine learning conference ICML.

Nota, a company specializing in AI model lightweighting and optimization, announced on the 13th that it took third place in the ‘Efficient Qwen Competition’ held at ICML 2026, a global machine learning conference. This competition is a technical contest that evaluates how much inference speed can be increased while maintaining the same performance using Qwen, an open-source AI model widely used by global AI developers.

AI responses remain the same, speed increases 7-fold… LLM optimization proves competitiveness

The competition tested technologies that improve response speed while maintaining accuracy by running the Qwen3.5-4B model in a single NVIDIA A10G GPU environment. With over 40 teams participating from around the world, Nota secured third place, recording an average inference performance that was 6.978 times faster.

Nota has enhanced performance by combining its proprietary AI lightweighting technologies: quantization and speculative decoding. Quantization is a technique that reduces a model’s memory usage and computational load, while speculative decoding speeds up response times by having a draft model generate answer candidates first, followed by a main model verifying them.

Furthermore, by applying the Sliding-window Attention technique, which performs calculations based on recent input information, unnecessary computations were reduced and inference efficiency was further enhanced. In particular, the proprietary quantization technology is characterized by improved speed while minimizing accuracy degradation through subsequent learning.

Along with its achievements in this competition, Nota also had two papers accepted at the ICML AdaptFM workshop regarding Mixture of Experts (MoE)-based large-scale language model quantization technology. It proposed a technology that minimizes performance degradation with limited memory and computational resources by utilizing an MoE structure that selects and executes only the necessary parts among multiple expert models.

During ICML, the company also held the ‘Nota AI – Korea Efficient Days’ event to engage in technology exchanges with officials and researchers from global AI companies such as OpenAI, Google, and Qualcomm. Building on this achievement, Nota plans to expand global technology cooperation in various fields, including on-device AI, physical AI, and LLM inference optimization.

Related