AMD Ryzen AI beats Intel Lunar Lake by 27% in LLM performance – with impressive NPU and iGPU power

With the Strix Point APUs, AMD is launching a new processor line on the market that has been specially developed for AI-supported applications and LLMs (Large Language Models). A direct comparison with Intel’s Lunar Lake chips reveals a clear difference in performance. According to AMD, the Strix Point APUs can outperform their competitors not only in terms of pure processing speed, but also in terms of latency reduction. This could be an advantage for applications based on fast response times and high computing power.

A central component of the new AMD APUs is the Ryzen AI 300 processor series, which is designed to deliver a higher number of tokens per second than Intel’s Lunar Lake SoCs, especially in the areas of AI and LLMs. When testing the Ryzen AI 9 HX 375 model in the LM Studio development environment, the Strix Point APU achieved up to 27 percent higher performance compared to the Intel Core Ultra 7 258V. Although the latter is not one of the absolute top models in the Lunar Lake series, it still shows good performance values and is positioned relatively close to the high-end variants in terms of hardware specifications.

The LM Studio developed by AMD is a software environment based on the Llama.cpp framework and enables users without in-depth knowledge of AI technology to apply LLMs. Llama.cpp is optimized for x86 CPUs and uses AVX2 instruction sets to boost performance for LLM applications. In combination with the GPU, the framework can be additionally accelerated, but a GPU is not mandatory for basic operation. According to AMD, the Strix Point APUs also achieve lower latency times in LM Studio compared to Lunar Lake: In a test with Meta Llama 3.2 1b Instruct, the Ryzen AI 9 HX 375 offers up to 3.5 times lower latency than the Intel Core Ultra 7 258V and achieves up to 50.7 tokens per second compared to 39.9 tokens per second.

To achieve this speed and low latency, both the Lunar Lake and Strix Point APUs rely on integrated graphics units for accelerated processing. AMD integrates the RDNA 3.5 graphics architecture into its Strix Point APUs and uses the Vulkan API to outsource certain tasks to the iGPU. This should increase processing efficiency and, according to AMD, deliver up to 31 percent higher performance in models such as Llama 3.2. Thanks to the variable graphics memory (VGM) function, the Ryzen AI 300 processors can also dynamically adjust the memory for graphics-intensive tasks. This optimizes energy efficiency and increases performance when GPU acceleration is used.

In further tests that AMD carried out with the same configurations in the Intel AI Playground, the Ryzen AI 9 HX 375 also showed advantages over the Intel Core Ultra 7 258V. The processor achieved up to 8.7 percent more performance in the Microsoft Phi 3.1 model and a 13 percent increase in the Mistral 7b Instruct 0.3 model. Although AMD has tested the Ryzen AI 9 HX 375 model against the Intel Core Ultra 7 258V, a direct comparison with the more powerful Core Ultra 9 288V would also be revealing, as the Ryzen AI 9 HX 375 is the most powerful model within the Strix Point range and could challenge Intel’s flagship model.

With the Strix Point APUs, AMD aims to provide powerful processors for use in AI workloads that are specifically tailored to the application of LLMs. In doing so, AMD focuses on supporting end users who may not have deep technical knowledge of AI. LM Studio, which is based on the Llama.cpp framework, allows these users to easily apply LLMs without having to understand the technical details in the background.

The developments at AMD and Intel show how the market for specialized AI hardware is becoming increasingly differentiated. As the demand for fast hardware for large language models continues to rise, the new APU generations offer approaches to operate these models more efficiently.

Source: AMD