The artificial intelligence (AI) sector is undergoing a transformation as the focus shifts from training large language models to inference and agentic AI. This pivot opens new opportunities for companies ready to adapt their technologies and capitalize on changing market demands.
During the initial phase of the AI boom, training foundational models required substantial computational power, with Nvidia emerging as a dominant player. The company's graphics processing units (GPUs) excelled in performance and were strategically integrated into early AI research through its CUDA software platform. This has provided Nvidia with a significant advantage, securing its position as a frontrunner in the AI space.
As the market transitions toward inference and agentic AI, companies like Advanced Micro Devices (AMD) are well-positioned to thrive. This shift requires higher CPU-to-GPU ratios: training models typically use about 1:8, inference operates at around 1:4, and agentic AI demands a balanced 1:1 ratio. AMD leads in high-performance data center CPUs, aligning well with the increasing needs of agentic AI. The rising demand for more CPU cores—essentially individual processing units—will likely push prices upward, benefiting AMD considerably.
AMD's chiplet GPUs are particularly suited for inference tasks due to their design for larger memory capacities. This is critical, as inference workloads are predominantly memory-bound. With existing contracts for large GPU inference deals, AMD is likely to become a key player in the agentic AI sector.
Memory manufacturers like Micron Technology are also set to benefit from this shift. The demand for dynamic random access memory (DRAM) is increasing, supported by an undersupplied market, which is contributing to rising prices. As inference tasks typically require more bandwidth, the focus on high-bandwidth memory (HBM) in AI chips will enhance Micron's prospects. Recent moves by major DRAM producers, including Micron, Samsung, and SK Hynix, to secure longer-term contracts reflect a strategic shift that could improve stability within the memory sector.
Broadcom is another company positioned to gain from the market's transition towards inference. As a leader in application-specific integrated circuits (ASICs), Broadcom helps clients develop specialized chips tailored for specific tasks, including inference applications. This diversification of use cases highlights the growing importance of inference technology in the broader AI ecosystem.
As the AI market evolves, the focus on inference and agentic AI is becoming more pronounced, creating fertile ground for companies like AMD, Micron, and Broadcom. Their unique technological advantages and strategic positioning suggest they could emerge as significant players in the AI-driven economy.
Investors and analysts will closely monitor these developments, especially as the combined market for AI inference and agentic AI is projected to reach $100 billion in the coming years. The shift from training to inference signifies a fundamental change in how AI technologies will be developed, implemented, and monetized in the future.
