Last Friday, the American stock market witnessed a surprising phenomenon: investors rushed to buy Broadcom shares while selling Nvidia stocksBroadcom's stock soared by an astonishing 27%, marking its highest single-day increase ever, as its market valuation surpassed the impressive $1 trillion markIn stark contrast, shares of Nvidia, a leader in the semiconductor industry, experienced a notable decline of 3.3%.

The catalyst for this buying frenzy was none other than Broadcom's CEO, Hock Tan, who made bold predictions during the company's earnings callHe projected that demand for customized AI chips, known as Application-Specific Integrated Circuits (ASICs), could reach between $60 billion to $90 billion by 2027.

Analysts are quick to highlight that if this forecast holds true, it could imply that Broadcom's AI business related to ASICs might witness a twofold growth each year in the upcoming three years (from 2025 to 2027). This not only significantly raises market expectations for ASICs but also suggests the onset of a potential boom period for these specialized chips.

The dramatic escalation in chip demands is further complicated by ongoing challenges in data availability and diminishing returns

The state of AI models has transitioned from the initial training phase to a focus on inferenceIn this pivotal first stage, pre-training allows models to continuously absorb data and iterate their learning processes.

In pursuit of enhancing model performance, major technology firms have been locked in a fierce competition to acquire the most powerful Nvidia GPUs available on the market, adhering to the principle known as the Scaling Law—suggesting that larger data volumes, computation loads, and model parameter sizes yield better outcomes.

Nonetheless, this intense and expansive model training is starting to drain global databasesAs the marginal benefits of model expansion persist, training costs continue to escalate, sparking debates about whether this phase of AI training is nearing its conclusion.

Recently, Ilya Sutskever, a notable figure and co-founder of OpenAI, addressed an audience at the NeurIPS 2024 conference, stating that the era of pre-training is on the verge of ending

He likened data to a finite fossil fuel for AI, observing that the quantity of data available for AI pre-training has likely peaked.

Noam Brown, another prominent member of OpenAI, echoed Sutskever's concerns, reflecting on the remarkable accomplishments AI has achieved since 2019 due to the expansion of data and computational resourcesYet, he pointed out that even advanced language models struggle with simple tasks, such as playing tic-tac-toe.

As discussions evolve, the question arises: Is Scaling All You Need? Should we continue to ramp up expenses to train better AI systems? The focus is now shifting towards the next phase for large AI models—logical reasoning.

This subsequent phase signifies a pivotal leap, as it seeks to develop specialized applications of AI across various sectors based on existing large models for practical deploymentPresently, AI agents have become a primary focus for many companies, with notable products like Google’s Gemini 2.0 and OpenAI’s o1 making headlines.

As these AI models mature, there is a growing perspective that inference chips, particularly ASICs, will begin to supersede training chips represented by GPUs, ushering in a new era for AI companies as they become increasingly favored.

Broadcom's optimistic outlook on the ASIC market aligns with external expectations regarding a paradigm shift in AI, which was notably reflected in the soaring stock prices last Friday.

So, what exactly are ASICs? Generally speaking, semiconductors can be categorized into standard semiconductors and Application-Specific Integrated Circuits (ASICs). Standard semiconductors conform to standardized specifications and can be utilized across various electronic devices, provided they meet basic requirements

Conversely, ASICs are semiconductors manufactured by firms tailored to meet specific product needs.

This distinction gives rise to two divergent paths for AI computation: a general-purpose route exemplified by Nvidia's GPUs, suitable for high-performance calculations, and a specialized pathway represented by ASICs.

In terms of operational efficiency, while GPUs excel in handling large-scale parallel processing tasks, they encounter challenges such as memory limitations when grappling with extensive matrix multiplicationsASICs, meticulously designed for these functions, circumvent these issues, offering superior performance and cost-effectiveness upon mass production.

In summary, while GPUs are currently known for their maturity and established supply chain, ASICs display unique advantages with their specialization, delivering enhanced processing speeds and lower energy consumption for specific tasks, making them ideal candidates for inference and edge computing applications.

Amidst tightening GPU availability and soaring costs, an increasing number of tech giants are joining the ranks of companies developing ASIC chips exclusively for internal use.

Industry experts frequently cite Google as a pioneer in the realm of AI ASICs, having debuted its first-generation Tensor Processing Unit (TPU) in 2015. Other significant ASIC examples include Amazon's Tranium and Inferentia, Microsoft's Maia, Meta's MTIA, and Tesla’s Dojo, all indicative of notable advancements in the field.

Two production powerhouses, Marvell and Broadcom, have long dominated the upstream supply chain for self-developed AI chips

alefox

Marvell’s ascent can be attributed to successful strategies implemented by its new leadershipCEO Matt Murphy, recognizing the shifting landscape post-2016, has pivoted the company’s focus toward chip customization for technology giants, capitalizing on the burgeoning AI wave.

Beyond major clients like Google and Microsoft, Marvell has recently secured a five-year partnership with Amazon AWS to design its proprietary AI chipsAnalysts anticipate that this collaboration will contribute to Marvell’s AI custom chip division doubling in revenue in the next fiscal year.

Broadcom, a primary competitor to Marvell, similarly services prominent clients, including Google, Meta, and ByteDance.

Experts predict that by 2027-2028, each client could attain a procurement scale of one million ASICs per yearWith the rapid expansion of additional clients, these technology companies’ custom chip orders are poised to deliver substantial AI revenue for Broadcom in the coming years.

As AI models transition into what is being termed the "second half," the battle for the inference segment is just kicking off