Microsoft's AI Chip Strategy Reduces Costs and Nvidia Dependence

As the competition for AI supremacy intensifies, Microsoft is reportedly developing its own AI chips to reduce dependence on Nvidia's products, aiming for a full-scale launch by 2024.

Nvidia currently produces the A100 and H100 GPUs, which are extensively utilized for training AI models. These GPUs are purchased in bulk by major cloud companies, including Microsoft’s own Azure division, and then rented out to AI customers. With demand high and capacity limited, others in the AI space are scrambling to acquire their own chips for their data centers to ensure capacity.

The explosion of AI products has created a sharp demand for computing resources. Industry analysts estimate that OpenAI may require an additional 30,000 A100 GPUs to support ChatGPT's commercialization efforts this year. Recent reports show that Twitter CEO Elon Musk purchased 10,000 GPUs for his own generative AI project. Nvidia’s H100 GPU retails for over $30,000 per unit, but due to high demand, they are selling for over $40,000 each on eBay.

Microsoft's chip initiative, code-named Project Athena, is reportedly progressing rapidly, with prototypes already available to a select group of employees for testing. If successful, Microsoft aims to implement Athena broadly within the company by early next year. By developing its own AI chip, Microsoft would join the ranks of Google, Meta, Amazon, and Apple as one of the few companies creating custom chips for AI development.

Dylan Patel, Chief Analyst at research firm SemiAnalysis, estimates that ChatGPT currently costs $700,000 per day to operate, or 36 cents per query. Microsoft's custom chips could potentially reduce operating costs by a third or more, while also supporting the continued large-scale deployment of AI-powered features in Bing, Office, and GitHub products. Microsoft is currently pursuing an aggressive AI strategy, which has prompted Google to declare a "code red" as they scramble to release their own AI products.

Industry insiders don't expect Microsoft to completely replace its Nvidia GPUs, but rather to diminish its overall dependence. If successful, Project Athena could provide Microsoft with additional leverage in future negotiations with Nvidia.

Initiated in 2019, Project Athena has reached a crucial milestone at an opportune time for Microsoft. As various groups vie for access to computing power, Microsoft has had to ration resources for its internal users. External customers are also encountering difficulties in securing sufficient cloud computing resources for their AI projects. 

And as Microsoft seeks to reclaim market share on the search engine side, it’s no doubt mindful that operating costs could explode. AI-powered search is significantly more expensive than traditional search engine queries, and Microsoft is clearly aware. “From now on, the [gross margin] of search is going to drop forever,” CEO Satya Nadella told the Financial Times.