The AI Wars: Battlefronts Shaping an Industrial (R)Evolution

Author photo: Colin Masson
ByColin Masson
Category:
Technology Trends

Introduction

The rapid pace of advancements in Artificial Intelligence (AI) can be overwhelming, especially for IT, OT and ET audiences not directly engaged in AI research and development. Every day, our inboxes and news feeds are inundated with announcements about the latest developments in AI, with little clarity on the value they may bring to mission critical Industrial AI use cases.

To make sense of this information overload, it can be helpful to think of the current market ‘hype’ or ‘bubble’ as the ‘AI Wars’. In the AI Wars, the campaigns, battles, and skirmishes represent the competition for control of our hearts, minds, and various aspects of AI development and monetization. Each campaign is a concerted effort towards a specific AI goal, each battle is for control of a significant AI technology breakthrough, and each skirmish is a minor advancement or setback in AI marketing monetization.

As for any war, the scale of investments in AI hardware and software is staggering. According to a report by Dell’Oro Group, accelerated computing is forecast to lift hyperscale cloud capex to 17 percent growth in 2024. Another report by AIMultiple suggests that global data center capex is projected to grow at an annual rate of 18% and reach $200 billion by 2028. This investment in AI hardware is feeding the AI innovation cycle by making it possible to train ever more accurate and capable frontier AI models, further stimulating AI software innovation and AI research.

Investments in AI startups have also seen a significant surge as AI infrastructure expands access. According to Forbes, the companies on 2024’s AI 50 list have raised a total of $34.7 billion in funding. Furthermore, cloud hyperscalers are investing heavily in expanding their AI infrastructure. For instance, AWS, Google, and Microsoft are developing their own AI chips to reduce their dependence on Nvidia for their Datacenter AI hardware.

AI Battlegrounds

Datacenter Hardware

The demand for more accurate foundation models is insatiable. Training these models requires billions of parameters, which in turn requires powerful datacenter hardware. The battle here is to develop hardware and infrastructure that can handle this immense computational load efficiently and cost-effectively. Datacenters are also investing in the capacity for the expected modernization of almost all categories of software seeking to take advantage of advancements in Gen AI.

Edge Hardware

The expansion of deployment opportunities from the cloud to edge and hybrid use cases is driving the development of AI-optimized chips and hardware for edge devices. The goal is to make AI more accessible and practical for a wider range of applications.

General Purpose AI Software Platforms

In this battleground the protagonists compete to offer Cloud AI platforms, Open Source AI platforms, and Enterprise AI platforms for both Narrow AI and General AI solution development and deployment. They strive to integrate a diverse array of AI models and techniques to train, validate, and deploy accurate AI models with specialized skills. The objective is to cater to an ever-expanding variety of AI use cases, from simple, single-task Narrow AI applications to complex, multi-task General AI systems. The challenge lies in creating a platform that is both versatile, catering to a wide range of AI applications, and user-friendly, enabling developers to create AI solutions efficiently. This involves providing comprehensive tools and frameworks that eliminate the need for developers to start from scratch, thereby accelerating the AI development process.

Edge AI Software

The battle for Edge AI software is about reducing the size and cost of deploying AI models to edge devices. These devices range from electronic devices (the most ubiquitous being the modern smartphone), to automobiles, homes, and industrial equipment in the distributed smart grid, in factories, and supply chain logistics infrastructure. The goal is to make AI ubiquitous, seamlessly integrating it into our everyday lives and industrial processes.

AI Gurus

Mainstream software and hardware providers are vying for the new age of AI Gurus. These are the individuals who can make the next breakthroughs in foundation models, in Causal AI, Neuro-Symbolic AI, Quantum AI etc - and reduce the size and cost of AI systems. Their knowledge and expertise are invaluable in pushing the boundaries of what AI can achieve.

Industrial Grade Date Scientists

Manufacturing IT, OT, and ET skills were already in short supply. Now, industrial organizations are looking for an even more scarce resource: AI experts who understand the manufacturing domain. These experts are crucial in bridging the gap between AI technology and its practical application in the industrial sector.

AI Lobbyist Campaigns

Just as in any war, there are lobbyists in the AI Wars. These are individuals, organizations, legislators and governments that try to shape public opinion and legal limitations that may impact the rules of engagement in the AI Wars. They do this through various means, such as public relations campaigns, advocacy, and lobbying efforts.

Industrial Organizations Prefer Neutrality

Most industrial organizations aren’t armed for the AI Wars, and don’t want to get embroiled in the battles and skirmishes. Instead, they prefer to leverage their strategic partners in the industrial software and automation landscape. These vendors cant avoid the AI Wars, especially the battlefront for AI talent that understands industrial use cases. On many battlefronts industrial automation and software vendors are already forging their multi-cloud and multi-AI alliances, waiting for clear winner (or more likely the Rule of Three will prevail as they do in most competitive markets).

Industrial organizations hope to benefit from the advancements in AI without having to invest heavily in developing their own AI capabilities, by leaning on their traditional partner ecosystem, and the alliances they forge to infuse and industrialize AI technologies.

 Recent Developments in the AI Wars

Nvidia’s Dominance and Rising Competition

Nvidia dominates the datacenter AI chip market, but there’s more competition than ever as startups, cloud companies, and other chipmakers ramp up development. Nvidia’s AI accelerators have between 70% and 95% of the datacenter market share for artificial intelligence chips. However, companies like AMD and Intel are introducing new AI chips and pricing strategies to offer stiff competition to Nvidia’s dominance in the datacenter, and they face much more competition on the edge AI battlefront.

Computex 2024 Announcements

At Computex 2024, several major tech companies announced their latest AI hardware:

  • AMD unveiled a multiyear, expanded AMD Instinct accelerator roadmap which will bring an annual cadence of leadership AI performance and memory capabilities at every generation. The updated roadmap starts with the new AMD Instinct MI325X accelerator, which will be available in Q4 2024. This accelerator is capable of 2614 TOPS. Following that, the AMD Instinct MI350 series, powered by the new AMD CDNA 4 architecture, is expected to be available in 2025 bringing up to a 35x increase in AI inference performance compared to AMD Instinct MI300 Series with AMD CDNA 3 architecture.

  • ARM announced its path to enabling more than 100 billion Arm devices ready for AI by the end of 2025. This includes the development of new AI chips and hardware that can handle the computational demands of AI.

  • Intel introduced its latest Xeon 6 processors for servers, and shared more details about its next-gen Lunar Lake chips for AI PCs. Intel’s Lunar Lake chips are capable of 120 TOPS, which is a significant improvement over its previous generation.

  • Nvidia continues to lead the AI chip market with its widely used H100 GPUs. However, the company faces increasing competition from other tech giants and startups developing their own AI chips. The Nvidia H100 GPU is capable of up to 30x the performance of the previous generation, A100.

  • Qualcomm announced its latest Edge AI Box solutions, which represent the cutting edge in security and surveillance space. It helps to reduce overall costs and accelerate new AI applications while upgrading legacy camera and security assets into smart IoT-and 5G-supported networks. Qualcomm’s Snapdragon X Elite chips are capable of 45 TOPS.

AWS, Google, and Microsoft’s Datacenter AI Hardware

AWS, Google Cloud Platform (GCP), and Microsoft Azure are all leveraging Nvidia in their datacenters because Nvidia's GPUs are currently some of the most powerful and efficient hardware available for training AI and machine learning workloads. Nvidia's GPUs are designed to handle the parallel computations that these workloads require, making them an excellent fit for these tasks.

However, while Nvidia's hardware is powerful, it's also expensive and may not be optimized for every possible workload. By investing in their own custom hardware, these cloud providers can tailor their chips to their specific needs and potentially achieve better performance, power efficiency, or cost-effectiveness.

Partnering with AMD and Intel allows these cloud providers to offer a wider range of hardware options to their customers for training, and deploying a broad range of AI solutions beyond performance hungry Foundation Models . AMD and Intel both offer a variety of CPUs and GPUs that can be used for different types of workloads. For example, some customers might prefer to use AMD's GPUs for certain types of machine learning workloads, while others might prefer Intel's CPUs for general-purpose computing tasks. By offering hardware from multiple vendors, AWS, GCP, and Azure can cater to a wider range of customer needs.

  • AWS has custom AI chips, Trainium and Inferentia, for training and running large AI models. These chips provide an alternative to general-purpose silicon. For example, AWS customers have embraced Nvidia’s widely used H100 GPUs as part of Amazon’s EC2 P5 instances for deep learning and high-performance computing. The AWS Trainium chip is capable of up to 50% cost-to-train savings over comparable Amazon EC2 instances, and the AWS Inferentia chip enables models to generate inferences more quickly and at lower cost, with up to 40% better price performance.

  • Google has made significant investments in AI hardware for datacenters. The company has developed multiple generations of its own ASICs optimized for TensorFlow, its Tensor Processing Units (TPUs) are used by Google Cloud for machine learning workloads. TPUs are custom-built to accelerate machine learning workloads and are tightly integrated into Google's software infrastructure. They offer significant advantages in terms of speed and power efficiency compared to general-purpose CPUs, NPUs and GPUs. In addition to TPUs, Google is reportedly working on its own Arm-based chips. These chips could potentially offer even greater performance and efficiency for AI workloads.

  • Microsoft has made significant investments in AI hardware for datacenters. The company has developed multiple generations of its Azure Maia AI Accelerator, which is optimized for artificial intelligence (AI) tasks and generative AI. The Maia AI Accelerator is capable of 48 TOPS, providing high performance for AI workloads. Microsoft's Azure Cobalt CPU, an Arm-based processor tailored to run general-purpose compute workloads on the Microsoft Cloud, is another key component of their datacenter AI hardware. Microsoft has also integrated NVIDIA’s new Blackwell chip and AMD's ND MI300X V5 into its Azure supercomputing infrastructure, providing the latest AI-optimized silicon. These investments are part of Microsoft's strategy to deliver infrastructure systems that have been designed from top to bottom and can be optimized with internal and customer AI workloads in mind.

Apple’s (A)bsolutely (I)ncredible Edge AI Products and Capabilities

Apple has been making significant strides in Edge AI with their Bionic and M4 chips. The M4 chip, announced in May 2024, advances the industry-leading power-efficient performance of Apple silicon. It includes the latest generation of ARM CPUs, an upgraded GPU, new ISP and Imaging DSP, and Apple’s next-gen TPU. The M4 chip is capable of 38 TOPS13, which is faster than the neural processing unit of any AI PC today. Combined with faster memory bandwidth, along with next-generation machine learning (ML) accelerators in the CPU, and a high-performance GPU, M4 makes the new iPad Pro an outrageously powerful device for AI.

Apple’s WWDC 2024 event, scheduled for June 10-14, is expected to focus entirely on software advancements as it plans to catch up with rivals on the AI front. The event is expected to feature announcements related to the operating systems of Apple devices such as the iPhone, iPad, and Mac. The company’s marketing boss Greg Joswiak has hinted that the event will be “Absolutely Incredible,” suggesting a major focus on AI.

Microsoft’s Big Bet on Edge AI - Copilot+ PCs

On the edge AI front, Microsoft introduced Copilot+ PCs at the BUILD 2024 event. These PCs are designed for AI and come with powerful new silicon capable of 40+ TOPS, all-day battery life, and access to the most advanced AI models. The local AI chips for the Copilot+PC initiative are designed to be less powerful than large language models (LLMs) like OpenAI’s GPT-4, instead prioritizing efficiency and cost-effectiveness. This makes them ideal for edge AI applications, where power efficiency and the ability to run AI applications directly on the device (without cloud connectivity) are critical. The first wave of Copilot+ PCs includes devices from Microsoft Surface and OEM partners Acer, ASUS, Dell, HP, Lenovo, and Samsung.

Microsoft's acquisition of Fungible, a company that develops data processing units (DPUs) optimized for AI workloads, is another key aspect of their edge AI hardware strategy. Microsoft plans to use Fungible's DPUs to accelerate the performance of Azure IoT Edge and other edge AI solutions.

Google’s Edge AI Investments

On the edge AI front, Google’s Pixel phones are equipped with a Tensor G3 chip, making them AI powerhouses. The Tensor G3 chip includes the latest generation of ARM CPUs, an upgraded GPU, new ISP and Imaging DSP, and Google’s next-gen TPU. This chip was custom-designed to run Google’s AI models. The Tensor G3 chip is capable of 38 TOPS, which allows it to perform a large number of operations in parallel, making it highly efficient for running AI workloads.

Google's Edge TPU is another example of its edge AI hardware. It's a purpose-built ASIC designed to run AI at the edge. It delivers high performance in a small physical and power footprint, enabling the deployment of high accuracy AI at the edge.

Understanding ASICs, FPGAs, GPUs, and NPUs

Understanding the AI Wars not only requires an understanding of the tools and techniques relevant to the Industrial AI (R)Evolution (see our taxonomy included in that Strategy Report), but also some background in the datacenter and edge hardware powering much of AI news, which is usually a mix of Application-Specific Integrated Circuits (ASICs),  Field Programmable Gate Arrays (FPGAs), Neural Processing Units (NPUs), and Graphics Processing Units (GPUs). Each of these hardware types has its own strengths and weaknesses, and their applicability depends on the specific AI use case.

  • Field Programmable Gate Arrays (FPGAs). FPGAs are integrated circuits that can be reprogrammed to perform specific tasks, making them highly flexible and adaptable. This flexibility is particularly beneficial in the rapidly evolving field of AI, where algorithms and requirements can change frequently.

    • Pros of FPGAs:

      • High performance for AI-related workloads.

      • Can be reconfigured multiple times for different purposes.

      • Offer low latency, which is better for applications that require real-time AI.

    • Cons of FPGAs:

      • Designing and optimizing FPGAs for specific AI tasks can be challenging.

      • FPGAs require specialized hardware and software optimizations to deliver high-performance AI processing.

      • They have limitations such as high power consumption and limited flexibility compared to CPUs.

 

  • Neural Processing Units (NPUs). NPUs are specialized hardware designed to accelerate machine learning algorithms, a type of artificial intelligence. They are highly efficient for AI model inference and training tasks. NPUs are used in edge devices like smartphones and IoT devices to provide AI capabilities without needing to connect to the cloud.

    • Pros of NPUs:

      • Optimized to handle the complex computations required by deep learning algorithms.

      • Particularly adept at handling AI-related tasks, such as speech recognition, background blurring in video calls, and photo or video editing processes like object detection.

    • Cons of NPUs:

      • Designing and optimizing NPUs for specific AI tasks can be challenging.

      • NPUs require specialized hardware and software optimizations to deliver high-performance AI processing.

      • They have limitations such as high power consumption and limited flexibility compared to CPUs.

 

  • Graphics Processing Units (GPUs). GPUs are a type of specialized circuit that is designed to rapidly manipulate memory to accelerate the creation of images. Built for high throughput, they are especially effective for parallel processing tasks, such as training large-scale deep learning applications.

    • Pros of GPUs:

      • Broad application potential beyond graphics.

      • Ideal for AI workloads like deep learning, where matrix operations and neural network computations are performed in parallel.

      • Versatile and can be used for a wide range of tasks.

    • Cons of GPUs:

      • GPUs have limitations regarding energy efficiency, thermal issues, endurance, and the ability to update applications with new AI algorithms.

 

  • Application-Specific Integrated Circuits (ASICs). ASICs are custom chips designed to perform a specific task very efficiently. In the context of AI, ASICs are often designed to accelerate specific types of AI workloads, such as deep learning inference or training.

    • Pros of ASICs:

      • Unparalleled performance for specific AI workloads.

      • High energy efficiency for specific tasks.

      • Ideal for AI applications requiring high performance and energy efficiency.

    • Cons of ASICs:

      • Lack of flexibility.

      • Cannot be easily repurposed or adapted for other tasks.

The choice between ASICs, FPGAs, NPUs, and GPUs will depend on the specific requirements of the AI workload, including factors like the need for flexibility, power efficiency, and the type of AI algorithm being used. Each type of hardware has its own strengths and weaknesses, and the best choice will depend on the specific AI use case. Industrial-grade use cases may require additional levels of fault tolerance to be considered.

For Cloud AI use cases, GPUs are often the go-to choice due to their high performance and ability to handle large volumes of data necessary for AI and deep learning. However, managing multiple GPUs on-premises can create a large demand on internal resources and be incredibly costly to scale. Alternatively, FPGAs offer a versatile solution that, while also potentially costly, provide both adequate performance as well as reprogrammable flexibility for emerging applications.

For Edge AI use cases, ASICs, FPGAs and NPUs are often more suitable. ASICs and FPGAs offer a significant advantage for Edge-AI applications, in terms of low-latency and power efficiency. NPUs, on the other hand, are commonly found in mobile devices and edge computing devices where power efficiency and the ability to run AI applications directly on the device (without cloud connectivity) are critical.

Everybody Wins if the Outcome of the ‘Wars’ is Democratization of Ethical and Sustainable AI

AI Wars, Campaigns, Battles, and Skirmishes provide a useful analogy for understanding the slew of announcements about the claimed AI (R)evolution. By viewing these developments through the lens of a war, we can better understand the dynamics at play and the direction in which AI is heading.

There will be winners, losers and casualties across the technology sector in the AI Wars, including the industrial automation and software sector. However, the AI Wars are not just about technology; they are about shaping the future of our world. Society will win if the outcome of the AI Wars is democratization of ethical and sustainable AI.

The massive investments in general-purpose GenAI Frontier Models are likely to benefit all organizations. However, the battlegrounds that may determine ‘revolutionary’ value realization for industrial organizations are likely to be in Edge AI hardware and software that can run a broad range of Industrial AI tools including ML, Neural Nets, SLMs (Small language models). Industrial AI’s value realizations is more likely to come from solutions that provide more accurate, predictable, and explainable ‘narrow’ AI, and lower-cost realization of industrial edge AI use cases.

Perhaps even more critical for Industrial AI may be the battle for AI talent that also understands the industrial domain – the ‘industrial-grade data scientist’. There’s already clear evidence that the battle for AI talent is one dimension in which leaders are embracing industrial AI to widen the digital divide. As AI continues to evolve and become more integrated into various industries, the demand for professionals who can bridge the gap between AI technology and practical application in specific industries will continue to grow.

Engage with ARC Advisory Group

Representative End User Clients
Representative Automation Clients
Representative Software Clients