Eugene de Souza, Red Hat

As large language models (LLMs) continue to grow in size and complexity, the demand for powerful hardware has skyrocketed. To meet this demand, the revenue for AI processors is projected to increase from $4 billion in 2020 to $38 billion by 2026. AI is also driving datacentre investment. Omdia’s latest Cloud and Datacentre Market Snapshot expects AI to surpass telecoms as the top server workload by 2027. Generative AI, with its vast datasets and billions of parameters that require training and fine-tuning, requires substantial compute, but managing AI infrastructure is complex and resource intensive.

It requires specialised IT skills like GPU optimisation, model deployment and scalability, but then there’s AI cloud. Also known as AI-as-a-Service (AIaas), AI cloud is infrastructure specifically designed for AI and ML workloads. It takes away the complexities around accessing advanced hardware like GPUs or Tensor Processing Units, which are prohibitively expensive and logistically challenging to deploy on-premises.