In parallel with these technological changes were advances in a core component that had been part of AI since the beginning: neural networks.

A type of mathematical model inspired by the human brain, neural networks take input (i.e., pictures of cars), pass it through neurons, or nodes within hidden layers of the network, to produce an output (a model of a “car”).

How AI is Built

The first major step to build an AI model is called training, which can require immense computational speed and scale (or a lot of patience).

By traversing the neural network forward and backward in a rapid-fire feedback loop, and fitting it to known aspects of the dataset, the AI model “learns” internal variables, also known as parameters, which are adjusted during training to help the model make accurate predictions. The behavior of the network depends on the strengths (or weights) of the connections between neurons.

Next is fine-tuning, where the pre-trained AI model is adjusted to incorporate new information or better fit a specific dataset or task.

Once a model is trained, it can be applied again and again for inference, which uses the model to take new inputs and generate new, relevant outputs.

From AI that Makes Decisions to AI that Makes Stuff

Deep learning supercharged neural networks by incorporating multiple layers that enhance accuracy and training efficiency. Neural networks and deep learning are behind a lot of the most popular and powerful tools in AI. (They’re not the only ones: Knowledge graphs, reinforcement learning and neuro-symbolic AI are a few other approaches having an impact.)

The most recent technological leaps delivered generative AI (GenAI), where machines can craft human-like responses and original visual content – sort of like having a super-smart, creative assistant that can respond to your questions and iterate on its creations. Today’s GenAI comes in different “flavors,” with each made possible by several key innovations.

Transformers are a type of neural network architecture introduced by Google in 2017. Transformers are trained on large bodies of text and require no human supervision as they process entire sentences simultaneously, improving the understanding of words in context by paying attention to the most relevant words. Examples: GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Transformers). Transformers, and the large language models (LLMs) they have enabled, revolutionized language translation, chatbots and content creation. They’ve even demonstrated emergent capabilities in text generation, coding and weather prediction. Examples: GPT-3, GPT-4.

For image generation, Diffusion models – conceptualized at Stanford in 2015 and inspired by diffusion processes in physics – have shown incredible progress and abilities. Examples: DALL-E 2, Stable Diffusion.

We’re still in the early days of AI. The field’s potential to transform almost every aspect of life is driving fervent investment in computing capacity and research and development; at the same time, it is generating excitement and concern globally.