Key Concepts of Generative AI

Before we go deeper into models, embeddings, and architectures, we need to pause and build a strong mental foundation.

Generative AI systems are not magic. They are built from a small set of core ideas, reused again and again at scale.

If you understand these concepts clearly, you will be able to reason about any GenAI system — even ones you’ve never seen before.

Important Note on Practicing Code (Read This Carefully)

Throughout this module, you’ll see Python code examples.

These examples are written so that you can:

• Run them in a local Python environment
• Use a Jupyter notebook
• Use Google Colab

You are not expected to memorize code.

Instead, you should:

• Understand what problem the code is solving
• Rewrite parts of it in your own way
• Change inputs and observe outputs

That is how engineers actually learn.

Concept 1: Tokens

Generative AI models do not see text the way humans do.

They do not read sentences. They read tokens.

A token is a chunk of text. It might be:

• A full word
• A part of a word
• A symbol or punctuation

Understanding tokens matters because:

• Models predict tokens, not sentences
• Cost and limits are measured in tokens
• Context length depends on tokens

Thinking Before Coding

Before writing code, think about this:

How would you split a sentence into smaller pieces?

Let’s simulate a very simple tokenization process.

Writing the Code

sentence = "Generative AI is powerful"
tokens = sentence.split()

print(tokens)
  

This is not how real tokenizers work, but it helps you understand the idea.

Real models use much more advanced tokenization, which you’ll learn later.

Concept 2: Context

Context is the information the model can see before generating the next token.

If the context is short, the model has less information. If it’s long, the model can reason better — up to a limit.

Thinking Before Coding

Ask yourself:

What happens if we give the model more words before asking it to continue?

Writing the Code

context_short = ["AI"]
context_long = ["Generative", "AI", "models", "learn", "patterns"]

print("Short context:", context_short)
print("Long context:", context_long)
  

In real GenAI systems, context size affects:

• Quality of output
• Latency
• Cost

Concept 3: Parameters

Parameters are the internal numbers that store what a model has learned.

When you hear:

“This is a 7B or 70B parameter model”

It refers to how much capacity the model has to store patterns.

More parameters generally mean:

• More expressive power
• Higher cost
• More compute required

Concept 4: Probability and Sampling

Generative models do not output a single answer.

They output a probability distribution over possible next tokens.

Sampling decides how the next token is chosen from that distribution.

Thinking Before Coding

Think about this:

If multiple next words are possible, how do we choose one?

Writing the Code

import random

choices = ["data", "patterns", "models"]
print(random.choice(choices))
  

Real systems use controlled sampling, not pure randomness.

Concept 5: Temperature

Temperature controls how random or deterministic the output is.

Lower temperature:

• More predictable
• Less creative

Higher temperature:

• More diverse outputs
• Higher risk of errors

You’ll tune this based on the application.

How These Concepts Work Together

In real GenAI systems:

• Tokens define what the model sees
• Context defines what it remembers
• Parameters define what it knows
• Sampling defines what it produces

Every GenAI product is a balance of these elements.

Practice

Quick Quiz

Recap: Tokens, context, parameters, and sampling form the core mechanics of all GenAI systems.

Next up: We’ll build a full Generative AI pipeline and see how these concepts connect end to end.