Transformer Architecture and Attention Mechanism
Master the foundations of Transformers and attention mechanisms to power AI applications. Understand tokenization, embeddings, and self-attention to enhance your AI skills.
Introduction
Learning Objectives
Strengths and Weaknesses in LLMs
Discriminative vs Generative AI
Quiz: Generative vs Discriminative AI
Transformers
Embeddings
Attention
Quiz: Introduction to Transformers
Similarity
Keys and Queries Matrices
Multi-Head Attention
Attention Mechanism
Sentiment Analysis
Applications
Quiz: Attention Mechanism
Feedforward Neural Networks
Types of Neural Networks
Softmax
How Do We Build Embeddings?
Tokenization
Positional Encoding
Transformer architecture
Revisiting LLMs
Quiz: Neural Networks and LLM Fundamentals
Introduction
Learning Objectives
How Does Tokenization Work?
Tokenizing a Song
Impact of Language Complexity
Punctuation Sensitivity