Harvard EPS-210

Machine Learning & Earth and Planetary Sciences

From Perceptrons to Foundation Models

Classic (1950sโ€“2000s)
Modern (2000sโ€“2017)
Cutting-Edge (2017โ€“Present)
Supervised Learning
Unsupervised Learning
Reinforcement Learning
Deep Learning
Generative Models
Scientific ML
Foundation Models
๐ŸŒ Earth & Planetary Science Applications
๐ŸŒ‹ Seismology & Geophysics
๐Ÿ›ฐ๏ธ Remote Sensing
๐ŸŒก๏ธ Climate Science
๐Ÿช Planetary Science
โ›ฐ๏ธ Geology & Geomorphology
Hover over ๐ŸŒ methods for applications
Traditional Machine Learning
๐Ÿ“Š
Supervised Learning
Learning from labeled examples with known outputs
Classification
Logistic Regression Naive Bayes k-NN SVM Decision Trees Random Forest AdaBoost XGBoost LightGBM CatBoost
Regression
Linear Regression Polynomial Regression Ridge / Lasso Elastic Net SVR Gradient Boosting Gaussian Process
Complexity
๐Ÿ”ฎ
Unsupervised Learning
Discovering hidden patterns without labeled data
Clustering
K-Means Hierarchical DBSCAN Mean Shift Spectral Clustering OPTICS HDBSCAN GMM
Dimensionality Reduction
PCA LDA SVD t-SNE UMAP ICA Autoencoders
Anomaly Detection
One-Class SVM Isolation Forest LOF COPOD
Complexity
๐ŸŽฎ
Reinforcement Learning
Learning through trial-and-error interactions with environment
Value-Based
Q-Learning SARSA DQN Double DQN Dueling DQN Rainbow
Policy-Based
REINFORCE A2C / A3C PPO TRPO SAC
Model-Based
Dyna-Q World Models MuZero Dreamer
Complexity
Deep Learning Architectures
๐Ÿง 
Neural Network Foundations
Core building blocks of deep learning
Basic Architectures
Perceptron MLP Hopfield Network Boltzmann Machine Deep Belief Nets ResNet DenseNet Capsule Networks
Activation Functions
Sigmoid Tanh ReLU Leaky ReLU ELU GELU SiLU/Swish
Regularization
Dropout Batch Norm Layer Norm RMSNorm Group Norm
Complexity
๐Ÿ‘๏ธ
Convolutional Networks
Spatial pattern recognition for images and signals
Image Classification
LeNet AlexNet VGGNet GoogLeNet ResNet EfficientNet ConvNeXt
Object Detection
R-CNN Fast R-CNN Faster R-CNN YOLO SSD YOLOv8 DETR
Segmentation
FCN U-Net DeepLab Mask R-CNN SAM
Complexity
๐Ÿ“œ
Sequence Models
Processing sequential and temporal data
Recurrent Networks
RNN LSTM GRU Bidirectional RNN Seq2Seq
Temporal Convolutions
WaveNet TCN
Attention Mechanisms
Bahdanau Attention Luong Attention Self-Attention Multi-Head Attention Flash Attention
State Space Models
S4 Mamba RWKV Hyena
Complexity
Transformer Architectures
โšก
Transformer Variants
Attention-based architectures revolutionizing AI
Encoder-Only (Understanding)
BERT RoBERTa ALBERT DeBERTa ELECTRA
Decoder-Only (Generation)
GPT GPT-2/3/4 LLaMA Claude Mistral Gemini
Encoder-Decoder
Original Transformer T5 BART mT5 UL2
Efficient Transformers
Longformer BigBird Performer Linformer FNet
Complexity
๐Ÿ–ผ๏ธ
Vision Transformers
Attention mechanisms for visual understanding
Image Models
ViT DeiT Swin Transformer BEiT MAE DINO DINOv2
Video Models
ViViT TimeSformer Video Swin
Complexity
๐ŸŒ
Multimodal Models
Bridging vision, language, and more
Vision-Language
CLIP ALIGN BLIP BLIP-2 Flamingo LLaVA GPT-4V
Audio-Visual
Whisper AudioLM SpeechT5
Complexity
Generative Models
๐ŸŽจ
Adversarial Networks
Generator-discriminator competition
GAN Variants
GAN DCGAN WGAN CycleGAN Pix2Pix StyleGAN StyleGAN2/3 BigGAN
Complexity
๐Ÿ”„
Variational Models
Probabilistic latent representations
VAE Family
VAE ฮฒ-VAE CVAE VQ-VAE VQ-VAE-2 NVAE
Complexity
โœจ
Diffusion Models
Iterative denoising for generation
Core Diffusion
DDPM DDIM Score-Based Stable Diffusion DALL-E 2/3 Imagen Midjourney
Video Diffusion
Make-A-Video Imagen Video Sora Gen-2
Advanced Variants
Latent Diffusion ControlNet DiT Consistency Models Flow Matching
Complexity
๐ŸŒŠ
Flow-Based Models
Invertible transformations for exact likelihood
Normalizing Flows
NICE RealNVP Glow Neural ODE FFJORD Continuous NF
Complexity
Scientific Machine Learning
๐Ÿ”ฌ
Physics-Informed ML
Embedding physical laws into neural networks
PDE Solvers
PINNs DeepONet Physics-Informed DeepONet Variational PINNs hp-VPINNs
Conservation Laws
Hamiltonian NN Lagrangian NN Neural Symplectic Dissipative SymODEN
Complexity
โˆ‚
Neural Operators
Learning mappings between function spaces
Operator Networks
FNO DeepONet U-FNO Geo-FNO GNOT Transolver
Spectral Methods
Spectral Neural Operator Wavelet Neural Operator Multipole Graph NO
Complexity
๐Ÿ•ธ๏ธ
Graph Neural Networks
Learning on non-Euclidean structures
Message Passing
GCN GraphSAGE GAT GIN PNA EGNN
Geometric Deep Learning
SchNet DimeNet PaiNN GemNet Equiformer
Graph Transformers
Graphormer Graph-BERT GPS TokenGT
Complexity
๐Ÿงฌ
Scientific Foundation Models
Domain-specific large-scale models
Biology & Chemistry
AlphaFold 2/3 ESMFold RFDiffusion AlphaMissense GNoME
Earth & Climate
FourCastNet Pangu-Weather GraphCast GenCast ClimaX Aurora
Complexity
Advanced Learning Paradigms
๐Ÿ”
Self-Supervised Learning
Learning representations without labels
Contrastive
SimCLR MoCo BYOL SwAV Barlow Twins VICReg
Masked Prediction
BERT MLM MAE BEiT Data2Vec I-JEPA
Complexity
๐ŸŽฏ
Meta-Learning
Learning to learn from few examples
Few-Shot Learning
Siamese Networks Matching Networks Prototypical Networks MAML Reptile Meta-SGD
Complexity
๐Ÿ—๏ธ
AutoML & NAS
Automated model design and optimization
Architecture Search
Random Search Bayesian Optimization NASNet DARTS EfficientNAS Once-for-All
Complexity
๐ŸŽ›๏ธ
Alignment & Fine-tuning
Aligning models with human preferences
Preference Learning
RLHF DPO PPO Fine-tuning Constitutional AI RLAIF
Parameter-Efficient
LoRA QLoRA Prefix Tuning Adapter Layers Prompt Tuning
Complexity