Neural Networks,
from first principles.
Architecture shapes everything. Not the tools, not the framework, not the GPU count — the architecture. These sessions examine how networks are built and why specific choices create specific outcomes.
Each session treats one concept with enough depth to be genuinely useful. No single formula fits all problems. The sessions below reflect that honestly.
Six sessions, one coherent arc
The curriculum progresses from mathematical foundations through practical implementation. Each session is self-contained but designed to build on the one before it.
What a neuron actually computes
The perceptron is not a metaphor. It performs a weighted sum, applies a nonlinearity, and outputs a scalar. Starting there prevents the vague intuitions that cause confusion later.
Depth, width, and the capacity question
Deeper is not always better. Layer count, hidden unit count, and skip connections interact in ways that depend on dataset structure, not on theoretical guarantees.
Gradient flow through the computation graph
Vanishing gradients are not a mystery once you trace how partial derivatives chain through layers. Session three works through the math, then demonstrates it in NumPy without relying on autograd.
Transformers: the mechanism before the hype
Scaled dot-product attention is four matrix operations. The session derives each one from the problem it was designed to solve, then builds a minimal transformer encoder from scratch.
Spatial priors and inductive bias in CNNs
Weight sharing is not a trick to reduce parameters — it is a structural assumption about the data. Session five examines ResNet-style residual paths and what motivated each design choice in 2015.
Reading an architecture paper without prior exposure
The final session is a live reading of a recent architecture paper. The goal is not to summarise the result but to demonstrate how to locate the architectural contribution inside dense notation.
Instructor
Orest spent four years building production vision systems before moving into curriculum work. The sessions reflect his preference for derivation over demonstration.
Format note
Sessions run live with a recorded archive available within 48 hours. Participants across all time zones have equal access to materials and Q&A transcripts.
Contact usdef forward(x, W, b):
z = x @ W + b
# ReLU activation
return np.maximum(0, z)
def backward(dout, z, x, W):
dz = dout * (z > 0)
dW = x.T @ dz
dx = dz @ W.T
return dx, dW