The Chain Rule: Composing Derivatives in Multi-Dimensional Calculus

Give me six hours to chop down a tree and I will spend the first four sharpening the axe, Abraham Lincoln

Recall

Definition. Differentiability at a point. Let $f : ℝ^n \to ℝ^m$ be a function and let x be an interior point of the domain of f, $x \in \text{interior(dom f)} $. The function f is differentiable at x if there exists a matrix $Df(x) \in ℝ^{m \times n}$ (the Jacobian) that satisfies $\lim_{\substack{z \in \text{dom} f \\ z \neq x, z \to x}} \frac{||f(z) - f(x) - Df(x)(z-x)||_2}{||(z-x)||_2} = 0$

This matrix Df(x) is called the derivative or the Jacobian matrix of f at the point x.

Definition. A function f is called differentiable if its domain f (dom(f) ⊆ ℝⁿ) is open and f is differentiable at every point of its domain (∀x ∈ dom(f)).

Figure. For f(x,y)=x²+y², the red plane at (1,1) is the Jacobian’s linear approximation.

Chain Rule

The Chain Rule is one of the most fundamental and powerful tools in calculus. It governs how derivatives behave under composition of functions — whether in single-variable or multivariable settings. In higher dimensions, it elegantly expresses the derivative of a composite function as the matrix product of the Jacobians of the component functions.

Chain Rule. Let $f: U \subseteq \mathbb{R}^n \to \mathbb{R}^m$ be differentiable at a point $x \in U$, and let $g: V \subseteq \mathbb{R}^m \to \mathbb{R}^p$ be differentiable at the point $y = f(x) \in V$. Then, the composition $h = g \circ f: U \subseteq \mathbb{R}^n \to \mathbb{R}^p$ is also differentiable at x, and its derivative (Jacobian matrix) is given by $\boxed{Dh(x) = Dg(f(x)) \cdot Df(x)}$. This matrix equation is known as the multivariable chain rule.

In terms of Jacobian matrices: $J_h(x) = J_g(f(x)) \cdot J_f(x)$

Let’s verify the dimensions to ensure consistency:

• $Df(x)$ is the Jacobian of f: an m×n matrix (since f maps $\mathbb{R}^n \to \mathbb{R}^m$).
• $Dg(f(x))$ is the Jacobian of g evaluated at f(x): a p×m matrix (since g maps $\mathbb{R}^m \to \mathbb{R}^p$).
• Their product: (p×m)·(m×n) = p×n, which matches the expected size of $Dh(x)$. The dimensions align perfectly.

Calculus is built on the idea of linear approximation. If f is differentiable at x, it behaves locally like a linear map $Df(x)$: $f(x + h) \approx f(x) + Df(x) \cdot h$.

When we compose functions $h = g \circ f$, we are essentially composing these linear maps. Since the composition of linear maps corresponds to matrix multiplication, the derivative of the composition must be the product of the derivatives (Jacobian matrices).

The chain rule states that the overall change in h is a combination of the changes in f and g, propagated through the composition. Think of h=g∘f as a two-stage process:

1. Input x ∈ ℝⁿ is transformed by f into an intermediate output f(x) ∈ ℝ^m. A small change $\Delta x$ in input produces a change $\Delta y \approx Df(x) \cdot \Delta x$ in intermediate output.
2. This change $\Delta y$ produces a change $\Delta z \approx Dg(y) \cdot \Delta y$ in final output.
3. Substituting: $\Delta z \approx Dg(y) \cdot (Df(x) \cdot \Delta x) = [Dg(y) \cdot Df(x)] \cdot \Delta x$.The total sensitivity of h to small changes in x is therefore the composition (or product) of individual sensitivities.

Special Case

The multivariable chain rule unifies all differentiation through matrix multiplication of Jacobians. Here are the three most important cases, from simple to general.

Case 1: Single-Variable Composition (Scalar-Scalar)

Setting: $f: \mathbb{R} \to \mathbb{R}$ and $g: \mathbb{R} \to \mathbb{R}$.

Chain rule (the familiar chain rule from Calculus I): $\boxed{h'(x) = g'(f(x))\cdot f'(x)}$, h’(x) is a scalar, a product of rates: rate of change of inner function times rate of change of outer function.

Case 2: Directional Derivative Along a Curve

Setting $\gamma: \mathbb{R} \to \mathbb{R}^n$ (a curve, for every “moment” $t \in \mathbb{R}$, the function $\gamma(t)$ tells you exactly where a particle is in n-dimensional space. As t flows forward, the particle traces out a “trail”), $g: \mathbb{R}^n \to \mathbb{R}$ (scalar field, h(t) is the value of the field along the curve), $h(t) = g(\gamma(t)): \mathbb{R} \to \mathbb{R}$

Chain Rule: $\boxed{h'(x) = \nabla g(\gamma(t)) \cdot \gamma'(t)} = \sum_{i=1}^n \frac{\partial g}{\partial x_i}(\gamma(t))\cdot \gamma'(t)$, This states that the rate of change of g along the curve is the dot product of the gradient and the velocity vector.

The dot product measures how much of the velocity is aligned with the gradient.

This is maximized when $\gamma'(t)$ points paralell to $\nabla g$, e.g., if g is temperature and γ(t) is our trajectory, h’(t) tells us how fast the temperature is changing as we move — depending on both the temperature gradient and the velocity.

You experience the fastest increase in temperature when you walk directly uphill in the temperature landscape — i.e., in the direction of the gradient.

$\gamma'(t)$ is a n x 1 vector (velocity), the direction and speed along the curve (it tells us how fast in what direction the particle moves). $\nabla g(\gamma(t))$ is a 1 x n row (gradient), the direction of steepest ascent of g. h’(t) is a scalar thar represents the rate of change of g along the curve.

Case 3: Change of Variables (Coordinate Transformation)

Setting $f: \mathbb{R}^n \to \mathbb{R}^n$ (coordinate change or transformation), $g: \mathbb{R}^n \to \mathbb{R}$ (scalar field), $h(u) = g(f(u))$ (the scalar field expressed in the new coordiantes u).

The multivariable chain rule gives: $Dh(u) = Dg(f(u)) \cdot Df(u)$.

In matrix form, $\underbrace{\nabla h(u)^T}_{1 \times n} = \underbrace{\nabla g(f(u))^T}_{1 \times n} \cdot \underbrace{J_f(u)}_{n \times n}$

Taking transposes yields the gradient transformation rule: $\boxed{\nabla h(u) = J_f(u)^T \cdot \nabla g(f(u))}$. Gradients do not transform like vectors — they transform by the transpose of the Jacobian $J_f^T$ (covariant transformation), not the Jacobian itself.

We first compute the gradient in the old coordinates.Then, we transform it using the transpose of the Jacobian to express it in the new coordinates.

$J_f(u)$ is the Jacobian of coordinate transformation. $\nabla g(f(u))$ is the gradient in original coordinates. $\nabla h(u)$ is the gradient in new coordiantes.

General Scalar-Valued Composition

Setting: $f:\mathbf{R}ⁿ \to \mathbf{R}, g:\mathbf{R} \to \mathbf{R}$, so $h = g \circ f: \mathbf{R}ⁿ \to \mathbf{R}$

The chain rule reduces to: $\nabla \mathbb{(g∘f)}(x) = \mathbb{g'}(\mathbb{f}(x))\nabla \mathbb{f}(x)$ where $\mathbb{g'}(\mathbb{f}(x))$ is an ordinary derivative of g evaluated at f(x) and ∇f(x) is the n-vector of partials, $\frac{\partial h}{\partial x_i} = g'(f(x)) \cdot \frac{\partial f}{\partial x_i}, \quad i = 1, \ldots, n$

The gradient of the composition points in the same direction as $\nabla f(x)$, but its magnitude is scaled by how rapidly $g$ changes at $f(x)$ (the scaling factor is g’(f(x))). ∇h(x) is the scaled sensitivity of the composition.

$\nabla f(x)$ is the sensitivity of $f$ to each input direction. If tells us how fast f changes if you nudge (poke gently) x in each coordinate direction. $g'(f(x))$ is just a scalar — the ordinary derivative of g evaluated at the number f(x). It tells us how sensitive g is to changes in its input at that specific value f(x). $\nabla h(x)$ is the gradient of the composition. It is simply the gradient of f, scaled by how strongly g reacts at f(x).

Fundamental principle: The chain rule states that the total derivative of a composition is the product (composition) of the derivatives, representing how perturbations propagate through successive transformations.

Illustrative Examples

• Trigonometric Composition. Let $f(t) = (\cos t, \sin t)$ mapping $\mathbb{R} \to \mathbb{R}^2, g(x, y) = x^2 + y^2$ mapping $\mathbb{R}^2 \to \mathbb{R}$. Then, their composition, $h(t) = g(f(t)) = \cos^2 t + \sin^2 t = 1$

Let’s compute their Jacobians: $Df(t) = \begin{pmatrix} -\sin t \\ \cos t \end{pmatrix}$ (2 × 1 column vector), $Dg(x, y) = \begin{pmatrix} 2x & 2y \end{pmatrix}$ (1 × 2 row vector).

Composition: $Dh(t) = Dg(f(t)) \cdot Df(t) = \begin{pmatrix} 2\cos t & 2\sin t \end{pmatrix} \begin{pmatrix} -\sin t \\ \cos t \end{pmatrix} = -2\cos t \sin t + 2\sin t \cos t = 0$

Consistency: Indeed, since h(x) ≡ 1 constant, its derivate is zero h’(t) = 0. ✓

• Polynomial Composition. $f(x, y) = (x^2 + y^2, x - y)$ mapping $\mathbb{R}^2 \to \mathbb{R}^2$, $g(u, v) = u + v$ mapping $\mathbb{R}^2 \to \mathbb{R}$, so $h(x, y) = (x^2 + y^2) + (x - y)$

The Jacobian matrices are $Df(x, y) = \begin{pmatrix} 2x & 2y \\ 1 & -1 \end{pmatrix}$, $Dg(u, v) = \begin{pmatrix} 1 & 1 \end{pmatrix}$

Composition: $Dh(x, y) = \begin{pmatrix} 1 & 1 \end{pmatrix} \begin{pmatrix} 2x & 2y \\ 1 & -1 \end{pmatrix} = \begin{pmatrix} 2x+1 & 2y-1 \end{pmatrix}$

Direct computation of the gradient of $h$ confirms this: $h(x, y) = x^2 + y^2 + x - y$, therefore $\nabla h = (2x + 1, 2y - 1)$ ✓

• The Euclidean Norm. Consider $f:\mathbb{R}^n\to\mathbb{R}$ defined by $f(\vec{x})=\|\vec{x}\|^2=\sum_{i=1}^n x_i^2$ (the squared Euclidean norm) and $g:(0,\infty)\to\mathbb{R}$ defined by $g(y)=\sqrt{y}$ (the square‑root function).

Their composition $h(\vec{x})=g(f(\vec{x}))=\|\vec{x}\|$ is the Euclidean norm.

Since $f(\vec {x})=x_1^2+\cdots +x_n^2$, we have $Df(\vec {x})=\nabla f(\vec {x})^T=(2x_1,\ldots ,2x_n)$. This is a $1\times n$ row vector.

$g$ is differentiable for $y>0$, with derivative $g'(y)=\frac{1}{2\sqrt{y}}$.

For any $\vec{x}\neq\vec{0}$, we have $f(\vec{x})>0$, so the chain rule applies:

$Dh(\vec{x}) = g'(f(\vec{x})) \cdot Df(\vec{x}) = \frac{1}{2\sqrt{\|\vec{x} \|^2}} \cdot 2\vec{x}^T = \frac{1}{\|\vec {x} \|}(x_1,\ldots ,x_n) = \frac{\vec {x}}{\|\vec{x} \|}$

The gradient of the norm points directly away from the origin. The vector $\vec {x}$ points from the origin to the point x. Normalizing it gives the unit radial direction.

The magnitude of the gradient is 1. The Euclidean norm is differentiable everywhere except the origin, and its gradient is the unit radial vector.

• The Softmax Function from Chain Rule. Consider the composition: $f: \mathbb{R}^n \to \mathbb{R}, f(\vec{x}) = \sum_{i=1}^n e^{x_i}$ (sum of exponentials), $g: \mathbb{R}^+ \to\mathbb{R}, g(y) = \ln(y)$ (natural logarithm). Then, $h(\vec{x}) = \ln(\sum e^{x_i})$ is known as the log-sum-exp function.

1. Jacobian of f. Since f maps to $\mathbb{R}$, its derivative is a row vector: $Df(\vec{x}) = (e^{x_1}, e^{x_2}, \ldots, e^{x_n})$. Equivalently, $\nabla f(\vec{x}) = \begin{pmatrix}e^{x_1}\\[6pt] e^{x_2}\\[6pt] \cdots \\[6pt] e^{x_n}\end{pmatrix}$.
2. Derivative of g, $g'(y) = 1/y$. Evaluated at $y = f(\vec{x}), g'(f(\vec{x})) = \frac{1}{\sum_{i=1}^n e^{x_i}}$.
3. Apply chain rule: $Dh(\vec{x}) = g'(f(\vec{x})) \cdot Df(\vec{x}) = \frac{1}{\sum_{i=1}^n e^{x_i}} (e^{x_1}, \ldots, e^{x_n})$.
4. Taking transposes to get the gradient: $\nabla h(\vec{x}) = \begin{pmatrix}\frac{e^{x_1}}{\sum_{i=1}^n e^{x_i}}\\[6pt] \frac{e^{x_2}}{\sum_{i=1}^n e^{x_i}}\\[6pt] \cdots \\[6pt] \frac{e^{x_n}}{\sum_{i=1}^n e^{x_i}}\end{pmatrix}$
5. Each component is $\frac{\partial h}{\partial x_i} = \frac{e^{x_i}}{\sum_{k=1}^n e^{x_k}}$.
6. This is the softmax function used in machine learning! $\sigma: \mathbb{R}^n \to \mathbb{R}^n, \sigma(\vec{x}_i) = \frac{e^{x_i}}{\sum_{k=1}^n e^{x_k}}$

• Chain Rule in Polar Coordinates

1. We transform a scalar field from Cartesian to polar coordinates using the composition: Coordinate map, $\mathbb{R}^2 \to \mathbb{R}^2, F(r, \theta) = (r\cos\theta, r\sin\theta)$ (Polar to Cartesian); $g: \mathbb{R}^2 \to \mathbb{R}, g(x, y)$, a function in Cartesian coordinates. Then, $h = g \circ F: \mathbb{R}^2 \to \mathbb{R}^2, h(r, \theta) = g(r\cos\theta, r\sin\theta)$ (same function in Polar coordinates).
2. Jacobian of F: The Jacobian matrix DF contains all partial derivatives of F: $DF = \begin{pmatrix} \cos\theta & -r\sin\theta \\ \sin\theta & r\cos\theta \end{pmatrix}$
3. Gradient of g: $Dg = (\nabla g)^T = (g_x, g_y)$
4. By the Chain Rule: $Dh = Dg \cdot DF = \begin{pmatrix} g_x & g_y \end{pmatrix} \begin{pmatrix} \cos\theta & -r\sin\theta \\ \sin\theta & r\cos\theta \end{pmatrix}$
5. Resulting Partial Derivatives: $\frac{\partial h}{\partial r} = g_x \cos\theta + g_y \sin\theta$, $\frac{\partial h}{\partial \theta} = -g_x r\sin\theta + g_y r\cos\theta$

Practical Component Form

While matrix multiplication gives a compact and elegant version of the chain rule ($J_{G\circ F}(x) = J_G(F(x)) J_F(x)$). However, in practice we often need individual partial derivatives, not the whole matrix. That leads to the component form of the chain rule.

Suppose $F : \mathbb{R}^n \to \mathbb{R}, y_j = f_j(x_1,\ldots,x_n), G : \mathbb{R}^m \to \mathbb{R}^p, z_k = g_k(y_1,\ldots,y_m)$.

The composition is: $H = G \circ F, \quad z_k = h_k(x_1,\ldots,x_n).$

Chain Rule (Component Form). For each output component $z_k$ and each input variable $x_i: \boxed{ \frac{\partial z_k}{\partial x_i} = \sum_{j=1}^{m} \frac{\partial z_k}{\partial y_j} \frac{\partial y_j}{\partial x_i}}$

Each variable $x_i$ influences $z_k$ through every intermediate variable $y_j$. We need to add up all possible “paths” from $x_i$ to $z_k$.

Application to Complex Analysis

A complex function $f(z) = u(x, y) + i v(x, y)$ can be seen as a vector field mapping $F : \mathbb{R}^2 \to \mathbb{R}^2, F(x, y) = (u(x, y), v(x, y))$.

Similarly, if $g(w) = a(s,t) + i b(s,t)$, then $G(s,t) = (a(s,t), b(s,t))$.

If we compose them, h(z) = g(f(z)), in real variables this becomes $H = G \circ F$ with $H : \mathbb{R}^2 \to \mathbb{R}^2.$

The Jacobian matrices satisfy: $J_H(x,y) = J_G(F(x,y)) J_F(x,y)$. This is ordinary matrix multiplication.

The Special Structure of Holomorphic Functions

If $f$ is holomorphic (complex differentiable), its real and imaginary parts must satisfy the Cauchy–Riemann equations: $u_x = v_y, \quad u_y = -v_x.$

The local behavior of any transformation $\mathbb{R}^2 \to \mathbb{R}^2$ is captured by its Jacobian matrix. By substituting the Cauchy-Riemann equations into the standard Jacobian for $f$, we force it into a very specific, highly structured form: $J_F = (\begin{smallmatrix}u_x & -v_x\\\ v_x & u_x\end{smallmatrix})$

Let $a = u_x, \quad b = v_x.$ Then, the Jacobian becomes: $J_F = (\begin{smallmatrix}a & -b\\\ b & -a\end{smallmatrix}).$

This special matrix shape perfectly mirrors the behavior of complex numbers. We can rewrite the complex number $a + ib$ in polar form as $re^{i\theta}$, where $r = \sqrt{a^2 + b^2}$ is the magnitude and $\theta$ is the angle.

When this matrix multiplies a vector, it acts exactly like multiplying by the complex number $a + ib$. It performs two geometric actions simultaneously:

1. Scaling the vector by a factor of $r$.
2. Rotating the vector counterclockwise by an angle $\theta$.

Therefore, the derivative of a holomorphic function acts locally as a pure rotation combined with a pure scaling. It does not shear, stretch unevenly, or reflect the space. This is why holomorphic functions with non-zero derivatives are called conformal —they preserve angles between intersecting curves.

Now, suppose we have two holomorphic functions, $f$ and $g$. By the Multivariable Chain Rule, the Jacobian of their composition $(f \circ g)$ is simply the matrix product of their individual Jacobians: $J_{f \circ g} = J_f \cdot J_g$

Since both functions are holomorphic, both of their Jacobian matrices share the same special “rotation and scaling” structure. Let’s multiply them:

$$\begin{pmatrix} a & -b \\ b & a \end{pmatrix} \begin{pmatrix} c & -d \\ d & c \end{pmatrix} = \begin{pmatrix} ac - bd & -(ad + bc) \\ ad + bc & ac - bd \end{pmatrix}$$

Notice the structure of the resulting matrix! It is perfectly of the form $\begin{pmatrix} X & -Y \\ Y & X \end{pmatrix}$, where $X = ac - bd$ and $Y = ad + bc$.

Because the product matrix retains this exact Cauchy-Riemann structure, the composition $(f \circ g)$ is strictly holomorphic. In other words, composition preserves holomorphicity.

This beautiful matrix arithmetic corresponds exactly to expanding the product of two complex numbers: $(a+ib)(c+id) = (ac - bd) + i(ad + bc)$