Understanding Derivatives

A derivative measures how a function changes as its input changes. Intuitively, it's the slope of the tangent line at a point on a curve.

Definition

For a function $f(x)$ , the derivative at point $x = a$ is:

f'(a) = \lim_{h \to 0} \frac{f(a + h) - f(a)}{h}

ℹ️ Intuition

Think of the derivative as the "instantaneous rate of change" — like your speedometer reading at a single moment in time, rather than average speed over a journey.

Basic Rules

The power rule is the most common derivative rule:

\frac{d}{dx} x^n = n x^{n-1}

For example, the derivative of $x^3$ is $3x^2$ .

Common Derivatives

Function	Derivative
$\sin x$	$\cos x$
$\cos x$	$-\sin x$
$e^x$	$e^x$
$\ln x$	$\frac{1}{x}$

Example in Code

Here's how you'd compute a numerical derivative in Python:

def numerical_derivative(f, x, h=1e-7):
    return (f(x + h) - f(x - h)) / (2 * h)
 
def f(x):
    return x ** 3
 
print(numerical_derivative(f, 2))  # ≈ 12.0

💡 Chain Rule

When composing functions, use the chain rule:

$\frac{d}{dx}[f(g(x))] = f'(g(x)) \cdot g'(x)$

Why Derivatives Matter

Derivatives are the foundation of optimization. When we set $f'(x) = 0$ , we find critical points — places where the function reaches a local maximum or minimum.

This is exactly what machine learning does: gradient descent follows the negative derivative to minimize a loss function.