Average speed is easy. If a car covers 120 miles in 2 hours, its average speed is

where \(\Delta\) (capital Greek delta) means "change in". But a speedometer claims something stronger: at precisely this instant, the car is travelling at 63 mph. What can that mean? At a single instant the car travels zero distance in zero time, and \(\frac{0}{0}\) is meaningless. Yet the speedometer reading is clearly not nonsense — a police radar gun and the car agree on it, and physics depends on it.

Here is a concrete version. A car accelerates steadily so that, \(t\) hours after setting off, it has travelled

How fast is it going at \(t = 1.5\) hours?

We cannot compute speed over an interval of zero length, but we can compute average speed over short intervals from \(t = 1.5\) to \(t = 1.5 + h\), and watch what happens as \(h\) shrinks: Try it: Watch the secants shrink.

The averages are homing in on 63. No single interval gives exactly 63 — every one gives a bit more, because the car is speeding up during the interval — but by making \(h\) small enough we can get as close to 63 as we like. So we define the instantaneous speed to be 63 mph: the value the average speeds approach as the interval shrinks to nothing.

Algebra confirms the table and shows exactly why:

For every nonzero \(h\), the average speed is \(63 + 21h\). As \(h\) gets closer to 0, this gets closer to 63. We never set \(h = 0\) (the original fraction would be \(\frac00\)); we ask what the expression approaches.

On the distance–time graph, each average speed is the slope of a secant line through two points of the curve. As the second point slides towards the first, the secants pivot towards the tangent line, which just touches the curve at \(t = 1.5\). Its slope is the instantaneous speed. This is the derivative, and it is the subject of Chapter 17. Before we can do that properly, we need to understand the process at its heart: the limit.

The pattern is general. We have a function \(f\) and a point \(a\), and we want to know what value \(f(x)\) approaches as \(x\) gets closer and closer to \(a\) — often precisely because we cannot (or do not want to) evaluate \(f(a)\) itself.

Two features of the definition are crucial.

Consider Try it: Explore the hole at 2.

At \(x = 2\) this is \(\frac00\): undefined. But a table of values near 2 tells a clear story:

From both sides, \(f(x)\) approaches 4. The algebra explains it: for \(x \ne 2\),

so the graph is the line \(y = x + 2\) with a single point missing at \((2, 4)\). Therefore \(\lim_{x\to2} f(x) = 4\), even though \(f(2)\) does not exist.

This is exactly the situation of the car: the average-speed formula \(\frac{d(1.5 + h) - d(1.5)}{h}\) is undefined at \(h = 0\), but equals \(63 + 21h\) everywhere else, so its limit as \(h \to 0\) is 63. Every derivative is a limit of this "0/0" type. That is why limits come first.

Try it: See a table lie.

Sometimes a function behaves differently on the two sides of a point. The sign function \(\frac{|x|}{x}\) equals \(-1\) for every negative \(x\) and \(+1\) for every positive \(x\) (and is undefined at 0). Try it: One-sided limits of |x|/x.

For the sign function, \(\lim_{x\to0^-} \frac{|x|}{x} = -1\) and \(\lim_{x\to0^{+}}\frac{|x|}{x} = 1\). They disagree, so \(\lim_{x\to0}\frac{|x|}{x}\) does not exist.

One-sided limits also handle edges of domains: \(\sqrt x\) is only defined for \(x \ge 0\), and \(\lim_{x\to0^{+}}\sqrt x = 0\) is the only sensible question to ask at 0.

Tables are slow and unreliable. We want rules that compute limits of complicated expressions from limits of simple ones.

Suppose \(\lim_{x\to a} f(x) = L\) and \(\lim_{x\to a} g(x) = M\) both exist, and \(c\) is a constant. Then:

together with the two basic limits \(\lim_{x\to a} c = c\) and \(\lim_{x\to a} x = a\).

These are intuitive — if \(f\) is near \(L\) and \(g\) is near \(M\), then \(f + g\) is near \(L + M\) — and they can be proved from the formal definition in Section 16.10. The essential condition is in the first line: both limits must exist. The quotient law has a second condition: the denominator’s limit must not be zero. Almost every interesting limit violates that second condition, which is why we need techniques.

Applying the laws repeatedly to a polynomial gives \(\lim_{x\to a} p(x) = p(a)\): just substitute. The same holds for a rational function \(\frac{p(x)}{q(x)}\) whenever \(q(a) \ne 0\), and, as we will see under continuity, for every "standard" function (roots, exponentials, logarithms, sine, cosine) at points of its domain.

Worked example. \(\lim_{x\to3}\dfrac{x^2 + 2x}{x - 1}\).

Always try substitution first. If it gives a number, you are done. If it gives \(\frac{c}{0}\) with \(c \ne 0\), the function blows up (examine the signs on each side). If it gives \(\frac{0}{0}\), you have an indeterminate form — the answer could be anything, and you need to do more work.

If substituting \(x = a\) makes a polynomial numerator and denominator both zero, then by the factor theorem (Chapter 7) both contain the factor \((x - a)\). Factor, cancel it (legitimate because \(x \ne a\) inside a limit), then substitute.

Worked example. \(\lim_{x\to3}\dfrac{x^2 - 9}{x^2 - x - 6}\). Substitution gives \(\frac{0}{0}\). Try it: Check this limit.

Worked example. \(\lim_{x\to1}\dfrac{x^3 - 1}{x - 1}\). Use \(x^3 - 1 = (x - 1)(x^2 + x + 1)\): Try it: Check this limit.

When square roots are involved, multiply by the conjugate (the same expression with the sign between the terms flipped) to use \((A - B)(A + B) = A^2 - B^2\) and clear the root.

Worked example. \(\lim_{x\to0}\dfrac{\sqrt{x + 9} - 3}{x}\). Substitution: \(\frac{3 - 3}{0} = \frac00\). Try it: Check this limit.

A table agrees: at \(x = 0.001\), \(\frac{\sqrt{9.001} - 3}{0.001} \approx 0.166662\).

Compound fractions are simplified by finding a common denominator.

Worked example. \(\lim_{x\to0}\dfrac{\frac{1}{x + 2} - \frac12}{x}\).

so the limit is \(\frac{-1}{2(0 + 2)} = -\frac14\).

Some limits resist algebra entirely. The most important one is

Substitution gives \(\frac00\), and there is nothing to factor. Yet this limit is the key to the derivative of sine (and so to all of oscillations, waves, and signal processing). A table, with \(x\) in radians, suggests the answer:

It looks like 1. To prove it we need a new tool.

The intuition is two police officers escorting a suspect: if both officers walk into the station, so does the suspect between them.

Warm-up example. \(\lim_{x\to0} x^2\sin(1/x)\). We know nothing useful about \(\sin(1/x)\) near 0 — it oscillates wildly — except that it stays between \(-1\) and 1. Multiplying by \(x^2 \ge 0\): Try it: Squeeze x² sin(1/x) to 0.

Both outer functions tend to 0, so \(\lim_{x\to0} x^2\sin(1/x) = 0\). The wild oscillation is crushed by the shrinking factor.

Take \(0 < x < \frac{\pi}{2}\) and draw an angle \(x\) (in radians) at the centre \(O\) of a unit circle. Let \(A = (1, 0)\), let \(B = (\cos x, \sin x)\) be the point on the circle, and let \(T\) be the point where the line \(OB\) meets the vertical tangent line at \(A\); by similar triangles \(AT = \tan x\). Try it: Tabulate sin x / x.

The three regions are nested, so their areas are in order:

(The triangle \(OAB\) has base 1 and height \(\sin x\); the sector has area \(\frac12r^2\theta\) with \(r = 1\), which is where radians are essential; triangle \(OAT\) has base 1 and height \(\tan x\).) Now manipulate:

As \(x \to 0^{+}\), \(\cos x \to 1\). The squeeze theorem gives \(\lim_{x\to0^{+}}\frac{\sin x}{x} = 1\). For negative \(x\), note that \(\frac{\sin(-x)}{-x} = \frac{-\sin x}{-x} = \frac{\sin x}{x}\) (the function is even), so the left-hand limit is the same. Therefore

Geometrically: for small angles, the arc length \(x\) and the half-chord \(\sin x\) are almost the same. This is the small-angle approximation \(\sin x \approx x\), used throughout physics and engineering (the simple pendulum, optics, surveying).

The small-angle approximation \(\sin x \approx \tan x \approx x\) (radians) turns angles into lengths with no trigonometry.

Worked example. \(\lim_{x\to0}\dfrac{\sin 3x}{x}\). Substitute \(u = 3x\) (so \(u \to 0\) as \(x \to 0\)): Try it: Check this limit.

Worked example. \(\lim_{x\to0}\dfrac{1 - \cos x}{x}\). Multiply by the "conjugate" \(1 + \cos x\): Try it: Check this limit.

These two limits are exactly what is needed in Chapter 17 to show that the derivative of \(\sin x\) is \(\cos x\).

For most functions you meet, the limit at a point is just the value at the point: \(\lim_{x\to2} x^2 = 4 = 2^2\). The interesting cases are where this fails. We want a name for the good behaviour, and a catalogue of the ways it can break.

A function is continuous if you can draw its graph without lifting your pen: no holes, no jumps, no vertical asymptotes, no wild oscillations. Small changes in input produce small changes in output. Physical quantities — the temperature in a room over time, the position of a car — are continuous; a step change in price at a tax threshold is not.

Condition 3 is the heart of it: the limit is what you get by substituting. That is why direct substitution works — it works precisely for continuous functions.

Each condition can fail, giving a catalogue that maps onto the pictures earlier in the chapter:

A great deal follows from the limit laws:

So \(e^{\sin x} + \sqrt{x^2 + 1}\) is continuous everywhere, with no work. The composition rule also lets limits pass inside continuous functions: \(\lim_{x\to a} g(f(x)) = g\left(\lim_{x\to a} f(x)\right)\) when \(g\) is continuous at the inner limit.

Worked example (making a piecewise function continuous). For which value of \(k\) is

continuous everywhere? Each piece is a polynomial, hence continuous; the only danger is the join at \(x = 2\).

So \(k = 1\).

Continuity is the promise that a small change in input gives a small change in output. Discontinuities are where that promise breaks, and they often matter more than the smooth parts.

Does \(x^3 - x - 2 = 0\) have a solution? It is a cubic with no rational roots (the candidates \(\pm1, \pm2\) all fail), so factoring will not help. But \(f(1) = 1 - 1 - 2 = -2 < 0\) and \(f(2) = 8 - 2 - 2 = 4 > 0\). The graph of a continuous function cannot get from below the axis to above it without crossing. That intuition is a theorem.

The IVT tells you a root exists; it does not tell you where. It also needs continuity: \(f(x) = \frac1x\) has \(f(-1) = -1 < 0 < 1 = f(1)\) but no root, because it is not continuous on \({[}-1, 1]\).

The IVT becomes an algorithm with one observation. If there is a sign change on \({[}a, b]\), evaluate \(f\) at the midpoint \(m\). Either \(f(m) = 0\) (done), or the sign change is in \({[}a, m]\), or it is in \({[}m, b]\). Keep the half with the sign change and repeat. The interval known to contain a root halves every step — it is binary search on the real line.

Worked example by hand. \(f(x) = x^3 - x - 2\) on \({[}1, 2]\). Try it: Step through this table.

(Step 1 in detail: \(f(1.5) = 3.375 - 1.5 - 2 = -0.125\). Since \(f(1.5) < 0 < f(2)\), the root lies in \({[}1.5, 2]\).) After 7 steps the root is pinned inside an interval of width \(2^{-7} \approx 0.0078\): it is about 1.52.

33 halvings shrink the interval from width 1 to about \(10^{-10}\), as predicted: \(\log_2 10^{10} \approx 33.2\).

What happens to \(f(x)\) as \(x\) grows without bound? This is the question of long-run behaviour: the steady state of a cooling cup of coffee, the terminal velocity of a falling object, the cost of an algorithm on enormous inputs.

The basic facts: \(\lim_{x\to\infty}\frac{1}{x} = 0\), and more generally \(\lim_{x\to\infty}\frac{1}{x^p} = 0\) for any \(p > 0\). Everything else is reduced to these.

A rational function at infinity is a \(\frac{\infty}{\infty}\) indeterminate form. The technique: divide numerator and denominator by the highest power of \(x\) in the denominator, so every term becomes a constant or tends to 0.

Worked example. \(\lim_{x\to\infty}\dfrac{3x^2 - 5x + 1}{2x^2 + 7}\). Try it: Limit at infinity, tabulated.

The lesson: at infinity, only the leading terms matter. \(3x^2 - 5x + 1\) "behaves like" \(3x^2\) and \(2x^2 + 7\) like \(2x^2\).

Finding vertical asymptotes. These occur where the denominator is zero but the numerator is not (after cancelling common factors — a common factor gives a hole instead). For

the numerator is never zero, so \(x = 1\) and \(x = -1\) are vertical asymptotes; the degrees are equal, so \(y = \frac{2}{1} = 2\) is the horizontal asymptote.

Worked example (∞ − ∞). \(\lim_{x\to\infty}\left(\sqrt{x^2 + x} - x\right)\). Both terms grow without bound; their difference is indeterminate. Rationalise: Try it: Tabulate √(x²+x) − x.

Beyond polynomials, the essential facts for limits at infinity (proved with tools from Chapter 18) are that, for any constants \(p > 0\) and \(b > 1\),

In words: logarithms grow slower than any power, and powers grow slower than any exponential. \(\ln x\) loses to \(x^{0.01}\) eventually (though "eventually" can be astronomically far out), and \(x^{100}\) loses to \(1.01^x\).

Limits at infinity describe where things settle, which is often the only question that matters in the long run.

Compound interest, compounded ever more frequently, leads to

a \(1^\infty\) indeterminate form: the base tends to 1 but the exponent grows. This is one standard definition of \(e\) (Chapter 12). We will come back to it below, because a computer gets it spectacularly wrong.

For 150 years after Newton and Leibniz, calculus worked spectacularly while its foundations were shaky: quantities were "infinitely small" but not zero, vanishing but not vanished. Bishop Berkeley mocked these as "the ghosts of departed quantities". The resolution, by Cauchy, Bolzano, and above all Weierstrass in the 19th century, was to eliminate the vague words "approaches" and "infinitely small" entirely, replacing them with statements about ordinary inequalities.

Think of proving \(\lim_{x\to a} f(x) = L\) as a game between a challenger and a prover.

The limit is \(L\) if the prover can win every round, no matter how small an \(\varepsilon\) the challenger chooses. Usually the prover wins by giving a formula for \(\delta\) in terms of \(\varepsilon\).

A linear function. Prove \(\lim_{x\to3}(2x + 1) = 7\). Try it: Play ε–δ for 2x + 1.

Scratch work (finding \(\delta\)): we need \(|(2x + 1) - 7| < \varepsilon\). Simplify: \(|2x - 6| = 2|x - 3| < \varepsilon\) exactly when \(|x - 3| < \frac{\varepsilon}{2}\). So choose \(\delta = \frac{\varepsilon}{2}\).

Proof: given \(\varepsilon > 0\), let \(\delta = \frac{\varepsilon}{2}\). If \(0 < |x - 3| < \delta\), then

The slope 2 means outputs move twice as fast as inputs, so the input window must be half as wide as the output window. If the challenger demands \(\varepsilon = 0.001\), the prover answers \(\delta = 0.0005\).

A quadratic. Prove \(\lim_{x\to2}x^2 = 4\). Try it: Play ε–δ for x².

Scratch work: \(|x^2 - 4| = |x - 2|\,|x + 2|\). The factor \(|x - 2|\) is what we control; the factor \(|x + 2|\) varies, so we bound it. If we insist \(\delta \le 1\), then \(|x - 2| < 1\) means \(1 < x < 3\), so \(|x + 2| < 5\). Then \(|x^2 - 4| < 5|x - 2|\), which is below \(\varepsilon\) when \(|x - 2| < \frac{\varepsilon}{5}\).

Proof: given \(\varepsilon > 0\), let \(\delta = \min\left(1, \frac{\varepsilon}{5}\right)\). If \(0 < |x - 2| < \delta\), then \(|x + 2| < 5\) (because \(\delta \le 1\)) and

Sequences: the ε–N version. For sequences the "closeness of input" is replaced by "far enough along": \(\lim_{n\to\infty} a_n = L\) means for every \(\varepsilon > 0\) there is an \(N\) such that \(|a_n - L| < \varepsilon\) for all \(n > N\). Example: \(\frac1n \to 0\), because given \(\varepsilon\), any \(N \ge \frac{1}{\varepsilon}\) works: \(n > N\) gives \(\frac1n < \frac1N \le \varepsilon\).

The car example suggests a tempting shortcut for computing a derivative: just evaluate the average rate of change with a tiny \(h\). Try it on \(\sin\) at \(x = 1\), where the true answer is \(\cos 1 \approx 0.5403023\): Try it: Shrink h yourself.

Mathematically, shrinking \(h\) always helps. Computationally, the error falls until \(h \approx 10^{-8}\) and then rises, until at \(h = 10^{-16}\) the answer is 0 — completely wrong. Two errors compete:

At \(h = 10^{-20}\) it is worse still: \(1 + 10^{-20}\) rounds to exactly \(1.0\) (the last line prints True), so the numerator is exactly zero. The best compromise is around \(h \approx \sqrt{\text{machine epsilon}} \approx 10^{-8}\), giving only about 8 correct digits. Better formulas (central differences), and better approaches altogether (automatic differentiation, as used in every ML framework), are the subject of Chapter 32.

The \((1 + \frac1n)^n \to e\) limit fails in the same way: Try it: Watch (1 + 1/n)ⁿ collapse.

Approaching \(e\) nicely, then drifting away, then collapsing to 1 once \(1 + 10^{-16}\) rounds to 1. The mathematical limit is a statement about exact arithmetic. Floating point approximates it well only in a middle range. This is why library functions like math.log1p and math.expm1 exist: they compute \(\ln(1 + x)\) and \(e^x - 1\) accurately for tiny \(x\) without forming \(1 + x\).

Many algorithms produce a sequence of approximations \(x_0, x_1, x_2, \ldots\) that should converge to the answer: square roots, Newton’s method, gradient descent, PageRank, iterative linear solvers. "The algorithm converges" means precisely \(\lim_{n\to\infty}x_n = x^*\). Heron’s ancient method for \(\sqrt a\) iterates \(x_{n+1} = \frac12\left(x_n + \frac{a}{x_n}\right)\) (if \(x_n\) is too big, \(\frac{a}{x_n}\) is too small, so average them):

Try it: Step Heron’s method for √2.

Three practical lessons follow from the limit concept:

Big-O notation, used informally in Chapter 14, is really a statement about limits at infinity. To compare a cost function \(f(n)\) with a reference function \(g(n)\), look at the ratio:

(The precise definition, \(f = O(g)\) if \(f(n) \le C g(n)\) for some constant \(C\) and all large \(n\), does not require the limit to exist, but when it does exist, this test settles the question.)

\(\frac{3n^2 + 10n}{n^2} = 3 + \frac{10}{n} \to 3\), so \(3n^2 + 10n = \Theta(n^2)\): the "divide by the highest power" technique, applied to algorithm costs. \(\frac{n\log n}{n^2} = \frac{\log n}{n} \to 0\), so \(n\log n = o(n^2)\): the growth hierarchy (logs lose to powers) is why \(O(n\log n)\) sorting beats \(O(n^2)\) sorting for large inputs, whatever the constant factors.