Rotating \((x, y)\) anticlockwise about the origin by \(\theta\):

In Chapter 27 this becomes multiplication by the rotation matrix \(\begin{bmatrix}\cos\theta & -\sin\theta \\ \sin\theta & \cos\theta\end{bmatrix}\). To rotate about some other point \((c_x, c_y)\), translate it to the origin, rotate, and translate back.

If \(\hat x\) approximates \(x\), the absolute error is \(|\hat x - x|\) and the relative error is \(|\hat x - x|/|x|\) (for \(x\ne0\)). Relative error \(10^{-k}\) means roughly \(k\) correct significant digits. Floating point controls relative error; that is its whole design.

The absolute value (or magnitude, or modulus) of \(x\), written \(|x|\), is its distance from 0:

The large brace introduces a piecewise definition: read each line as "this value, under this condition". So \(|5| = 5\), \(|-5| = -(-5) = 5\), \(|0| = 0\). The distance between \(a\) and \(b\) on the line is \(|a - b|\).

The absolute value \(|x|\) is the distance from \(x\) to 0 on the number line, so it is never negative: \(|5| = 5\), \(|-5| = 5\), \(|0| = 0\). Formally,

(The second line is not a typo: if \(x\) is negative, \(-x\) is positive.) More usefully, \(|x - a|\) is the distance between \(x\) and \(a\). Read \(|x - 3| < 2\) as "\(x\) is less than 2 away from 3".

It is the distance from \(x\) to 0 on the number line, so it is never negative. \(\lvert x - a \rvert\) is the distance between \(x\) and \(a\). The graph is a V with its corner at the origin.

An accumulated quantity is a total built up from a rate over time (or over any other variable). Distance accumulates from speed, water in a tank accumulates from flow rate, money accumulates from income per month. If the rate is constant, total = rate × duration. If the rate varies, total = area under the rate graph. The integral is the tool for computing that area.

Analytic geometry (also coordinate geometry or Cartesian geometry) is the study of geometry using a coordinate system, so that geometric facts are proved and computed with algebra. It contrasts with synthetic geometry — Euclid’s style in Chapter 9 — which reasons directly from axioms and diagrams without numbers.

\(\mathbb{R}\) is the set of real numbers. \(\mathbb{R}^2\) is the set of all ordered pairs \((x, y)\) of real numbers, i.e. the plane. \(\mathbb{R}^n\) is the set of all ordered lists of \(n\) real numbers, \((x_1, x_2, \ldots, x_n)\).

So \(f : \mathbb{R}^2 \to \mathbb{R}\) reads "\(f\) takes a pair of reals to a real".

When there are many inputs we bundle them into one object written in bold: \(\mathbf{x} = (x_1, x_2, \ldots, x_n)\), and write \(f(\mathbf{x})\) instead of \(f(x_1, \ldots, x_n)\). In handwriting, bold becomes an underline or an arrow: \(\vec{x}\). A model with a million weights is just a function \(L : \mathbb{R}^{1\,000\,000} \to \mathbb{R}\).

An angle is formed by two rays with a common endpoint, the vertex. The angle with vertex \(B\) and rays through \(A\) and \(C\) is written \(\angle ABC\) (vertex in the middle) or just \(\angle B\). Its measure is written \(m\angle ABC\) in US textbooks, though most people just write \(\angle ABC = 40^\circ\).

Angles are measured in degrees: a full turn is \(360^\circ\). (In Chapter 13 you’ll meet radians, where a full turn is \(2\pi\).)

\(\pm\) means "plus or minus" and \(\mp\) means "the opposite sign": take the top signs together or the bottom signs together. The cosine formula flips the sign.

An antiderivative of \(f\) is a function \(F\) with \(F'(x) = f(x)\). For example, \(F(x) = x^3/3\) is an antiderivative of \(f(x) = x^2\), because \(\frac{d}{dx}(x^3/3) = x^2\). So is \(x^3/3 + 7\): antiderivatives are only determined up to an added constant.

\(\min_\theta L(\theta)\) is the smallest value of \(L\). \(\operatorname{argmin}_\theta L(\theta)\) is the argument — the value of \(\theta\) — that achieves it. For \(L(\theta) = (\theta - 3)^2 + 1\), \(\min_\theta L = 1\) but \(\operatorname{argmin}_\theta L = 3\). Programmers know this distinction: min(xs) versus xs.index(min(xs)).

An arithmetic sequence (or arithmetic progression, AP) adds a fixed common difference \(d\) each step: \(a_n = a_{n-1} + d\). Its explicit form, with first term \(a_1\), is \(a_n = a_1 + (n-1)d.\) The word is pronounced a-rith-MET-ic (stress on the third syllable) when used as an adjective.

When operators of equal rank appear in a row, operator associativity decides grouping. Subtraction and division are left-associative: \(a - b - c = (a - b) - c\). Exponentiation is right-associative: \(a^{b^c} = a^{(b^c)}\), so \(2^{3^2} = 2^9 = 512\), not \(8^2 = 64\). (This is a different meaning of "associative" from the law in Section 1.4: here it is a parsing rule for notation.)

An asymptote is a line that a curve approaches more and more closely without reaching it (at least far out). \(y = 0\) is a horizontal asymptote of \(y = 2^x\): as \(x\) becomes very negative, \(2^x\) gets as close to 0 as you like but stays positive.

The average value of \(f\) on \({[a,b]}\) is

Geometrically, it is the height of the rectangle on \({[a,b]}\) with the same area as the region under \(f\).

An axiom or postulate is a statement accepted without proof (Euclid used "postulate" for geometric ones, "common notion" for general logic such as "things equal to the same thing are equal to each other").

A theorem is a statement proved from axioms and earlier theorems. A lemma is a small theorem proved mainly as a step towards a bigger one. A corollary is a result that follows almost immediately from a theorem.

A conjecture is a statement believed true but not yet proved.

The axis of symmetry is the vertical line through the vertex, \(x = -\dfrac{b}{2a}\). The parabola is its own mirror image in this line. Consequence: the two roots (when they exist) sit at equal distances either side of it, so the vertex is exactly halfway between the roots.

The base (or radix) \(b\) of a positional system is how many times larger each position is than the one to its right. A digit is one of the symbols \(0, 1, \ldots, b-1\). Positional (or place-value) notation means a digit’s value is the digit times \(b^{\text{position}}\), counting positions from 0 at the right. When the base is not obvious we write it as a subscript: \(101101_2 = 45_{10}\).

In \(a^n\), \(a\) is the base and \(n\) is the exponent (also called the index — plural indices — or power). The whole expression \(a^n\) is "a power of \(a\)" and is read "\(a\) to the \(n\)", "\(a\) to the power \(n\)", or "\(a\) raised to the \(n\)th". \(a^1 = a\). In code it is written a**n (Python), a^n (spreadsheets, math software), or pow(a, n) in C; in C and Java ^ means XOR.

\(H\) is a hypothesis (has the disease), \(E\) the evidence (tested positive).

For tests: sensitivity = \(P( + \mid \text{disease})\) (true-positive rate); specificity = \(P(- \mid \text{healthy})\); false-positive rate = \(1 -\) specificity.

\(E(h) = O(h^p)\) as \(h\to0\) means \(|E(h)|\le C h^p\) for some constant \(C\) and all small enough \(h\). A method with error \(O(h^p)\) has order \(p\): halving \(h\) divides the error by about \(2^p\). This is the same symbol as in algorithm analysis, but there \(n\to\infty\); here \(h\to0\).

Binary is base 2, with digits 0 and 1. A single binary digit is a bit; eight bits make a byte. Hexadecimal ("hex") is base 16, with digits \(0\text{–}9\) then A=10, B=11, C=12, D=13, E=14, F=15. Because \(16 = 2^4\), each hex digit corresponds to exactly four bits (a nibble). Octal is base 8 (three bits per digit). Programming languages mark bases with prefixes: 0b101101, 0x2D, 0o55 all mean 45.

A binomial is an expression with two terms, such as \(x + 3\) or \(2a - 5b\) (bi = two, nomen = name). A monomial has one term, a trinomial three. A product is the result of a multiplication; "the product of \(x + 3\) and \(x + 2\)" means \((x + 3)(x + 2)\).

A binomial is a polynomial with two terms, such as \(x + y\) or \(2x - 1\).

The summation sign \(\sum\) (capital Greek sigma, for "sum") packages a long sum. \(\sum_{k=0}^{n} a_k\) means "let the index \(k\) run through the integers from the lower limit \(0\) to the upper limit \(n\), and add up the terms \(a_k\)": \(a_0 + a_1 + \cdots + a_n\).

A graph is bipartite if its vertices can be split into two sets \(L\) and \(R\) (the parts) so that every edge joins a vertex of \(L\) to a vertex of \(R\). \(K_{a,b}\) is the complete bipartite graph with \(|L| = a\), \(|R| = b\) and all \(ab\) edges between them. A matching is a set of edges no two of which share an endpoint; a perfect matching covers every vertex.

A bit is the information in one fair yes/no answer. The surprise or self-information of an outcome with probability \(p\) is \(-\log_2 p\) bits. Entropy \(H\) is average surprise: the uncertainty of a random source before you see the outcome. With natural logarithms the unit is the nat (\(1 \text{ nat} \approx 1.443\) bits); ML papers often use nats without saying so.

A variable is free in an expression if the expression’s value depends on it, and bound if it is only a placeholder inside some construction. In \(\sum_{i=1}^{n} i^2\) (the sum of \(i^2\) as \(i\) runs from 1 to \(n\), see Chapter 14), \(n\) is free and \(i\) is bound: renaming \(i\) to \(k\) changes nothing. Programmers know this as scope: a loop counter is local to the loop, exactly like a bound variable.

For integers \(a, b\), not both zero, there exist integers \(x, y\) with

The \(x, y\) are Bézout coefficients. They are not unique. A linear combination of \(a\) and \(b\) is any expression \(ax + by\). The integer combinations of \(a\) and \(b\) are exactly the multiples of \(\gcd(a, b)\).

The Cartesian plane (or coordinate plane, xy-plane) is the plane together with two perpendicular number lines, the axes. They meet at the origin, \(O = (0, 0)\). The horizontal axis is the x-axis; the vertical axis is the y-axis.

Catastrophic cancellation is the loss of significant digits when two nearly equal floating-point numbers are subtracted. The subtraction itself is exact; the problem is that the inputs' small rounding errors become large relative to the small result. The cure is almost always algebraic: rewrite the expression so the subtraction never happens.

The ceiling \(\lceil x \rceil\) is the smallest integer \(\ge x\): \(\lceil 2.1 \rceil = 3\), \(\lceil 5 \rceil = 5\). The floor \(\lfloor x \rfloor\) is the largest integer \(\le x\): \(\lfloor 2.9 \rfloor = 2\). In code, math.ceil and math.floor; for positive integers, \(\lceil N/k \rceil\) is (N + k - 1) // k and \(\lfloor N/k \rfloor\) is N // k.

If \(X_1, \ldots, X_n\) are i.i.d. with mean \(\mu\) and standard deviation \(\sigma\), then for large \(n\),

The sum \(X_1 + \cdots + X_n\) is likewise approximately \(N(n\mu, n\sigma^2)\). The quantity \(\sigma/\sqrt n\) is the standard error of the mean.

By sides:

By angles: acute (all angles acute), right (one right angle), obtuse (one obtuse angle). In a right triangle the side opposite the right angle is the hypotenuse (always the longest side) and the other two are the legs.

Triangle \(ABC\) is written \(\triangle ABC\).

A coefficient is the numerical factor multiplying the variable part of a term. In \(7x^2 - 3x + 4\), \(7\) is the coefficient of \(x^2\), and \(-3\) is the coefficient of \(x\) (the sign is part of it). In \(x\) or \(-y\) the coefficients are \(1\) and \(-1\), usually not written. In \(ax^2\), with \(a\) a parameter, we also call \(a\) the coefficient.

A coefficient is the number multiplying a power of the variable. In \(7x^2 - 3x + 4\), \(7\) is the coefficient of \(x^2\) and \(-3\) is the coefficient of \(x\). A missing power has coefficient \(0\); a bare \(x^2\) has coefficient \(1\).

In \(ax^2 + bx + c\): \(a\) is the leading (or quadratic) coefficient, \(b\) the linear coefficient, and \(c\) the constant term. In \(7x^2 - 3x + 4\), \(a = 7\), \(b = -3\), \(c = 4\). Always read the signs with the coefficients: in \(x^2 - 5x + 6\), \(b = -5\), not \(5\). The condition \(a \neq 0\) matters: if \(a = 0\) the equation is linear, not quadratic.

A combination (or \(r\)-combination) of \(n\) distinct objects is an unordered selection of \(r\) of them, that is, a subset of size \(r\). The number of them is the binomial coefficient

read "\(n\) choose \(r\)". Other notations: \(C(n, r)\), \({}_nC_r\), \({}^nC_r\) (common in UK textbooks). By convention \(\binom{n}{r} = 0\) if \(r < 0\) or \(r > n\).

Combinatorics is the mathematics of counting arrangements and selections, and of deciding whether arrangements with given properties exist. To enumerate a set is to list or count its elements. An outcome is one specific result of a choice process (for example, one particular poker hand). For a finite set \(S\), the notation \(|S|\) means the number of elements of \(S\), called its cardinality.

In code: Python’s math.log(x) is the natural log, and math.log10, math.log2 and math.log(x, b) give the others. The same holds for C’s log, JavaScript’s Math.log, NumPy’s np.log and Excel’s LN. (Excel’s LOG is base 10.) If a bare "log" shows up, check which base is meant.

An operation is commutative if order of the inputs does not matter: \(a + b = b + a\), \(ab = ba\). It is associative if grouping does not matter: \((a + b) + c = a + (b + c)\), \((ab)c = a(bc)\). Multiplication distributes over addition: \(a(b + c) = ab + ac\). Mnemonic: commute = move around; associate = who you group with; distribute = hand out to each.

Complement rule: \(P(A^c) = 1 - P(A)\).

Addition rule: \(P(A \cup B) = P(A) + P(B) - P(A \cap B)\). If \(A\) and \(B\) are mutually exclusive, the last term is 0.

Monotonicity: if \(A \subseteq B\) then \(P(A) \le P(B)\). (A more specific event can’t be more probable.)

Complement \(A^c\): everything in \(\Omega\) not in \(A\). Union \(A \cup B\): "\(A\) or \(B\) or both" (mathematical "or" is always inclusive). Intersection \(A \cap B\): "both \(A\) and \(B\)". Events are mutually exclusive (or disjoint) if they share no outcomes, \(A \cap B = \varnothing\). The symbol \(\varnothing\) is the empty set.

Completing the square means rewriting \(x^2 + px\) as \(\left(x + \tfrac{p}{2}\right)^2 - \left(\tfrac{p}{2}\right)^2\), so that the variable appears only once, inside a square. It is the key step in solving any quadratic and in finding its vertex (Section 8.5).

The complex conjugate of \(z = a + bi\) is \(\bar z = a - bi\) (also written \(z^*\), especially in physics). Geometrically it is the reflection of \(z\) in the real axis. Key facts:

A complex number is an expression \(z = a + bi\) where \(a\) and \(b\) are real numbers. \(a\) is the real part, written \(\operatorname{Re}(z)\); \(b\) is the imaginary part, written \(\operatorname{Im}(z)\). Note that the imaginary part is the real number \(b\), not \(bi\). The set of all complex numbers is written \(\mathbb{C}\). Two complex numbers are equal exactly when their real parts are equal and their imaginary parts are equal. A number with \(b = 0\) is real; a number with \(a = 0\), \(b \ne 0\) is called purely imaginary.

The components of a vector are the numbers in its list, one per axis. The vector \((3, 4)\) has x-component 3 and y-component 4. In general the \(i\)-th component of \(\mathbf{v}\) is written \(v_i\), so \(\mathbf{v} = (v_1, v_2, \ldots, v_n)\). The set of all lists of \(n\) real numbers is written \(\mathbb{R}^n\) ("R-n"), so \((3, 4) \in \mathbb{R}^2\) means "(3, 4) is a vector in two-dimensional real space".

The composition of \(f\) and \(g\) is the function \(f \circ g\), read "\(f\) composed with \(g\)", "\(f\) after \(g\)", or "\(f\) of \(g\)":

Order: \(g\) is applied first, even though it is written second. Read \(f \circ g\) from right to left, like nested function calls f(g(x)) in code. The small circle \(\circ\) is not a multiplication dot.

\(f \circ g\) is defined at \(x\) only when \(x\) is in the domain of \(g\) and \(g(x)\) is in the domain of \(f\).

\(f\) is concave up (UK: convex) on an interval where its slope is increasing — the graph bends like a cup, \(\cup\). It is concave down (UK: concave) where its slope is decreasing — a cap, \(\cap\). Test: \(f'' > 0\) ⇒ concave up; \(f'' < 0\) ⇒ concave down. An inflection point is a point on the graph where the concavity changes.

The condition number \(\kappa\) of a problem measures how much relative changes in the input are amplified in the output:

For a differentiable function of one variable, \(\kappa(x) = \left|\dfrac{x\,f'(x)}{f(x)}\right|\). For a linear system \(A\mathbf x=\mathbf b\), \(\kappa(A)=\|A\|\,\|A^{-1}\|\) (Chapter 27). A problem is well-conditioned if \(\kappa\) is modest and ill-conditioned if it is large. For the matrix above, \(\kappa\approx 4\times10^{4}\).

For events \(A\) and \(B\) with \(P(B) > 0\), the conditional probability of \(A\) given \(B\) is

Read the vertical bar \(\mid\) as "given". \(B\) is the conditioning event (what you know); \(A\) is the event you’re asking about. \(P(A)\), without a condition, is sometimes called the unconditional or marginal probability.

Meaning: of all the probability sitting on \(B\), what fraction also lies in \(A\)? Geometrically: zoom into \(B\) and rescale it to have total probability 1.

A 95% confidence interval for a parameter is an interval computed from the data by a procedure that captures the true parameter in 95% of repeated samples. For a mean:

where \(z^* = 1.96\) for 95% (1.645 for 90%, 2.576 for 99%). The quantity \(z^* \cdot \text{SE}\) is the margin of error. The confidence level (95%) is the long-run success rate of the procedure.

For small samples, replace \(z^*\) by a slightly larger \(t^*\) from the t-distribution (Section 31.6.6) to account for estimating \(\sigma\) by \(s\).

Two triangles are congruent if they match in:

In a congruence statement \(\triangle ABC \cong \triangle DEF\) the order matters: \(A\) corresponds to \(D\), \(B\) to \(E\), \(C\) to \(F\). US textbooks then use CPCTC, "corresponding parts of congruent triangles are congruent", to conclude that any other matching pair of parts is equal.

Fix a positive integer \(m\), the modulus. Integers \(a\) and \(b\) are congruent modulo \(m\), written

if \(m \mid (a - b)\). Equivalently, \(a\) and \(b\) leave the same remainder when divided by \(m\). Read it as "\(a\) is congruent to \(b\) mod \(m\)". For example, \(14 \equiv 2 \pmod{12}\), \(-10 \equiv 2 \pmod{12}\), and \(100 \equiv 4 \pmod{12}\).

The three-bar symbol \(\equiv\) was Gauss’s choice (1801). It looks like "=" on purpose.

A conic section (conic) is a curve obtained by intersecting a plane with a double cone: circle, ellipse, parabola or hyperbola. Each (except the circle) can also be defined by a focus (a fixed point; plural foci) and a directrix (a fixed line). The vertex is the point where a parabola turns; the axis of symmetry is the line it is mirror-symmetric about.

An undirected graph is connected if there is a path between every pair of vertices. Otherwise it splits into connected components: maximal connected pieces. Being "joined by a path" is an equivalence relation (reflexive, symmetric, transitive; see Chapter 21), and the components are its equivalence classes.

A directed graph is strongly connected if there is a directed path from every vertex to every other, and weakly connected if it is connected when you ignore the arrows.

A logical connective (or operator) builds a new proposition from one or more existing ones. The five standard connectives are:

A proposition built from others using connectives is compound; one that cannot be broken down further is atomic. Some books write \(\sim p\) or \(\overline{p}\) for negation, \(\Rightarrow\) for \(\to\), and \(\Leftrightarrow\) or "iff" for \(\leftrightarrow\).

US textbooks use all four words. A common summary is "consistent independent / consistent dependent / inconsistent".

A constant is a quantity with a fixed value throughout a discussion. Literal numbers such as \(10\) or \(-\tfrac{3}{4}\) are constants, and so are named numbers such as \(\pi \approx 3.14159\) and \(e \approx 2.71828\).

The constant term is a term with no variable in it: the \(4\) in \(7x^2 - 3x + 4\). If there is no such term, the constant term is \(0\).

The constant term is the term with no variable, i.e. the coefficient of \(x^0\). In \(7x^2 - 3x + 4\) it is \(4\). It is the value of the polynomial at \(x = 0\).

A function \(f\) is continuous at \(a\) if all three conditions hold:

\(f\) is continuous on an interval if it is continuous at every point of it (using one-sided limits at closed endpoints), and simply continuous if it is continuous at every point of its domain. A point where continuity fails is a discontinuity.

For the implication \(P \Rightarrow Q\) ("if \(P\) then \(Q\)"):

The symbol \(\neg\) means "not". See Chapter 21 for the truth tables.

An infinite series \(\sum_{k=1}^{\infty} a_k\) converges to \(S\) if its partial sums \(S_n = \sum_{k=1}^{n} a_k\) get arbitrarily close to \(S\) and stay there as \(n \to \infty\) (read "as \(n\) tends to infinity"). We then write \(\sum_{k=1}^{\infty} a_k = S\) and call \(S\) the sum of the series. If the partial sums do not approach any finite number, the series diverges. The symbol \(\infty\) (infinity) here is not a number; \(\sum^{\infty}\) is shorthand for "the limit of the partial sums".

A function \(f\) is convex if the straight line (chord) between any two points on its graph lies on or above the graph:

For a twice-differentiable function of one variable this is the same as \(f''(x) \ge 0\) everywhere: the slope never decreases. For several variables the condition is that the Hessian matrix (Chapter 20) is positive semidefinite. Examples: \(x^2\), \(e^x\), \(\lvert x \rvert\). Non-examples: \(\sin x\), \(x^3\), \(-x^2\).

A point’s coordinates are the ordered pair \((x, y)\). The first number, the x-coordinate (old name: abscissa), is the signed horizontal distance from the y-axis. The second, the y-coordinate (old name: ordinate), is the signed vertical distance from the x-axis. "Ordered" matters: \((2, 5)\) and \((5, 2)\) are different points.

A linear program (LP) maximises or minimises a linear objective function (like \(30x + 20y\)) subject to linear inequality constraints. The corner-point principle: if an LP has an optimal solution and its feasible region has at least one vertex, then some vertex is optimal. (If the objective’s level lines are parallel to an edge, every point on that edge ties.)

The norm (length) of a vector is \(\lVert\mathbf{v}\rVert = \sqrt{\mathbf{v}\cdot\mathbf{v}} = \sqrt{\sum_j v_j^2}\). A unit vector has norm 1; normalising means dividing by the norm. Cosine similarity is the dot product of the two normalised vectors. For unit vectors, dot product and cosine similarity are the same number — which is why systems often normalise everything once at indexing time and then just take dot products.

The covariance of \(X\) and \(Y\), with means \(\mu_X\) and \(\mu_Y\), is

Positive when \(X\) and \(Y\) tend to be above (or below) their means together; negative when one tends to be high as the other is low; zero (uncorrelated) when there’s no linear tendency. \(\operatorname{Cov}(X, X) = \operatorname{Var}(X)\).

Variance of a sum, in general: \(\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\operatorname{Cov}(X, Y)\).

A critical point (or stationary point) of \(f\) is a point where \(\nabla f = \mathbf{0}\), i.e. every first partial derivative is zero.

A saddle point is a critical point that is neither a local maximum nor a local minimum: \(f\) rises in some directions and falls in others.

The Hessian is the matrix of second partial derivatives:

It is symmetric (\(f_{xy} = f_{yx}\)) and plays the role of the second derivative: it measures curvature.

A critical point of \(f\) is a point \(c\) in its domain where \(f'(c) = 0\) or \(f'(c)\) does not exist. Where \(f'(c) = 0\) specifically, UK texts say stationary point: the tangent is horizontal. Every local maximum or minimum of a function at an interior point is a critical point — but not every critical point is a maximum or minimum.

The cross product (vector product) \(\mathbf{a}\times\mathbf{b}\) of two 3D vectors is a 3D vector perpendicular to both, of length \(\|\mathbf{a}\|\,\|\mathbf{b}\|\sin\theta\), oriented by the right-hand rule. Contrast the dot product, which works in any dimension and returns a scalar involving \(\cos\theta\). In unit-vector form: \(\mathbf{i}\times\mathbf{j} = \mathbf{k}\), \(\mathbf{j}\times\mathbf{k} = \mathbf{i}\), \(\mathbf{k}\times\mathbf{i} = \mathbf{j}\).

Cross-entropy \(H(p, q)\) is the average surprise when outcomes come from \(p\) but you score them with \(q\). KL divergence \(D_{\mathrm{KL}}(p\,\Vert\,q)\) is the excess, \(H(p,q) - H(p)\). It measures how different \(q\) is from \(p\), but it is not a distance: in general \(D_{\mathrm{KL}}(p\,\Vert\,q) \ne D_{\mathrm{KL}}(q\,\Vert\,p)\). The double bar \(\Vert\) separates the two arguments.

For a true label \(y \in \{0, 1\}\) and predicted probability \(p\), the binary cross-entropy is \(-[y\ln p + (1-y)\ln(1-p)]\). If \(y = 1\) it is \(-\ln p\): zero when \(p = 1\), growing without bound as \(p \to 0\). Confident wrong answers are punished severely. It is also called log loss or negative log-likelihood.

The cumulative distribution function of \(X\) is

It rises from 0 (far left) to 1 (far right) and never decreases. Then \(P(a < X \le b) = F(b) - F(a)\). For a discrete variable \(F\) is a staircase, jumping by \(p(x)\) at each value; for a continuous one it’s a smooth curve whose slope is the density (\(f = F'\), Chapter 17).

The median is the value where \(F(x) = 0.5\); more generally the \(q\)-th quantile (or \(100q\)-th percentile) is where \(F(x) = q\). "p99 latency" is the 0.99 quantile.

A directed acyclic graph (DAG) is a directed graph with no directed cycle. A topological order (or topological sort) of a directed graph is a listing of all its vertices such that for every edge \(u \to v\), \(u\) comes before \(v\). A vertex with in-degree 0 is a source; with out-degree 0, a sink.

The digits after the decimal point are the decimal places: the first is tenths, the second hundredths, the third thousandths. A terminating decimal has finitely many non-zero decimal places (\(0.125\)). A repeating (or recurring) decimal ends in a block of digits that repeats forever. The repeating block is marked with a bar (US) or dots (UK): \(0.1\overline{6} = 0.1666\ldots\), \(0.\overline{142857} = 0.142857142857\ldots\). The length of the repeating block is the period.

SI (decimal): kilo (k) \(= 10^3\), mega (M) \(= 10^6\), giga (G) \(= 10^9\), tera (T) \(= 10^{12}\), peta (P) \(= 10^{15}\). IEC (binary): kibi (Ki) \(= 2^{10}\), mebi (Mi) \(= 2^{20}\), gibi (Gi) \(= 2^{30}\), tebi (Ti) \(= 2^{40}\), pebi (Pi) \(= 2^{50}\). So 1 KB \(= 1000\) bytes and 1 KiB \(= 1024\) bytes. Lower-case b means bits, upper-case B bytes: 1 Gbit/s is 125 MB/s.

The definite integral \(\int_a^b f(x)\,dx\) is a number: the limit of Riemann sums of \(f\) over \({[a,b]}\). Geometrically it is the signed area between the graph of \(f\) and the \(x\)-axis from \(a\) to \(b\). A function for which this limit exists is called integrable on \({[a,b]}\).

The degree of a polynomial is the highest power of the variable that appears with a non-zero coefficient. \(4x^3 - 100x^2 + 600x\) has degree 3. A non-zero constant such as \(5\) has degree 0. Names: degree 1 linear, 2 quadratic, 3 cubic, 4 quartic, 5 quintic.

The degree of a vertex \(v\), written \(\deg(v)\), is the number of edges incident to it (a loop counts twice). In a directed graph a vertex has an in-degree (arrows arriving) and an out-degree (arrows leaving). A vertex of degree 0 is isolated; a vertex of degree 1 is a leaf (or pendant vertex). A graph in which every vertex has the same degree \(k\) is \(k\)-regular.

The degree of a term is the total number of variable factors in it: \(7x^2\) has degree 2, \(-3x\) has degree 1, \(4\) has degree 0, and \(5x^2y\) has degree 3. The degree of a polynomial (a sum of such terms with whole-number powers) is the largest degree among its terms. Degree 1 means linear, degree 2 quadratic, degree 3 cubic. You will use these words constantly from Chapter 5 on.

\(\Delta\) (capital Greek delta) means "change in". \(\Delta x = x_2 - x_1\) is the change in \(x\) from a first point \((x_1, y_1)\) to a second \((x_2, y_2)\); the subscripts 1 and 2 simply label the points. \(\Delta y = y_2 - y_1\). Slope is "rise over run":

The Greek capital letter delta, \(\Delta\), is read "change in". So \(\Delta x = x_2 - x_1\) is the change in \(x\) going from the first point to the second. The subscripts \(x_1, x_2\) are just labels ("x of point 1", "x of point 2"), not powers or multiplications.

The difference quotient is \(\dfrac{f(x+h) - f(x)}{h}\), the slope of a secant.

The derivative \(f'(x)\) is the limit of the difference quotient as \(h \to 0\): the slope of the tangent, or the instantaneous rate of change of \(f\).

A function is differentiable at \(x\) if that limit exists (as a finite number). It is differentiable (on an interval) if it is differentiable at every point there.

Differentiation is the process of finding a derivative. The word comes from Leibniz’s "differences", the tiny changes \(dx\) and \(dy\).

The determinant of a square matrix, written \(\det A\) or \(|A|\), is the signed factor by which the transformation scales area (2D) or volume (3D and up).

A differential equation is an equation involving an unknown function and its derivatives, such as \(\frac{dy}{dt} = ky\). A solution is a function that makes the equation true. An initial condition such as \(y(0) = y_0\) picks out one solution from the family. An ordinary differential equation (ODE) has a single independent variable (usually time).

A differential equation is an equation involving an unknown function and its derivatives. An ordinary differential equation (ODE) has one independent variable (usually time); a partial differential equation (PDE) has several, with partial derivatives (heat flow, waves, fluid dynamics). An initial-value problem specifies the starting state, such as \(y(0) = y_0\). The order is the highest derivative that appears.

A differential equation relates an unknown function to its derivatives. An initial value problem (IVP) gives \(\mathbf y'(t) = F(t,\mathbf y)\) together with a starting value \(\mathbf y(t_0)=\mathbf y_0\). Any higher-order equation becomes a first-order system by adding variables: for the pendulum take the state \(\mathbf y=(\theta,\omega)\) with angular velocity \(\omega=\theta'\), so \(\theta'=\omega,\ \omega'=-\tfrac gL\sin\theta\). The step size \(h\) is the time increment between computed states.

Dimensional analysis (also called the factor-label method or unit analysis) converts units by multiplying by conversion factors — fractions equal to 1, such as \(\tfrac{60 \text{ s}}{1 \text{ min}}\) — chosen so that unwanted units cancel. A dimension is the kind of quantity (length, time, mass); a unit is a particular scale for it (metre, second, kilogram). You can only add quantities with the same dimension, and both sides of a correct equation have the same dimension.

The directional derivative \(D_{\mathbf{u}} f(\mathbf{a})\) is the rate of change of \(f\) at \(\mathbf{a}\) per unit distance moved in the direction of the unit vector \(\mathbf{u}\). It equals \(\nabla f(\mathbf{a}) \cdot \mathbf{u}\).

The discriminant of \(ax^2 + bx + c\) is the number

(\(\Delta\) is capital Greek delta; some books write \(D\).) It "discriminates" between the cases: \(\Delta > 0\), two distinct real roots; \(\Delta = 0\), one repeated real root; \(\Delta < 0\), no real roots. Section 8.7 explains why.

The distance formula: the distance between \((x_1, y_1)\) and \((x_2, y_2)\) is \(d = \sqrt{(x_2 - x_1)^2 + (y_2 - y_1)^2}\). It is the Pythagorean theorem written in coordinates. This straight-line distance is called the Euclidean distance.

For points \(p = (x_1, y_1)\) and \(q = (x_2, y_2)\):

A metric is any distance function that is never negative, is zero only from a point to itself, is symmetric, and obeys the triangle inequality \(d(p, r) \le d(p, q) + d(q, r)\) (a detour never helps). All three above are metrics.

The distributive law says multiplication "distributes over" addition: \(a(b + c) = ab + ac\). To expand (or "multiply out") an expression is to use this law to remove brackets, producing a sum of terms. The opposite move, rewriting a sum as a product, is factoring (Section 4.9).

In \(p(x) \div d(x)\), \(p\) is the dividend and \(d\) the divisor. Division produces a quotient \(q(x)\) and remainder \(r(x)\) such that

If \(r = 0\), \(d\) divides \(p\) exactly, and \(d\) is a factor of \(p\).

In \(23 = 4 \cdot 5 + 3\) the number being divided (23) is the dividend. The number you divide by (5) is the divisor. The number of whole times it fits (4) is the quotient, and what is left (3) is the remainder. "Quotient" comes from the Latin quotiens, "how many times".

\(b\) divides \(a\), written \(b \mid a\) (a vertical bar, read "b divides a"), if the remainder is 0, i.e. \(a = qb\) for some integer \(q\). Then \(b\) is a factor (or divisor) of \(a\), and \(a\) is a multiple of \(b\). So \(3 \mid 12\) and \(5 \nmid 12\). Do not confuse \(b \mid a\) (a true/false statement) with \(b / a\) (a number).

For integers \(a\) and \(b\), we say \(a\) divides \(b\), written \(a \mid b\), if \(b = ka\) for some integer \(k\). Equivalently, dividing \(b\) by \(a\) leaves remainder 0. Then \(a\) is a divisor (or factor) of \(b\), and \(b\) is a multiple of \(a\).

So \(3 \mid 12\) because \(12 = 4 \cdot 3\), and \(5 \nmid 12\) ("5 does not divide 12"). The vertical bar is a statement (true or false), not an operation: \(3 \mid 12\) is not the number 4. Don’t confuse it with the fraction \(3/12\).

For any integer \(a\) and any positive integer \(b\), there are unique integers \(q\) (the quotient) and \(r\) (the remainder) with

We write \(r = a \bmod b\), read "\(a\) mod \(b\)".

For integers \(a\) and \(b > 0\) there are unique integers \(q\) (the quotient) and \(r\) (the remainder) with

Example: \(47 = 6 \times 7 + 5\), so dividing 47 by 7 gives quotient 6, remainder 5. In code, q, r = divmod(a, b), or a // b and a % b.

The domain of an expression is the set of values of the variable for which it is defined. For a rational expression, that is every real number except the roots of the denominator. The domain of \(\dfrac{x + 1}{x^2 - 4}\) is all real \(x\) except \(x = \pm 2\), written \(x \neq \pm 2\).

For a function \(f : A \to B\):

Example: \(f : \mathbb{R} \to \mathbb{R},\ f(x) = x^2\). Domain \(\mathbb{R}\), codomain \(\mathbb{R}\), range \(\lbrack 0, \infty)\), since squares are never negative.

The dot product of two vectors multiplies matching components and adds the results:

It also has a geometric meaning, which Chapter 26 derives from the law of cosines:

where \(\theta\) is the angle between the arrows. So the dot product is largest when the vectors point the same way (\(\cos 0 = 1\)), zero when they are perpendicular (\(\cos 90^\circ = 0\)), and most negative when they point opposite ways.

The dot product (scalar product, inner product) of two vectors of the same dimension is

It is a scalar. Other notations: \(\langle \mathbf{a}, \mathbf{b} \rangle\) (mathematics, physics), \(\mathbf{a}^\mathsf{T}\mathbf{b}\) (linear algebra, "a-transpose b"; see Chapter 27), and a @ b or np.dot(a, b) in NumPy.

The double integral \(\iint_R f\,dA\) is the signed volume between the surface \(z = f(x, y)\) and the region \(R\) of the plane. \(dA\) stands for a tiny element of area, \(dx\,dy\) in Cartesian coordinates and \(r\,dr\,d\theta\) in polar coordinates (the \(r\) is the Jacobian determinant from earlier).

An iterated integral evaluates it as two ordinary integrals, inner first. While doing the inner integral, the outer variable is held constant, exactly as with partial derivatives.

\(\lbrack a, b] \times [c, d]\) (read "\(a\) to \(b\) cross \(c\) to \(d\)") is the rectangle of points with \(a \le x \le b\) and \(c \le y \le d\).

The doubling time of an exponentially growing quantity is the time it takes to double. The half-life of a decaying quantity is the time it takes to halve. For an exponential process these are constant: going from 100 to 200 takes exactly as long as going from 1,000 to 2,000.

With a half-life \(T\), the amount after time \(t\) is

Here \(N_0\) ("N nought") is the starting amount and \(t/T\) counts how many half-lives have gone by.

The variable of integration is a dummy variable: its name does not matter. \(\int_0^2 x^2\,dx\), \(\int_0^2 t^2\,dt\) and \(\int_0^2 u^2\,du\) are all the same number, \(8/3\). It is exactly like a loop variable or a bound variable in a lambda: sum(i*i for i in r) and sum(k*k for k in r) are the same.

The eccentricity \(e\) of a conic is the ratio (distance to focus) / (distance to directrix), which is the same for every point on it. \(e = 0\): circle. \(0 < e < 1\): ellipse (for the standard ellipse, \(e = c/a\)). \(e = 1\): parabola. \(e > 1\): hyperbola. One number classifies them all.

A nonzero vector \(\mathbf{v}\) is an eigenvector of a square matrix \(A\) if

for some scalar \(\lambda\) (Greek "lambda"), called the eigenvalue belonging to \(\mathbf{v}\). In words: \(A\) maps \(\mathbf{v}\) to a multiple of itself. \(\lambda > 1\) stretches, \(0 < \lambda < 1\) shrinks, \(\lambda < 0\) flips (and scales by \(|\lambda|\)), \(\lambda = 0\) crushes \(\mathbf{v}\) to zero. Any nonzero multiple of an eigenvector is also an eigenvector with the same eigenvalue, so really an eigenvector names a direction (a line through the origin). The zero vector is never counted as an eigenvector, since \(A\mathbf{0} = \lambda\mathbf{0}\) for every \(\lambda\) and tells you nothing.

An embedding is a map from discrete objects (words, sentences, images, users, products) to vectors in \(\mathbb{R}^k\), chosen so that geometric relationships reflect real ones: similar objects get nearby vectors. Sparse embeddings (TF-IDF) have one dimension per vocabulary word and mostly zeros; dense embeddings have a few hundred learned dimensions, all typically non-zero.

An equation is a statement that two expressions are equal: left-hand side (LHS) \(=\) right-hand side (RHS). An equation may be true or false depending on the values of its variables. Example: \(2(x + 3) = 14\) is true when \(x = 4\) and false otherwise.

The standard form (or centre–radius form) of a circle with centre \((h, k)\) and radius \(r\) is \((x - h)^2 + (y - k)^2 = r^2\). The circle centred at the origin is \(x^2 + y^2 = r^2\); with \(r = 1\) it is the unit circle, the star of Chapter 13. Expanding standard form gives the general form \(x^2 + y^2 + Dx + Ey + F = 0\).

An equivalence relation is a relation that is reflexive, symmetric and transitive. It behaves like "is the same as, in some respect". Common notation is \(a \sim b\).

The equivalence class of \(a\) is \(\lbrack a] = \{\, x \in A \mid x \sim a \,\}\), everything equivalent to \(a\). The classes form a partition of \(A\): they are nonempty, pairwise disjoint, and their union is all of \(A\). Every element lives in exactly one class.

Two equations are equivalent if they have exactly the same solution set. \(2x + 3 = 11\), \(2x = 8\), and \(x = 4\) are equivalent: each is true for \(x = 4\) and for nothing else. Solving an equation means replacing it by a chain of equivalent equations, ending with one whose solution is obvious.

Two equations are equivalent if they have exactly the same solution set. \(2x + 1 = 7\), \(2x = 6\) and \(x = 3\) are equivalent. Solving an equation means producing a chain of equivalent equations that ends in something so simple, like \(x = 3\), that the solution set can be read off.

Two systems are equivalent if they have exactly the same solution set. Elimination and substitution are chains of equivalent systems, each simpler than the one before, ending in a system so simple you can read the answer off (like \(x = 3,\ y = 2\)).

If a prime \(p\) divides a product \(ab\), then \(p \mid a\) or \(p \mid b\).

An Euler path (or Euler trail) is a trail that uses every edge of the graph exactly once. An Euler circuit is an Euler path that starts and ends at the same vertex. A graph with an Euler circuit is called Eulerian.

To evaluate an expression is to compute its numerical value for given values of its variables. To substitute is to replace a variable by something else, a number or another expression. Evaluation is substitution followed by arithmetic.

Most functions are neither. The names come from powers: \(x^n\) is even exactly when \(n\) is even, and odd when \(n\) is odd.

An integer is even if \(2 \mid n\) and odd otherwise. Whether a number is even or odd is its parity. Every odd number can be written as \(2k + 1\) for some integer \(k\).

\(p \oplus q\) ("\(p\) xor \(q\)") is true when exactly one of \(p, q\) is true. It equals \((p \vee q) \wedge \neg(p \wedge q)\), and it is the negation of the biconditional: \(p \oplus q \equiv \neg(p \leftrightarrow q)\).

The expected value (or expectation, or mean) of a discrete random variable \(X\) is

the sum of each possible value times its probability: a weighted average of the values, weighted by probability. It is often written \(\mu\) (Greek mu) or \(\mu_X\).

Interpretation: the long-run average of \(X\) over many independent repetitions.

An explicit formula (or closed form, or general term) gives \(a_n\) directly in terms of \(n\), for example \(a_n = n^2\).

A recursive definition (or recurrence relation) gives one or more initial conditions and a rule expressing \(a_n\) in terms of earlier terms, for example \(a_1 = 1,\ a_n = a_{n-1} + 2n - 1\). Without the initial condition a recurrence does not pin down a single sequence.

An exponential function has the variable in the exponent:

Compare it with a power function such as \(x^2\), where the variable is in the base. \(x^2\) and \(2^x\) are very different functions.

An exponential expression has the variable in the exponent: \(2^n\), \(1.05^t\), \(10^x\). A polynomial (or power) expression has the variable in the base: \(n^2\), \(n^3\), \(x^{10}\). For any base \(b > 1\) and any power \(k\), \(b^n\) eventually exceeds \(n^k\) and the ratio grows without bound. Exponential growth with base \(b > 1\) has a doubling time; exponential decay (base between 0 and 1) has a half-life.

An expression is a meaningful combination of numbers, variables, and operations, with no equals sign: \(7x^2 - 3x + 4\), \(\frac{a+b}{2}\), \(2\pi r\). An expression has a value once its variables are given values; it does not assert anything.

An extraneous solution is a value produced by a valid-looking algebraic process that does not actually satisfy the original equation. Steps such as squaring both sides or combining logarithms can introduce them, because they can enlarge the set of allowed inputs. Always substitute back into the original equation.

The factors of a term (or of a number) are the pieces that are multiplied together to make it. The term \(6x^2y\) has factors \(6\), \(x\), \(x\), \(y\) (and combinations of them such as \(2x\) or \(3xy\)). Terms are added; factors are multiplied. Keeping these two words apart avoids many errors.

A factor of an expression is something that divides it exactly, so that the expression equals the factor times something else. \((x - 3)\) is a factor of \(x^2 - 9\) because \(x^2 - 9 = (x - 3)(x + 3)\). To factor (British: to factorise) means to rewrite a sum as a product.

To factor (US) or factorise (UK) an expression is to rewrite it as a product. The greatest common factor (GCF), also called the highest common factor (HCF), of several terms is the largest expression that divides all of them: the greatest common divisor of the coefficients times each shared variable raised to its lowest power. The GCF of \(12x^3y\), \(-18x^2y^2\), and \(6x^2y\) is \(6x^2y\).

\((x - r)\) is a factor of \(p(x)\) if and only if \(p(r) = 0\). ("If and only if" means both directions hold: a factor gives a zero, and a zero gives a factor.)

A quadratic \(ax^2 + bx + c\) with integer coefficients factors into linear factors with integer (or rational) coefficients if and only if its discriminant is a perfect square. Reason: the roots \((-b \pm \sqrt\Delta)/(2a)\) are rational exactly when \(\sqrt\Delta\) is.

For a positive integer \(n\), n factorial, written \(n!\), is the product of all integers from \(1\) to \(n\): \(n! = n \times (n-1) \times \cdots \times 2 \times 1\). So \(3! = 6\), \(5! = 120\), \(10! = 3\,628\,800\). By convention \(0! = 1\).

\(n!\) counts the ways to arrange \(n\) distinct objects in a row. The useful identity \(n! = n \times (n-1)!\) is the recursive definition you would write in code.

If \(p\) is prime and \(p \nmid a\), then

Equivalently, \(a^p \equiv a \pmod p\) for every integer \(a\). ("Little" distinguishes it from Fermat’s Last Theorem.)

A floating-point number is stored as a sign, a significand (or mantissa) of fixed length, and an exponent: in binary, \(\pm 1.b_1 b_2 \ldots b_{52} \times 2^{e}\) for an IEEE 754 double. It is binary scientific notation (Chapter 3). Only numbers that are integers divided by a power of 2 can be represented exactly, and only if they fit in the available bits. The gap between neighbouring floats grows with their size; relative precision is roughly 16 significant decimal digits.

\(\lfloor 3.7 \rfloor = 3\), \(\lceil 3.2 \rceil = 4\), \(\lfloor 5 \rfloor = \lceil 5 \rceil = 5\). Careful with negatives: \(\lfloor -3.5 \rfloor = -4\) (down means towards \(-\infty\), not towards 0), and \(\lceil -3.5 \rceil = -3\).

These are step functions: their graphs are flat stairs with a jump at each integer.

The floor \(\lfloor x \rfloor\) is the greatest integer \(\le x\): \(\lfloor 3.5 \rfloor = 3\) and \(\lfloor -3.5 \rfloor = -4\). Truncation drops the fractional part, rounding toward 0: \(-3.5 \to -3\). They agree for nonnegative numbers and differ for negative non-integers.

FOIL (First, Outer, Inner, Last) is a US mnemonic for multiplying two binomials: \((a + b)(c + d) = ac + ad + bc + bd\). It is not a separate rule, just the distributive law (the four cells of a 2-by-2 area model) with a fixed order. It does not extend to longer polynomials; the grid does.

An intercept is where a graph crosses an axis: the x-intercept has \(y = 0\), the y-intercept has \(x = 0\).

A formula is an equation that expresses a rule or relationship between quantities, usually with one quantity isolated on the left, its subject: \(A = \ell w\), \(F = \tfrac{9}{5}C + 32\), \(v = u + at\). A formula is a recipe: give values to the quantities on the right and it tells you the subject.

A function \(f\) from a set \(A\) to a set \(B\) is a rule that assigns to each element of \(A\) exactly one element of \(B\).

We write \(f : A \to B\), read "\(f\) maps \(A\) to \(B\)" or "\(f\) from \(A\) to \(B\)". The arrow is about the whole function; it says where inputs come from and where outputs live.

A function \(f : A \to B\) is a relation from \(A\) to \(B\) in which each element of the domain \(A\) is paired with exactly one element of the codomain \(B\). "Exactly one" has two halves: at least one (defined everywhere on \(A\)) and at most one (no input has two outputs).

A function of several variables (or multivariable function) takes more than one input number and returns one output number. For example, \(f(x, y) = x^2 + 3y^2\) takes two inputs, and \(T(x, y, z)\) could give the temperature at a point in a room.

A scalar-valued function returns a single number. (A function returning several numbers, such as a position in space, is vector-valued. We’ll meet one when we discuss the Jacobian.)

Fundamental theorem of algebra: every polynomial of degree \(n \ge 1\) with complex coefficients has at least one complex root. Consequently it factors completely into \(n\) linear factors,

so it has exactly \(n\) roots in \(\mathbb{C}\), counted with multiplicity (a repeated factor like \((z - 2)^2\) counts twice). A number system in which every polynomial has a root is called algebraically closed.

Every integer \(n > 1\) can be written as a product of primes, and this factorization is unique apart from the order of the factors.

If \(f\) is continuous on \({[a, b]}\) and \(A(x) = \int_a^x f(t)\,dt\), then \(A\) is differentiable and

In words: the rate of change of accumulated area is the current height. Differentiation undoes integration.

If \(f\) is continuous on \({[a,b]}\) and \(F\) is any antiderivative of \(f\), then

The bracket notation \(\big[F(x)\big]_a^b\) (also written \(F(x)\big|_a^b\)) means "evaluate at the top limit, minus evaluate at the bottom limit".

The greatest common divisor \(\gcd(a, b)\) of two integers, not both zero, is the largest positive integer dividing both. US schools call it the greatest common factor (GCF), and UK schools the highest common factor (HCF). If \(\gcd(a, b) = 1\), then \(a\) and \(b\) are coprime (or relatively prime). They need not be prime themselves: 8 and 15 are coprime. By convention \(\gcd(a, 0) = |a|\).

A geometric sequence (or geometric progression, GP) multiplies by a fixed common ratio \(r\) each step: \(a_n = r\,a_{n-1}\). Its explicit form, with first term \(a_1\), is \(a_n = a_1 r^{\,n-1}.\)

The gradient of \(f(x_1, \ldots, x_n)\) is the vector of its partial derivatives:

The symbol \(\nabla\) is called nabla (or "del"). Read \(\nabla f(\mathbf{x})\) as "the gradient of \(f\) at the point \(\mathbf{x}\)". It is a vector with one component for each input, and it can be different at every point. In some texts it is also written \(\operatorname{grad} f\) or \(\frac{\partial f}{\partial \mathbf{x}}\).

A (proper) colouring of a graph assigns a colour to each vertex so that adjacent vertices get different colours. The chromatic number \(\chi(G)\) (Greek letter chi) is the fewest colours that suffice. \(\chi(G) \le 2\) exactly when \(G\) is bipartite; \(\chi(K_n) = n\); a graph containing a triangle needs at least 3.

The graph of \(f\) is the set of all points \((x, f(x))\) for \(x\) in the domain. In set-builder form: \(\{(x, y) : x \in \text{domain},\ y = f(x)\}\).

So "the point \((a, b)\) is on the graph" means exactly "\(f(a) = b\)". Reading a graph is evaluating a function with your eyes: go along to \(x = a\), then up or down to the curve, and read off the height.

A graph \(G = (V, E)\) consists of a set \(V\) of vertices (also called nodes) and a set \(E\) of edges, each edge joining two vertices. An edge between \(u\) and \(v\) is written \(\{u, v\}\) or simply \(uv\). \(u\) and \(v\) are its endpoints; they are adjacent (neighbours), and the edge is incident to each of them. The number of vertices \(|V|\) is often written \(n\) and the number of edges \(|E|\) is written \(m\).

The singular of vertices is vertex. "Graph" here has nothing to do with the graph of a function (Chapter 11); the word was coined by J. J. Sylvester in 1878 from "chemicograph", a diagram of molecular bonds.

The greatest common divisor \(\gcd(a, b)\) (also called greatest common factor, GCF, or highest common factor, HCF) is the largest positive integer dividing both \(a\) and \(b\). The least common multiple \(\operatorname{lcm}(a, b)\) is the smallest positive integer that both divide. Example: \(\gcd(12, 18) = 6\), \(\operatorname{lcm}(12, 18) = 36\). For positive integers, \(\gcd(a, b) \cdot \operatorname{lcm}(a, b) = ab\). Two numbers with gcd 1 are coprime (or relatively prime).

The greatest common factor of several terms is the largest expression dividing all of them. For \(6x^3 - 9x^2\) it is \(3x^2\), so \(6x^3 - 9x^2 = 3x^2(2x - 3)\). Take the GCF of the coefficients and the lowest power of each variable that appears in every term.

If something grows by a percentage \(r\) per step (written as a decimal, so 5% means \(r = 0.05\)), its growth factor is \(b = 1 + r\). If it decays (shrinks) by \(r\) per step, then \(b = 1 - r\).

\(r\) is the growth rate (or decay rate). \(b\) is the growth factor. Don’t mix the two up: a 5% rate is a factor of 1.05.

A Hamiltonian path visits every vertex exactly once. A Hamiltonian cycle is a Hamiltonian path that returns to its start along an edge. The weighted version, "find the cheapest Hamiltonian cycle in a complete weighted graph", is the travelling salesman problem (TSP).

The second derivative is the derivative of the derivative: \(f''(x)\), or \(\dfrac{d^2y}{dx^2}\) ("dee two y by dee x squared"), or \(\ddot{s}\) in mechanics. The Leibniz form comes from applying the operator twice: \(\frac{d}{dx}\left(\frac{dy}{dx}\right)\).

The \(n\)th derivative is written \(f^{(n)}(x)\) (parentheses, so it is not mistaken for a power) or \(\dfrac{d^n y}{dx^n}\).

In mechanics: position \(s\), velocity \(v = \dot s = \frac{ds}{dt}\), acceleration \(a = \dot v = \ddot s = \frac{d^2s}{dt^2}\). The third derivative is called jerk.

Homogeneous coordinates represent a 2D point \((x, y)\) as \((x, y, 1)\) and a 3D point \((x, y, z)\) as \((x, y, z, 1)\), so that translations, rotations, scalings and (in 3D graphics) perspective projections are all matrix multiplications: \(3\times 3\) matrices for 2D graphics, \(4\times 4\) for 3D. Directions use a final coordinate of 0. More generally \((X, Y, W)\) with \(W \neq 0\) represents the point \((X/W, Y/W)\); dividing by \(W\) is what makes perspective work.

In \(p \to q\), \(p\) is the hypothesis (or antecedent, or premise) and \(q\) is the conclusion (or consequent). All of the following say exactly \(p \to q\):

The biconditional \(p \leftrightarrow q\) ("\(p\) if and only if \(q\)", often "iff") means \((p \to q) \wedge (q \to p)\): each is necessary and sufficient for the other.

An identity is an equation that is true for every value of its variables, such as \(2(x + 3) = 2x + 6\) or \((a + b)^2 = a^2 + 2ab + b^2\). Some books emphasise this with a triple bar, \(2(x + 3) \equiv 2x + 6\), read "is identically equal to". Identities are the tools of algebra: each one licenses you to replace one expression by another.

An identity is an equation that holds for every value of the variable for which both sides are defined, such as \((a + b)^2 = a^2 + 2ab + b^2\). An ordinary equation such as \(\sin x = \tfrac12\) holds only for particular values. Some books write \(\equiv\) for identities.

An identity element for an operation leaves things unchanged: \(a + 0 = a\) (0 is the additive identity), \(a \times 1 = a\) (1 is the multiplicative identity). An inverse undoes: \(-a\) is the additive inverse (or opposite) of \(a\) because \(a + (-a) = 0\); \(\tfrac{1}{a}\) is the multiplicative inverse (or reciprocal) of \(a \neq 0\) because \(a \cdot \tfrac{1}{a} = 1\).

An identity is an equation true for every value of the variable, such as \(2(x + 3) = 2x + 6\). A contradiction is true for no value, such as \(x + 1 = x + 2\). A conditional equation is true for some values and false for others, such as \(2x = 6\). The symbol \(\varnothing\) denotes the empty set, the set with no elements; \(\mathbb{R}\) denotes the set of all real numbers.

The imaginary unit \(i\) is a number with the defining property \(i^2 = -1\). (Electrical engineers write \(j\), since \(i\) means current; so does Python.) The square roots of a negative number \(-a\) (with \(a > 0\)) are \(\pm i\sqrt{a}\); for example \(\sqrt{-9} = 3i\).

A function is given explicitly when \(y\) is isolated: \(y = \sqrt{25 - x^2}\). It is given implicitly by an equation relating \(x\) and \(y\) without isolating either: \(x^2 + y^2 = 25\). Implicit differentiation finds \(dy/dx\) directly from the implicit equation.

An improper integral is one where the interval is unbounded (a limit of integration is \(\pm\infty\)) or the integrand blows up somewhere in the interval. It is defined as a limit of ordinary integrals and is said to converge if that limit is a finite number and diverge otherwise.

A function \(f\) is increasing on an interval if \(a < b\) implies \(f(a) < f(b)\) for any two points in the interval, and decreasing if \(a < b\) implies \(f(a) > f(b)\). Test: \(f' > 0\) on an interval ⇒ increasing there; \(f' < 0\) ⇒ decreasing.

On an interval \(I\):

Some books say "strictly increasing" for this and "increasing" or "non-decreasing" for the version with \(\le\). A local maximum is a point higher than all nearby points; a local minimum, lower. They occur where a function switches between increasing and decreasing.

The indefinite integral \(\int f(x)\,dx\) (no limits) denotes the family of all antiderivatives of \(f\):

\(C\) is the constant of integration: an arbitrary constant, because adding a constant does not change a derivative. A definite integral is a number; an indefinite integral is a family of functions.

When one quantity is computed from another, the input is the independent variable and the output is the dependent variable. In \(y = mx + b\), \(x\) is independent, \(y\) is dependent. In a formula such as \(A = \pi r^2\), \(r\) is independent and \(A\) is dependent.

Events \(A\) and \(B\) are independent if

Equivalently (when \(P(B) > 0\)), \(P(A \mid B) = P(A)\): knowing \(B\) happened doesn’t change the probability of \(A\). Events that aren’t independent are dependent.

Several events \(A_1, \ldots, A_n\) are mutually independent if the product rule holds for every sub-collection, e.g. \(P(A_1 \cap A_2 \cap A_3) = P(A_1)P(A_2)P(A_3)\).

An indeterminate form is a limit whose naive substitution gives an expression like \(\frac00\), \(\frac{\infty}{\infty}\), \(\infty - \infty\), \(0\cdot\infty\), or \(1^\infty\). "Indeterminate" means the form alone does not determine the answer: \(\frac{x}{x}\), \(\frac{x^2}{x}\), and \(\frac{x}{x^2}\) are all \(\frac00\) at 0, with limits 1, 0, and "does not exist". The expression must be transformed until substitution works.

An indeterminate form is a limit shape whose value cannot be decided from the limits of the pieces alone: \(\frac{0}{0}\), \(\frac{\infty}{\infty}\), \(0\cdot\infty\), \(\infty - \infty\), \(0^0\), \(1^\infty\), \(\infty^0\). L’Hôpital applies directly only to the first two; the others are first rewritten as a quotient.

The indicator of an event \(A\), written \(I_A\) or \(\mathbf{1}_A\), is the random variable that equals 1 if \(A\) happens and 0 otherwise. Its expectation is the probability of the event:

\(<\) and \(>\) are strict inequalities; \(\le\) and \(\ge\) are non-strict (or weak). The narrow end of the symbol points at the smaller number.

\(\lim_{x \to a} f(x) = \infty\) means \(f(x)\) becomes larger than any given number when \(x\) is close enough to \(a\) (\(x \ne a\)). Similarly for \(-\infty\). The limit does not exist as a real number; the notation records the direction of failure. The line \(x = a\) is then a vertical asymptote of the graph.

For \(f : A \to B\):

A function has a two-sided inverse \(f^{-1} : B \to A\) exactly when it is bijective.

If \(f\) sends \(x\) to \(y\), we write \(f(x) = y\), read "\(f\) of \(x\) equals \(y\)".

The parentheses in \(f(x)\) mean "apply \(f\) to", not multiplication. \(f(x)\) is one object, the output.

An integer is a whole number that may be positive, negative, or zero: \(\ldots, -3, -2, -1, 0, 1, 2, 3, \ldots\). The set is written \(\mathbb{Z}\), from the German Zahlen, "numbers". The integers are closed under addition, subtraction, and multiplication: doing any of those to two integers gives an integer.

Integration by parts is the formula \(\int u\,dv = uv - \int v\,du\), obtained by integrating the product rule. For definite integrals: \(\int_a^b u\,dv = \big[uv\big]_a^b - \int_a^b v\,du\). A rough guide for choosing \(u\) is LIATE: Logarithmic, Inverse trig, Algebraic (powers of \(x\)), Trigonometric, Exponential, taking as \(u\) whichever comes first in that list.

The interior angles of a polygon are the angles inside it at each vertex. Extend one side past a vertex and the angle between the extension and the next side is an exterior angle. Each exterior angle forms a linear pair with its interior angle.

Intermediate value theorem (IVT): if \(f\) is continuous on the closed interval \({[}a, b]\) and \(N\) is any number between \(f(a)\) and \(f(b)\), then there is at least one \(c\) in \({[}a, b]\) with \(f(c) = N\).

The most-used special case: if \(f\) is continuous on \({[}a, b]\) and \(f(a)\) and \(f(b)\) have opposite signs, then \(f\) has a root (a zero) somewhere in \((a, b)\).

An interval is an unbroken stretch of the number line. A square bracket means the endpoint is included; a round bracket (parenthesis) means it is excluded.

The symbol \(\infty\) (infinity) is not a number; \(\lbrack a, \infty)\) means "no upper limit". Infinity therefore always gets a round bracket. On a number line, a filled dot marks an included endpoint and an open circle an excluded one.

Interval notation describes runs of real numbers. A square bracket includes the endpoint; a round bracket excludes it.

Set-builder notation \(\{x \in \mathbb{R} : x \ne 2\}\) reads "the set of real \(x\) such that \(x \ne 2\)". The colon (sometimes a vertical bar) means "such that".

Python programmers already know this: range(2, 5) is the half-open interval \(\lbrack 2, 5)\) of integers.

An inverse operation undoes another. Adding \(k\) and subtracting \(k\) are inverses; multiplying by \(k\) and dividing by \(k\) are inverses, provided \(k \neq 0\). Applying an operation and then its inverse returns you to where you started.

\(\sin^{-1} x\) (also \(\arcsin x\)), \(\cos^{-1} x\) (\(\arccos x\)) and \(\tan^{-1} x\) (\(\arctan x\)) return the angle whose sine, cosine or tangent is \(x\).

The principal ranges are \(-90^\circ \le \sin^{-1} x \le 90^\circ\), \(0^\circ \le \cos^{-1} x \le 180^\circ\) and \(-90^\circ < \tan^{-1} x < 90^\circ\). There are infinitely many angles with a given sine, so the inverse function has to pick one (see Section 13.3).

\(\sin^{-1} x\) is not \(1/\sin x\). The \(-1\) here means "inverse function", as in \(f^{-1}\) (Chapter 11), not "reciprocal". This is a notorious notational clash, and it’s why many people prefer writing arcsin.

A square matrix \(A\) is invertible (or non-singular) if there’s a matrix \(A^{-1}\) with \(AA^{-1} = A^{-1}A = I\). Otherwise it is singular. For a square matrix, all of these are equivalent: \(A\) is invertible; \(\det A \neq 0\); the columns are linearly independent; the columns span the whole space; \(A\mathbf{x} = \mathbf{b}\) has exactly one solution for every \(\mathbf{b}\); \(A\mathbf{x} = \mathbf{0}\) only for \(\mathbf{x} = \mathbf{0}\). Non-square matrices never have a two-sided inverse.

An irrational number is a real number that is not rational — it cannot be written as a ratio of integers. Its decimal expansion never terminates and never settles into a repeating block. Examples: \(\sqrt{2} = 1.41421356\ldots\), \(\pi = 3.14159265\ldots\), \(e = 2.71828182\ldots\).

A polynomial is irreducible over a given number system if it cannot be written as a product of two polynomials of lower degree with coefficients in that system. \(x^2 + 4\) is irreducible over the real numbers; \(x^2 - 2\) is irreducible over the rationals but factors over the reals as \((x - \sqrt 2)(x + \sqrt 2)\). "Factor completely" means: keep going until every factor is irreducible (usually over the integers or rationals, unless stated).

The Jacobian of \(\mathbf{F}(x, y) = \big(F_1(x, y), F_2(x, y)\big)\) is the matrix of all first partial derivatives:

Row \(i\) is \(\nabla F_i\). The Jacobian is the multi-output version of the derivative: small input changes \(\Delta\mathbf{x}\) produce output changes \(\Delta\mathbf{F} \approx J\,\Delta\mathbf{x}\). Matrix–vector multiplication is explained in Chapter 27.

Two lines that meet at a right angle are perpendicular, written \(\ell_1 \perp \ell_2\). Lines in a plane that never meet are parallel, written \(\ell_1 \parallel \ell_2\).

Unless stated otherwise, "graph" in this chapter means a simple undirected graph.

A probability assigns to each event \(A\) a number \(P(A)\) such that

An axiom is a starting rule we accept without proof; everything else is derived from the axioms.

In triangle \(ABC\), the angles at the vertices are \(A\), \(B\), \(C\), and the side opposite each angle gets the matching lower-case letter: side \(a\) faces angle \(A\), and so on. The angles always add to \(180^\circ\).

To optimise \(f(\mathbf{x})\) subject to a constraint \(g(\mathbf{x}) = c\), look for points where \(\nabla f = \lambda\nabla g\) and \(g = c\). The number \(\lambda\) (Greek "lambda") is the Lagrange multiplier. It measures how fast the optimal value of \(f\) changes as the constraint level \(c\) is relaxed: \(\lambda = d f^*/dc\). Economists call it the shadow price of the constraint.

In any triangle with sides \(a\), \(b\), \(c\), where \(\theta\) is the angle between sides \(a\) and \(b\) (opposite side \(c\)):

When \(\theta = 90^\circ\), \(\cos\theta = 0\) and it reduces to Pythagoras. The \(-2ab\cos\theta\) term is the correction for the triangle not being right-angled. (Derived in Chapter 13.)

For non-zero bases and integer exponents: \(a^m a^n = a^{m+n}\), \(a^m / a^n = a^{m-n}\), \((a^m)^n = a^{mn}\), \((ab)^n = a^n b^n\), \((a/b)^n = a^n / b^n\), \(a^0 = 1\), \(a^{-n} = 1/a^n\). They extend to fractional exponents for positive bases. Each can be re-derived by writing out the factors.

For \(b, c > 0\), \(b, c \ne 1\), \(M, N > 0\), and any real \(k\):

For any triangle:

(and the same with the letters cycled: \(b^2 = a^2 + c^2 - 2ac\cos B\), \(c^2 = a^2 + b^2 - 2ab\cos C\)).

When \(A = 90^\circ\), \(\cos A = 0\) and the law of cosines is Pythagoras. The term \(-2bc\cos A\) is the correction for a non-right angle. It is negative (side shorter) for acute \(A\) and positive (side longer) for obtuse \(A\).

The least common multiple \(\operatorname{lcm}(a, b)\) is the smallest positive integer that both \(a\) and \(b\) divide. It is the "lowest common denominator" you used when adding fractions in Chapter 2.

The leading term is the term of highest degree; its coefficient is the leading coefficient. In \(-2x^5 + x^2 - 9\) the leading term is \(-2x^5\) and the leading coefficient is \(-2\). A polynomial whose leading coefficient is \(1\) is called monic.

The learning rate (or step size) \(\eta\) is the positive number multiplying the gradient in each update. Machine learning calls it the learning rate. Numerical analysis calls it the step size and often writes \(\alpha\).

Each \(\mathbf{x}_k\) is an iterate. The method converges if the iterates approach a limit (ideally a minimum), and diverges if they run off, typically growing without bound.

The learning rate \(\eta\) is the step-size multiplier in gradient descent. A mini-batch is a small random subset of the training data used to estimate the gradient for one step. An epoch is one full pass through the training data — with 1,000 examples and batches of 32, one epoch is 32 steps (the last batch is short). Momentum \(\beta\) (Greek beta, typically 0.9) controls how much of the previous velocity carries into the next step.

The least-squares regression line (or line of best fit) of \(y\) on \(x\) is \(\hat y = a + bx\) with

\(b\) is the slope (change in predicted \(y\) per unit of \(x\)); \(a\) is the intercept. The line always passes through the point of means \((\bar x, \bar y)\). \(\hat y\) ("y-hat") denotes a predicted value. A residual is \(y_i - \hat y_i\).

Like terms have exactly the same variable part, with each variable raised to the same power. \(5x^2y\) and \(-2yx^2\) are like terms (order of multiplication does not matter). \(3x\) and \(3x^2\) are not like terms; neither are \(xy\) and \(x\). Only like terms can be combined into a single term.

Like terms have the same variable raised to the same power: \(3x^2\) and \(-5x^2\) are like terms; \(3x^2\) and \(3x\) are not. Only like terms can be combined: \(3x^2 - 5x^2 = -2x^2\).

\(\lim_{x\to\infty} f(x) = L\) means \(f(x)\) can be made as close to \(L\) as we like by taking \(x\) sufficiently large. Similarly for \(x \to -\infty\). If such an \(L\) exists, the horizontal line \(y = L\) is a horizontal asymptote. For a sequence (Chapter 14), \(\lim_{n\to\infty} a_n = L\) means the same thing with \(n\) restricted to whole numbers; this is exactly the "convergence" of the previous chapter, now with a name.

\(\lim_{x\to a} f(x) = L\) means: for every \(\varepsilon > 0\) there exists a \(\delta > 0\) such that

Here \(|x - a|\) is the distance between \(x\) and \(a\) (Chapter 5); "\(0 < |x - a|\)" excludes \(x = a\) itself; and \(\Longrightarrow\) reads "implies". In the language of logic (Chapter 21): \(\forall\varepsilon > 0\ \exists\delta > 0\ \forall x\ (0 < |x - a| < \delta \Rightarrow |f(x) - L| < \varepsilon)\).

We write

and say "the limit of \(f(x)\) as \(x\) approaches (or tends to) \(a\) is \(L\)" if \(f(x)\) can be made as close to \(L\) as we like by taking \(x\) sufficiently close to \(a\), but not equal to \(a\). Reading the notation piece by piece: lim abbreviates "limit"; the subscript \(x \to a\) says which variable is moving and where it is heading; \(f(x)\) is the expression being watched; \(L\) is the value it approaches. An alternative notation is \(f(x) \to L \text{ as } x \to a\).

A linear equation in one unknown \(x\) is an equation that can be rearranged into the form

where \(a\) and \(b\) are constants. The unknown appears only to the first power: no \(x^2\), no \(\sqrt{x}\), no \(1/x\), no \(x\) multiplied by another unknown. The name comes from the Latin linea, "line": graph \(y = ax + b\) and you get a straight line (Section 5.10).

An equation is linear if each unknown appears only to the first power, multiplied by a constant, and unknowns are never multiplied together. Its standard form is

Here the subscripted \(x_1, x_2, \ldots, x_n\) are the unknowns, the \(a_i\) are their coefficients, and \(b\) is the constant term (or right-hand side). So \(3x - 2y = 7\) is linear. \(xy = 7\), \(x^2 + y = 1\) and \(\sqrt{x} = y\) are not.

A linear transformation (or linear map) \(T: \mathbb{R}^n \to \mathbb{R}^m\) is a function from \(n\)-dimensional vectors to \(m\)-dimensional vectors satisfying \(T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v})\) and \(T(c\mathbf{v}) = cT(\mathbf{v})\). The arrow notation \(T: \mathbb{R}^n \to \mathbb{R}^m\) reads "T maps R-n to R-m": the domain is \(\mathbb{R}^n\), the codomain \(\mathbb{R}^m\). Geometrically: the origin is fixed, and grid lines stay straight, parallel and evenly spaced.

The linearisation (or tangent line approximation) of \(f\) at \(a\) is \(L(x) = f(a) + f'(a)(x - a)\), and \(f(x) \approx L(x)\) for \(x\) near \(a\).

The differentials \(dx\) and \(dy\) restate this: \(dx\) is a small change in the input, and \(dy = f'(x)\,dx\) is the corresponding change along the tangent line. The true change is \(\Delta y = f(x + dx) - f(x)\), and \(\Delta y \approx dy\). This is where Leibniz’s notation \(\frac{dy}{dx}\) earns its look: \(dy = \frac{dy}{dx}\,dx\).

The linearisation (or linear approximation) of \(f\) at \(\mathbf{a}\) is \(L(\mathbf{x}) = f(\mathbf{a}) + \nabla f(\mathbf{a}) \cdot (\mathbf{x} - \mathbf{a})\). Its graph, for two inputs, is the tangent plane.

The total differential writes the same idea in terms of small changes: \(df = f_x\,dx + f_y\,dy\).

\(f\) has a local maximum at \(c\) if \(f(c) \ge f(x)\) for all \(x\) near \(c\); a local minimum if \(f(c) \le f(x)\) nearby. A global (or absolute) maximum or minimum compares against every point of the domain. Extremum (plural extrema) means "maximum or minimum". UK texts often say turning point for a local extremum.

A local minimum is a point lower than everything nearby. A global minimum is lower than (or equal to) everything. A saddle point is a critical point (gradient zero) that is a minimum along some directions and a maximum along others — like the middle of a horse saddle.

For a base \(b > 0\), \(b \ne 1\), and a positive number \(x\):

Read \(\log_b x\) as "log base \(b\) of \(x\)". It is the exponent you put on \(b\) to get \(x\). In the equation \(b^y = x\):

The double arrow \(\Longleftrightarrow\) means "if and only if": each statement implies the other, so they are two ways of saying the same thing.

A logarithmic scale positions a value \(v\) at distance proportional to \(\log v\). Equal steps along it are equal multiplications. A decade is a factor of 10, and an octave (in music, and in frequency analysis) is a factor of 2.

Two propositions \(A\) and \(B\) are logically equivalent, written \(A \equiv B\), if they have identical truth tables. Equivalently, \(A \leftrightarrow B\) is true in every row.

A tautology is a proposition that is true in every row (for example \(p \vee \neg p\)). A contradiction is false in every row (for example \(p \wedge \neg p\)). A proposition that is neither is contingent. A proposition is satisfiable if at least one row makes it true.

\(\equiv\) is a statement about two propositions (a "meta" claim), not a connective inside one.

The lowest common denominator of several fractions is the least common multiple of their denominators: the smallest positive number each denominator divides into. For \(\tfrac{1}{3}\) and \(\tfrac{1}{4}\) it is \(12\); for \(\tfrac{1}{4}\) and \(\tfrac{1}{6}\) it is \(12\), not \(24\). (See Chapter 2.) Multiplying an equation through by the LCD is called clearing fractions.

Machine epsilon \(\varepsilon_{\text{mach}}\) is the gap between 1 and the next larger floating-point number. For doubles \(\varepsilon_{\text{mach}} = 2^{-52}\approx 2.22\times10^{-16}\). The unit roundoff \(u=\varepsilon_{\text{mach}}/2 = 2^{-53}\approx1.11\times10^{-16}\) is the largest relative error committed by rounding a real number (in range) to the nearest double. "Doubles carry about 16 significant decimal digits" is shorthand for \(-\log_{10}u\approx 15.95\).

The magnitude, length or norm of \(\mathbf{v}\) is written \(\|\mathbf{v}\|\) (double bars; some books use single bars \(|\mathbf{v}|\)). It’s the Euclidean norm, also called the L2 norm and written \(\|\mathbf{v}\|_2\) in machine-learning papers.

The symbol \(\sum_{i=1}^{n} v_i^2\) is summation notation: \(\Sigma\) (capital Greek sigma, for "sum") means "add up", the index \(i\) runs from the value below (1) to the value above (\(n\)), and \(v_i^2\) is the term being added for each \(i\). So \(\sum_{i=1}^{3} v_i^2 = v_1^2 + v_2^2 + v_3^2\). In code it’s sum(x*x for x in v).

\(x \mapsto x^2\) (a barred arrow) reads "\(x\) maps to \(x^2\)". It describes what happens to a single input, and lets you talk about a function without naming it. Compare:

\(x \in A\) reads "\(x\) is an element of \(A\)" (or "\(x\) is in \(A\)"). \(\mathbb{R}\) is the set of all real numbers, \(\mathbb{Z}\) the integers, \(\mathbb{N}\) the natural numbers (see Chapter 1).

A Markov chain is a random process that moves between states, where the probability of the next state depends only on the current one (it has no memory of earlier history). Its transition matrix \(P\) has entry \(p_{ij}\) = probability of moving to state \(i\) from state \(j\). With this convention (column-stochastic), columns sum to 1 and distributions update as \(\mathbf{x} \mapsto P\mathbf{x}\). Many statistics texts use the transpose (row-stochastic, rows sum to 1, \(\mathbf{x}^\mathsf{T} \mapsto \mathbf{x}^\mathsf{T}P\)); check which a source uses. A steady state (or stationary distribution) is a probability vector with \(P\mathbf{x} = \mathbf{x}\): an eigenvector with eigenvalue 1, scaled so its entries sum to 1.

A matrix is a rectangular array of numbers. An \(m \times n\) matrix ("m by n") has \(m\) rows and \(n\) columns; rows first, always. The entry in row \(i\), column \(j\) is written \(a_{ij}\) (so \(a_{12}\) is row 1, column 2). As a transformation, an \(m\times n\) matrix takes \(n\)-dimensional vectors to \(m\)-dimensional vectors: column \(j\) is where the \(j\)-th basis vector lands. A square matrix has \(m = n\) and maps a space to itself. The word is Latin for "womb" (the thing from which something grows), chosen by James Sylvester in 1850.

For data \(x_1, x_2, \ldots, x_n\):

A statistic is any number computed from data. Statistics (the subject) studies how to compute and interpret them.

The midpoint of the segment from \((x_1, y_1)\) to \((x_2, y_2)\) is \(M = \left(\dfrac{x_1 + x_2}{2}, \dfrac{y_1 + y_2}{2}\right)\), the coordinate-wise average. A segment (line segment) is the part of a line between two endpoints.

An inverse of \(a\) modulo \(m\) is an integer \(a^{-1}\) with

It exists if and only if \(\gcd(a, m) = 1\), and it is unique mod \(m\). For example, \(3^{-1} \equiv 4 \pmod{11}\) because \(3 \cdot 4 = 12 \equiv 1\). The notation \(a^{-1}\) here never means the fraction \(1/a\).

The modulus (or absolute value, or magnitude) of \(z = a + bi\) is its distance from 0: \(|z| = \sqrt{a^2 + b^2}\) (Pythagoras). For a real number it agrees with the ordinary absolute value.

The argument \(\arg z\) is the angle from the positive real axis to the arrow for \(z\), measured anticlockwise, usually in radians. It is defined only for \(z \ne 0\) and only up to adding multiples of \(2\pi\). The principal argument is the choice in \((-\pi, \pi]\).

Polynomials are also named by their number of terms (after combining like terms): a monomial has one term (\(-6x^3\)), a binomial two (\(x^2 - 9\)), a trinomial three (\(x^2 + 5x + 6\)). Beyond three we just say "polynomial".

A multiset is like a set but elements may appear more than once: \(\{M, I, I, I, I, S, S, S, S, P, P\}\). The number of distinct orderings of a multiset with \(n\) items, of which \(n_1\) are of type 1, \(n_2\) of type 2, …, \(n_k\) of type \(k\), is the multinomial coefficient

With two types it is just \(\binom{n}{n_1}\): arranging \(r\) identical H’s and \(n-r\) identical T’s is choosing which \(r\) positions get an H.

The natural numbers are the counting numbers \(0, 1, 2, 3, \ldots\). The set of them is written \(\mathbb{N}\) (a "blackboard bold" N). Whether \(0 \in \mathbb{N}\) is a convention that varies: computer scientists and logicians almost always include 0; some number theorists start at 1. When it matters, authors write \(\mathbb{N}_0\) (with zero) or \(\mathbb{Z}^+\) (positive integers, without zero). This book includes 0.

A neuron computes \(\phi(\mathbf{w}\cdot\mathbf{x} + b)\) for weights \(\mathbf{w}\), bias \(b\), and activation function \(\phi\) (Greek phi). Common activations: the sigmoid \(\sigma\); \(\tanh\); and the ReLU (rectified linear unit) \(\operatorname{ReLU}(z) = \max(0, z)\). A layer applies many neurons to the same input. A hidden layer is any layer between input and output. The forward pass evaluates the network; the backward pass computes gradients.

A \(2\times 3\) matrix has 3 columns (images of \(\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3\) in 3D), each with 2 entries (they land in 2D). So it maps \(\mathbb{R}^3 \to \mathbb{R}^2\), for example by dropping the depth of a 3D point to draw it on paper: \(\begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \end{bmatrix}\) keeps \(x\), \(y\) and discards \(z\). An \(m\times n\) matrix eats \(n\)-vectors and produces \(m\)-vectors. A neural-network layer from 768 to 3072 dimensions is a \(3072\times 768\) matrix.

The normal (or Gaussian) distribution with mean \(\mu\) and standard deviation \(\sigma\), written \(N(\mu, \sigma^2)\), has density

It is symmetric about \(\mu\) (mean = median = mode), bell-shaped, and its tails thin out very fast. The standard normal is \(Z \sim N(0, 1)\); its CDF is written \(\Phi(z)\) (capital phi). The \(\sqrt{2\pi}\) makes the total area 1.

Note: the second parameter in \(N(\mu, \sigma^2)\) is the variance. \(N(100, 225)\) has standard deviation 15.

Lagrange (prime) notation: \(f'(x)\), \(f''(x)\) for the second derivative. Compact; good when the function has a name.

Leibniz notation: if \(y = f(x)\), write \(\dfrac{dy}{dx}\) ("dee y by dee x"). It looks like \(\dfrac{\Delta y}{\Delta x}\) with the deltas made infinitely small, which is exactly the intuition. Value at a point: \(\left.\dfrac{dy}{dx}\right\rvert_{x=a}\).

Operator notation: \(\dfrac{d}{dx}\) is an operator: a machine that takes a function and returns its derivative. \(\dfrac{d}{dx}\left(x^2\right) = 2x\) reads "the derivative with respect to \(x\) of \(x^2\) is \(2x\)". Also written \(D_x\) or \(D\).

Newton (dot) notation: \(\dot{x}\) for the derivative of \(x\) with respect to time, \(\ddot{x}\) for the second. Common in mechanics and physics.

\(e \approx 2.71828\) is Euler’s number. It is irrational (its decimal expansion never repeats), like \(\pi\). Two key facts about it:

In \(\dfrac{a}{b}\) the top number \(a\) is the numerator (it numbers, or counts, the parts) and the bottom number \(b\) is the denominator (it denominates, or names, the size of the parts: quarters, eighths). The line between them is the fraction bar or vinculum, and it means division. \(a/b\) (with a slash, or solidus) means the same thing. The denominator can never be 0 (Section 1.4).

Numerical analysis is the branch of mathematics that designs and analyses algorithms for approximating answers to problems (roots, integrals, derivatives, solutions of equations) when exact symbolic answers are unavailable, too expensive, or meaningless because the data are themselves approximate. Its two central questions are how fast does the method approach the answer? and how much error does it introduce?

In an optimisation problem, the objective function is the quantity you want to maximise or minimise (volume, area, cost, error). A constraint is a condition the variables must satisfy (fixed fence length, fixed volume). The feasible set or domain is the set of values satisfying all constraints. The best value of the objective is the optimum; the \(x\) achieving it is the optimiser (\(\arg\max\) or \(\arg\min\)).

The objective function (in ML, the loss or cost function) is the quantity being minimised, often written \(L(\theta)\) or \(f(\mathbf{x})\). The parameters \(\theta\) (Greek theta) are the knobs; when there are many, \(\theta\) is a vector. A minimiser \(\theta^*\) (read "theta star") is a parameter value where the objective is smallest. The minimum is the smallest value \(L(\theta^*)\); the minimiser is where it occurs.

The odds of an event with probability \(p\) are \(p/(1-p)\): a probability of 0.8 is odds of 4 to 1. The logit is the log-odds, \(\operatorname{logit}(p) = \ln\frac{p}{1-p}\), which runs over all of \(\mathbb{R}\). The sigmoid \(\sigma(z) = 1/(1 + e^{-z})\) is the inverse of the logit. So logistic regression says: the log-odds are a linear function of the input. Each extra unit of \(x\) multiplies the odds by \(e^{w}\).

\(\lim_{x\to a^-} f(x) = L\) (the left-hand limit) means \(f(x)\) approaches \(L\) as \(x\) approaches \(a\) from below (\(x < a\)). \(\lim_{x\to a^{+}} f(x) = L\) (the right-hand limit) means the same with \(x > a\). The superscript \(-\) or \({+}\) is a direction, not a sign on \(a\): \(x \to 2^-\) means \(x\) increases towards 2 through values like 1.9, 1.99, …

Key fact: the two-sided limit \(\lim_{x \to a} f(x)\) exists and equals \(L\) exactly when both one-sided limits exist and both equal \(L\).

A function is one-to-one (or injective) if different inputs always give different outputs: \(x_1 \ne x_2 \Rightarrow f(x_1) \ne f(x_2)\). Equivalently, \(f(x_1) = f(x_2) \Rightarrow x_1 = x_2\).

If \(f\) is one-to-one with range \(R\), its inverse function \(f^{-1} : R \to \text{domain of } f\) is defined by

Read \(f^{-1}\) as "\(f\) inverse". The inverse undoes \(f\) and vice versa: \(f^{-1}(f(x)) = x\) for every \(x\) in the domain of \(f\), and \(f(f^{-1}(y)) = y\) for every \(y\) in its range. In composition notation, \(f^{-1} \circ f\) and \(f \circ f^{-1}\) are identity functions. The domain and range swap roles.

In a right triangle, pick one of the two acute angles and call it \(\theta\) (the Greek letter theta, the traditional name for an angle).

"Opposite" and "adjacent" depend on which angle you are looking from. Swap to the other acute angle and they swap too.

(Uses Chapter 19; skip freely.) For a continuous variable the sum \(\sum x\,p(x)\) becomes an integral:

For the exponential, integrating by parts: \(\int_0^\infty t\,\lambda e^{-\lambda t}\,dt = \left[-t e^{-\lambda t}\right]_0^\infty + \int_0^\infty e^{-\lambda t}\,dt = 0 + \frac1\lambda\). Every rule from the discrete case (linearity, the variance formulas) carries over unchanged.

\(a < b\) means "\(a\) is less than \(b\)": \(a\) lies to the left of \(b\) on the number line. \(a > b\) means "greater than". \(a \le b\) means "less than or equal to", \(a \ge b\) "greater than or equal to", and \(a \neq b\) "not equal to". The narrow end of \(<\) points at the smaller number. A number is positive if it is \(> 0\), negative if \(< 0\), and non-negative if \(\ge 0\); zero is neither positive nor negative.

A method has order of convergence \(p\) if \(|e_{n+1}| \approx C|e_n|^p\) for large \(n\). \(p = 1\) is linear convergence (a fixed number of digits gained per step — bisection gains one binary digit); \(p = 2\) is quadratic (digits double — Newton at a simple root). The secant method has \(p = \tfrac{1+\sqrt5}{2} \approx 1.618\), the golden ratio.

Let \(e_n=|x_n-r|\) be the error after \(n\) steps. A method converges with order \(p\) if

for some constant \(C>0\) as \(n\to\infty\). Linear convergence (\(p=1\), need \(C<1\)) adds a fixed number of correct digits per step. Quadratic convergence (\(p=2\)) roughly doubles the number of correct digits per step. Superlinear means anything faster than linear.

Precedence says which operations are done first. The standard order is:

US mnemonic: PEMDAS ("Please Excuse My Dear Aunt Sally"). UK: BODMAS or BIDMAS (Brackets, Orders/Indices, Division, Multiplication, Addition, Subtraction). They are the same rule.

An ordered pair \((a, b)\) is like a two-element set except that order matters: \((1, 2) \neq (2, 1)\), and \((a, b) = (c, d)\) exactly when \(a = c\) and \(b = d\). More generally an \(n\)-tuple is \((a_1, \ldots, a_n)\).

The Cartesian product of \(A\) and \(B\) is the set of all ordered pairs with first entry from \(A\) and second from \(B\):

\(\mathbb{R} \times \mathbb{R} = \mathbb{R}^2\) is the coordinate plane of Chapter 10, which is where the name (after Descartes) comes from.

Two vectors are orthogonal if their dot product is zero. For nonzero vectors this means they meet at 90°. (The zero vector is orthogonal to everything, by this definition.) "Orthogonal" is the general-dimension word; "perpendicular" is usual in 2D and 3D geometry; "normal" is used for a vector perpendicular to a surface. A set of vectors that are mutually orthogonal and all of length 1 is orthonormal; the standard basis is the classic example.

The graph of \(y = ax^2 + bx + c\) is a parabola: a symmetric U-shaped curve. It opens upwards (a "cup", \(\cup\)) if \(a > 0\) and downwards (a "cap", \(\cap\)) if \(a < 0\). From Greek parabolē, "a placing alongside"; the name comes from the ancient Greek classification of conic sections (Chapter 10).

Two non-vertical lines with slopes \(m_1, m_2\) are

The reciprocal of \(\frac{a}{b}\) is \(\frac{b}{a}\) (flip it); the negative reciprocal flips and changes sign. Special case: horizontal (\(m = 0\)) and vertical lines are perpendicular, but the product rule cannot express this since a vertical slope does not exist.

A parameter is a letter that is held fixed within one problem but could be changed to give a related problem. In \(y = mx + b\), \(m\) and \(b\) are parameters: each choice of them gives one line from a whole family of lines. In \(ax^2 + bx + c = 0\), the coefficients \(a, b, c\) are parameters.

A parameter is a letter that stands for an arbitrary value you’re free to choose, as opposed to an unknown you’re trying to find (compare Chapter 4). A parametric solution such as \((3 - 2t,\ t)\) describes an infinite solution set by saying how every solution is built from the parameter. The unknown you set equal to the parameter is called a free variable.

The partial derivative of \(f\) with respect to \(x\) is the rate of change of \(f\) as \(x\) varies with every other input held fixed.

Notations, all meaning the same thing:

The curly \(\partial\) is read "partial" (or "del", or "dee"). It replaces the straight \(d\) of \(\frac{df}{dx}\) as a reminder that other variables exist and are being held fixed. To evaluate at a point, write \(\frac{\partial f}{\partial x}(1, 2)\) or \(f_x(1, 2)\).

A partial order is a relation that is reflexive, antisymmetric and transitive. It behaves like "comes before or equals". A set with a partial order is a poset. It is partial because two elements may be incomparable: neither \(a \preceq b\) nor \(b \preceq a\). If every pair is comparable it is a total (or linear) order, like \(\le\) on numbers.

Events \(B_1, B_2, \ldots, B_n\) form a partition of \(\Omega\) if they are mutually exclusive and together cover \(\Omega\): every outcome lies in exactly one \(B_i\). The law of total probability says, for any event \(A\),

Here \(\sum_{i=1}^{n}\) ("sigma") means "add the following expression for \(i = 1, 2, \ldots, n\)". In a tree diagram, this is "add up every path that ends in \(A\)". Simplest case: \(P(A) = P(B)P(A \mid B) + P(B^c)P(A \mid B^c)\).

A partition of \({[a, b]}\) splits it into smaller subintervals using points \(a = x_0 < x_1 < \dots < x_n = b\). With equal widths, each subinterval has width

and the partition points are \(x_i = a + i\,\Delta x\) for \(i = 0, 1, \dots, n\). The symbol \(\Delta\) ("delta") means "a change in", so \(\Delta x\) is read "delta x": a small step in \(x\).

A circle is the set of all points in a plane at a fixed distance, the radius \(r\), from a fixed point, the centre.

Percent means "per hundred" (Latin per centum): \(p\% = \tfrac{p}{100}\). So \(35\% = 0.35 = \tfrac{7}{20}\) and \(150\% = 1.5\). A percentage point (pp) is the unit for the difference between two percentages: from 4% to 5% is a rise of 1 percentage point, which is a 25% relative increase.

A permutation of a set is an arrangement of all its elements in an order. An \(r\)-permutation of \(n\) distinct objects is an ordered selection of \(r\) of them (no repeats). The number of \(r\)-permutations is written \(P(n, r)\), \({}_nP_r\) or \(n^{\underline{r}}\) (read "\(n\) to the \(r\) falling"):

\(\pi\) (Greek lowercase pi) is the ratio of any circle’s circumference to its diameter:

It is the same for every circle because all circles are similar: scaling a circle by \(k\) multiplies both \(C\) and \(d\) by \(k\), leaving the ratio unchanged. \(\pi\) is irrational (its decimal expansion never repeats; Chapter 1) and even transcendental (it is not a root of any polynomial with integer coefficients).

A piecewise function uses different formulas on different parts of its domain:

The big brace is read "\(f(x)\) equals \(x + 1\) if \(x < 1\), and \(x - 3\) if \(x \ge 1\)". The conditions must not overlap (else some \(x\) would get two outputs) and together should cover the whole domain. On a graph, a filled dot shows an endpoint that belongs to a piece; an open dot shows one that does not.

In Gaussian elimination the pivot is the entry used to eliminate the entries below it in the same column. Partial pivoting chooses, at each column, the row with the largest absolute value in that column (among the rows not yet used) and swaps it into the pivot position. Back substitution solves the resulting upper-triangular system from the last equation upward.

A graph is planar if it can be drawn in the plane with no edges crossing. Such a drawing is a plane embedding. The edges divide the plane into regions called faces, including one unbounded outer face. A planar graph can also have non-planar drawings; planarity asks whether some crossing-free drawing exists.

A point is a location with no size. Points are named with capital letters: \(A\), \(B\), \(P\).

A line is straight, has no thickness, and extends forever in both directions. The line through \(A\) and \(B\) is written \(\overleftrightarrow{AB}\).

A plane is a flat surface with no thickness that extends forever in every direction. A sheet of paper is a finite piece of one.

Points on the same line are collinear; points (or lines) in the same plane are coplanar.

A point’s polar coordinates \((r, \theta)\) are its distance \(r\) from the origin (the pole) and the angle \(\theta\) from the positive \(x\)-axis (the polar axis), measured anticlockwise.

The first line is the unit-circle definition scaled by \(r\). In the second, atan2 is the quadrant-aware inverse tangent (Section 13.9). Polar coordinates aren’t unique: \((r, \theta)\), \((r, \theta + 2\pi)\) and \((-r, \theta + \pi)\) all name the same point.

The polar form (or trigonometric form, or modulus–argument form) of a nonzero complex number is \(z = r(\cos\theta + i\sin\theta)\), where \(r = |z|\) and \(\theta = \arg z\). It is often abbreviated \(r\operatorname{cis}\theta\) ("r cis theta": cos + i sin), and, after the next section, written \(re^{i\theta}\). Conversion: \(a = r\cos\theta\), \(b = r\sin\theta\); \(r = \sqrt{a^2 + b^2}\), \(\theta = \operatorname{atan2}(b, a)\).

A polygon is a closed plane figure made of straight segments (sides or edges) meeting at vertices. It is simple if its sides don’t cross each other, and convex if every interior angle is less than \(180^\circ\) (equivalently, the segment joining any two interior points stays inside). A polygon with a "dent" is concave. A diagonal joins two non-adjacent vertices.

A regular polygon has all sides equal and all angles equal. Names by number of sides: triangle (3), quadrilateral (4), pentagon (5), hexagon (6), heptagon (7), octagon (8), decagon (10), \(n\)-gon in general.

Perimeter is the total length of the boundary. Area is the amount of surface enclosed, measured in square units.

A polynomial in \(x\) is an expression of the form

where \(n\) is a non-negative whole number and the \(a_k\) are constants (the coefficients). From Greek poly (many) plus Latin nomen (name, term): "many terms".

The power set \(\mathcal{P}(A)\) is the set of all subsets of \(A\), including \(\varnothing\) and \(A\) itself. For \(A = \{a, b, c\}\):

If \(|A| = n\) then \(|\mathcal{P}(A)| = 2^n\), because building a subset means making an independent in-or-out decision for each of the \(n\) elements. A subset of an \(n\)-element set is an \(n\)-bit string; that is why bitmasks represent sets.

In probability language (Chapter 29): recall is \(P(\text{flagged} \mid \text{positive})\) and precision is \(P(\text{positive} \mid \text{flagged})\), the two directions of conditioning. Precision is the medical test’s "probability of disease given a positive test", and so it depends on the base rate.

A predicate \(P(x)\) is a statement involving a variable \(x\) that becomes a proposition when \(x\) is given a value. The set of allowed values is the domain (or universe of discourse). With domain the integers, \(P(x)\): "\(x\) is even" gives \(P(4)\) true and \(P(7)\) false. Predicates can have several variables: \(L(x, y)\): "\(x < y\)".

A prime is an integer \(p > 1\) whose only positive divisors are 1 and \(p\). The first few are \(2, 3, 5, 7, 11, 13\). An integer \(n > 1\) that is not prime is composite. A fact we will prove by strong induction in Section 22.10: every integer \(n > 1\) has at least one prime divisor.

A prime is a natural number greater than 1 whose only positive divisors are 1 and itself: \(2, 3, 5, 7, 11, 13, \ldots\). A natural number greater than 1 that is not prime is composite: it can be written as a product of two smaller natural numbers. 1 is neither prime nor composite. 2 is the only even prime.

A prime is an integer \(p > 1\) whose only positive divisors are 1 and \(p\). An integer \(n > 1\) that is not prime is composite: it can be written \(n = ab\) with \(1 < a, b < n\). The number 1 is neither. It is called a unit.

The primes start 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, …​ and 2 is the only even one.

A prime factorization writes a number as a product of primes, usually grouped with exponents in increasing order: \(360 = 2^3 \cdot 3^2 \cdot 5\). The exponent of \(p\) is called the multiplicity of \(p\).

A continuous random variable \(X\) has a probability density function \(f(x)\) if

(The integral notation is from Chapter 19; "area under the curve" is all you need here.)

A density satisfies \(f(x) \ge 0\) and has total area 1. The height \(f(x)\) is not a probability (it can exceed 1); it’s probability per unit length, like population density is people per square mile. For a short interval of width \(\Delta x\), \(P(x \le X \le x + \Delta x) \approx f(x)\,\Delta x\).

For a continuous variable, \(P(X = a) = 0\) for every single value \(a\), so \(P(a \le X \le b) = P(a < X < b)\): endpoints don’t matter.

A random experiment is any process whose result is uncertain: rolling a die, sampling a user, running a randomized algorithm. Each possible result is an outcome. A probability is a number from 0 to 1 attached to a set of outcomes: 0 means impossible, 1 means certain. Probabilities are often written as fractions (\(\tfrac14\)), decimals (0.25) or percentages (25%); these are the same number in different clothes (Chapter 2).

Odds are a different way to express the same information: odds of \(a{:}b\) in favour means probability \(\frac{a}{a + b}\). Probability \(0.2\) is odds \(1{:}4\).

The probability mass function of a discrete random variable \(X\) is \(p(x) = P(X = x)\). It satisfies \(p(x) \ge 0\) for every \(x\), and \(\sum_x p(x) = 1\) (the sum over all possible values). "Mass" because each value carries a lump of probability, like beads of different weights on a number line.

The support of \(X\) is the set of values with \(p(x) > 0\).

\(\displaystyle\prod_{i=m}^{n} f(i) = f(m)\cdot f(m+1)\cdots f(n)\). An empty product (upper bound below lower bound) equals 1, the identity for multiplication, just as an empty sum equals 0, the identity for addition. This is why \(0! = 1\). In code: total = 0 before a summing loop, prod = 1 before a multiplying loop.

A key reading: dotting with a unit vector measures the component in that direction. That’s why \(\mathbf{v}\cdot\mathbf{i}\) is just \(v_1\).

Chapter 21 and Chapter 22 treat all of this formally.

A proper fraction has numerator smaller than denominator in size (\(\tfrac{3}{4}\)); its value lies between \(-1\) and 1. An improper fraction has numerator at least as large as denominator (\(\tfrac{7}{4}\)). A unit fraction has numerator 1 (\(\tfrac{1}{4}\)). A mixed number writes an improper fraction as a whole number plus a proper fraction: \(1\tfrac{3}{4} = 1 + \tfrac{3}{4} = \tfrac{7}{4}\). "Improper" is not an insult; in algebra improper fractions are usually preferred.

Antisymmetric is not the opposite of symmetric: a relation can be both (equality) or neither ("follows").

A proposition (or statement) is a declarative sentence that is either true or false, but not both. "7 is prime" is a proposition (true). "7 is even" is a proposition (false). "Close the door" (a command) and "\(x > 3\)" (whose truth depends on an unknown \(x\)) are not propositions. We name propositions with lowercase letters \(p, q, r\). The truth value of a proposition is T (true) or F (false), sometimes written 1 and 0.

A programming function is pure if

A pure function is a mathematical function. Calling it twice with the same arguments must give the same result, so the call can always be replaced by its value (referential transparency).

In a right triangle with legs \(a\) and \(b\) and hypotenuse \(c\),

Converse: if a triangle’s sides satisfy \(a^2 + b^2 = c^2\), the angle opposite \(c\) is a right angle.

A Pythagorean triple is a set of positive whole numbers satisfying the equation: (3, 4, 5), (5, 12, 13), (8, 15, 17), (7, 24, 25), and any multiple such as (6, 8, 10).

QED abbreviates the Latin quod erat demonstrandum, "which was to be demonstrated". Modern texts usually end a proof with a small filled square \(\blacksquare\), called a tombstone or Halmos symbol after the mathematician Paul Halmos who popularised it. Either mark says "the proof is complete; you can stop checking here".

The axes cut the plane into four quadrants, numbered with Roman numerals anticlockwise starting from the top right: I (x positive, y positive), II (x negative, y positive), III (both negative), IV (x positive, y negative). Points that lie on an axis belong to no quadrant.

A quadratic expression is a polynomial of degree 2: \(ax^2 + bx + c\) with \(a \neq 0\). A quadratic equation is an equation that can be rearranged to

its standard form. The word comes from Latin quadratum, "a square", because the defining feature is the squared unknown \(x^2\). (Confusingly, quad usually means four; here it refers to the four sides of a square.)

The solutions of \(ax^2 + bx + c = 0\) (\(a \neq 0\)) are

Read it as "the axis of symmetry \(-b/(2a)\), plus or minus a half-width \(\sqrt{\Delta}/(2a)\)". The two roots are symmetric about the vertex.

Quadrature means numerical approximation of a definite integral. The name comes from the ancient problem of "squaring" a shape: constructing a square with the same area. A quadrature rule is a weighted sum \(\sum w_i f(x_i)\) of function values at chosen nodes \(x_i\).

Watch the transatlantic trap: in the UK a trapezium has one pair of parallel sides; in the US a trapezium traditionally has none.

The domain is often written into the quantifier: \(\forall x \in \mathbb{Z}\) ("for every integer \(x\)"). A variable governed by a quantifier is bound; one that is not is free. Only a statement with no free variables is a proposition.

A queue is a list where items are added at the back and removed from the front (FIFO, "first in, first out"), like people waiting in line. A stack adds and removes at the same end (LIFO, "last in, first out"), like a pile of plates. In Python, collections.deque is an efficient queue (append, popleft); a plain list is an efficient stack (append, pop).

An angle of \(\theta\) radians at the centre of a circle of radius \(r\) cuts off an arc of length \(s = r\theta\). Radians are a ratio of two lengths, so they have no physical unit, and "rad" is often left out. If no unit is written, assume radians.

Converting: multiply degrees by \(\dfrac{\pi}{180}\) to get radians, and multiply radians by \(\dfrac{180}{\pi}\) to get degrees.

Common values: \(30^\circ = \tfrac{\pi}{6}\), \(45^\circ = \tfrac{\pi}{4}\), \(60^\circ = \tfrac{\pi}{3}\), \(90^\circ = \tfrac{\pi}{2}\), \(180^\circ = \pi\), \(270^\circ = \tfrac{3\pi}{2}\), \(360^\circ = 2\pi\).

A random variable is a function from the sample space \(\Omega\) to the real numbers: it assigns a number to every outcome. Random variables are written with capital letters (\(X\), \(Y\), \(S\)); their particular values with lower-case letters (\(x\), \(y\), \(s\)).

\(X = x\) denotes the event "\(X\) takes the value \(x\)", i.e. the set of outcomes \(\{\omega \in \Omega : X(\omega) = x\}\). So \(P(X = x)\) is an ordinary probability of an event.

A random variable is discrete if its possible values can be listed (finitely many, or a list like 0, 1, 2, …). It is continuous if it can take any value in an interval (a time, a length, a temperature).

The name is a historical accident: a random variable is neither random (it’s a fixed rule) nor a variable (it’s a function). The randomness is in which outcome occurs.

A randomized algorithm makes random choices during its execution. Two families:

Mnemonic: in Las Vegas you always leave with a (correct) answer but you don’t know how long you’ll be there; in Monte Carlo you leave on time but might have lost.

The largest finite double is about \(1.80\times10^{308}\); the smallest positive normal double is about \(2.23\times10^{-308}\); the smallest positive subnormal is about \(4.9\times10^{-324}\). A result too large to represent overflows to \(\pm\infty\). A result too small underflows, first gradually through the subnormals (losing precision) and finally to zero.

The rank of a matrix is the number of linearly independent columns (equivalently, of independent rows: the two counts always agree), which is the dimension of the set of all outputs \(A\mathbf{x}\) (the column space or image). An \(n\times n\) matrix is invertible exactly when its rank is \(n\) (full rank). The input directions squashed to zero form the null space (or kernel); its dimension is \(n - \text{rank}\).

The Greek capital delta, \(\Delta\), means "change in". If a quantity goes from \(d_1\) to \(d_2\), then \(\Delta d = d_2 - d_1\). It is a single symbol: \(\Delta t\) is not \(\Delta \times t\).

The average rate of change of \(d\) with respect to \(t\) over an interval is \(\dfrac{\Delta d}{\Delta t}\). For distance against time this is average speed; for any function it is the slope of the straight line joining the two endpoints of the graph.

A ratio compares quantities: \(a : b\) (read "a to b") means "for every \(a\) of the first, there are \(b\) of the second", and it carries the same information as the fraction \(\tfrac{a}{b}\). A rate is a ratio of quantities with different units: 60 km per hour, $3 per kilogram. A unit rate has 1 in the second place: "per one". A proportion is an equation between two ratios, \(\tfrac{a}{b} = \tfrac{c}{d}\). Two quantities are directly proportional, written \(y \propto x\), if \(y = kx\) for a fixed number \(k\), the constant of proportionality.

For \(a > 0\) and integers \(m\), \(n\) with \(n > 0\): \(a^{1/n} = \sqrt[n]{a}\) and \(a^{m/n} = (\sqrt[n]{a})^m\). All the laws of exponents continue to hold. So \(\sqrt{x} = x^{1/2}\), \(\dfrac{1}{\sqrt{x}} = x^{-1/2}\), \(\sqrt[3]{x^2} = x^{2/3}\). Converting roots to fractional exponents is how calculus handles them (Chapter 17).

A rational expression is a quotient of two polynomials, \(\dfrac{P(x)}{Q(x)}\) with \(Q\) not the zero polynomial. "Rational" here means "ratio", as in rational numbers \(p/q\).

A rational function is a quotient of two polynomials, \(R(x) = \dfrac{P(x)}{Q(x)}\), with \(Q\) not the zero polynomial (the name comes from ratio, as with rational numbers). Its natural domain is every real number except the zeros of \(Q\).

An asymptote is a line that the graph approaches ever more closely:

"\(x \to \infty\)" reads "as \(x\) grows without bound". The precise language for "gets as close as you like" is the limit, defined in Chapter 16.

A rational number is any number that can be written as a ratio \(\dfrac{p}{q}\) of integers \(p\) and \(q\) with \(q \neq 0\). The set is written \(\mathbb{Q}\), for quotient. Examples: \(\tfrac{1}{2}\), \(-\tfrac{7}{3}\), \(5 = \tfrac{5}{1}\), \(0.125 = \tfrac{1}{8}\). In decimal form a rational number either terminates or eventually repeats (shown in Chapter 2).

If a polynomial with integer coefficients has a rational root \(p/q\) in lowest terms, then \(p\) divides the constant term \(a_0\) and \(q\) divides the leading coefficient \(a_n\). For a monic polynomial (\(a_n = 1\)), every rational root is an integer dividing \(a_0\).

The real numbers, written \(\mathbb{R}\), are all the rational and irrational numbers together: every point on the number line. Every length, every measurement, every point on a continuous scale is a real number. The defining extra property of \(\mathbb{R}\) over \(\mathbb{Q}\) is that it has no gaps (mathematicians call this completeness).

The reciprocal of a non-zero number \(x\) is \(\tfrac{1}{x}\), the number you multiply \(x\) by to get 1. The reciprocal of \(\tfrac{a}{b}\) is \(\tfrac{b}{a}\) ("turn it upside down"), because \(\tfrac{a}{b} \cdot \tfrac{b}{a} = \tfrac{ab}{ba} = 1\). Zero has no reciprocal.

Three more names you’ll meet in older texts and in calculus:

Note the pairing: *co*secant goes with sine, and secant goes with *co*sine. They add no new information, only convenient shorthand.

A recurrence relation defines each term of a sequence from earlier terms. Its order is how far back it reaches: \(a_n = 3a_{n-1}\) is first order, \(F_n = F_{n-1} + F_{n-2}\) is second order. An order-\(k\) recurrence needs \(k\) initial conditions. Solving a recurrence means finding an explicit formula for \(a_n\).

Short definitions of vocabulary and notation, written to be scanned quickly. Together they form the glossary and the cheat sheet in the appendices: the "I remember learning this, remind me what it means" reference.

A (binary) relation \(R\) from set \(A\) to set \(B\) is a subset of the Cartesian product: \(R \subseteq A \times B\). We write \(a \mathrel{R} b\) for \((a, b) \in R\). A relation on \(A\) is a relation from \(A\) to itself. Example on \(\{1,2,3\}\): "less than" is \(\{(1,2), (1,3), (2,3)\}\).

Relaxing an edge \(u \to v\) of weight \(w\) means: if \(\mathrm{dist}(u) + w < \mathrm{dist}(v)\), set \(\mathrm{dist}(v) = \mathrm{dist}(u) + w\) and \(\mathrm{parent}(v) = u\). It asks "is going via \(u\) better than the best route to \(v\) known so far?" The symbol \(\infty\) (infinity) stands for "no route found yet".

The remainder when \(p(x)\) is divided by \((x - r)\) is \(p(r)\).

A model is a function with adjustable parameters that maps inputs to predictions; here \(\hat y = b + wx\) with intercept \(b\) and slope (or weight) \(w\). A residual is \(r_i = y_i - \hat y_i\), observed minus predicted. The sum of squared errors is \(\text{SSE} = \sum_i r_i^2\); dividing by \(n\) gives the mean squared error (MSE). The design matrix \(X\) has one row per example and one column per parameter; for a line, row \(i\) is \(\lbrack 1,\ x_i]\), the 1 multiplying the intercept.

The residue of \(a\) mod \(m\) is the remainder \(a \bmod m\), one of \(0, 1, \ldots, m-1\). The residue class \(\lbrack a]_m\) is the set of all integers congruent to \(a\). For example, \(\lbrack 2]_{12} = \{\ldots, -10, 2, 14, 26, \ldots\}\). There are exactly \(m\) classes, and the set of them is written \(\mathbb{Z}_m\) or \(\mathbb{Z}/m\mathbb{Z}\).

A Riemann sum for \(f\) on \({[a, b]}\) is any sum of the form \(\sum_{i=1}^{n} f(x_i^*)\,\Delta x\): heights times widths, added. Each term is the (signed) area of one thin rectangle. Named after Bernhard Riemann (1826–1866), who made the definition precise.

A root of the equation \(p(x) = 0\), also called a zero of the polynomial \(p\), is a number \(r\) with \(p(r) = 0\). On the graph \(y = p(x)\), real roots are the \(x\)-coordinates where the curve meets the \(x\)-axis (the x-intercepts). "Root", "zero" and "solution" are used almost interchangeably; strictly, equations have solutions or roots and functions have zeros.

A root or zero of \(f\) is a value \(r\) with \(f(r) = 0\); graphically, where the curve crosses or touches the \(x\)-axis. An iterative method produces a sequence of approximations \(x_0, x_1, x_2, \ldots\) (read "x-nought, x-one, …"; the subscript is the step number), each computed from the previous one by the same rule. \(x_0\) is the initial guess. The method converges if \(x_n \to r\) as \(n \to \infty\) (a limit of a sequence, Chapter 14).

A root or solution of \(ax^2 + bx + c = 0\) is a value of \(x\) that makes the equation true. The same number is called a zero of the function \(f(x) = ax^2 + bx + c\), because \(f\) takes the value zero there. On a graph, real roots are the x-intercepts: where the curve meets the \(x\)-axis. A quadratic has at most two roots.

The nth roots of unity are the \(n\) complex solutions of \(z^n = 1\): \(\omega^k = e^{2\pi i k/n}\) for \(k = 0, 1, \ldots, n-1\), where \(\omega = e^{2\pi i/n}\) is the primitive nth root of unity (\(\omega\) is the Greek letter omega). They lie on the unit circle at the vertices of a regular \(n\)-gon, one vertex at 1. For \(n \ge 2\) they sum to zero.

To round a number is to replace it by a nearby number with fewer digits. Rounding "to the nearest 10", "to 2 decimal places" (2 d.p.), or "to 3 significant figures" (3 s.f.) specifies which digit is the last one kept. The standard rule: look at the first digit dropped; if it is 5 or more round up, otherwise round down. Truncation simply drops digits (always toward zero). The symbol \(\approx\) means "approximately equal to".

A sample point \(x_i^*\) (read "x i star") is the point inside the \(i\)-th subinterval where you measure the height \(f(x_i^*)\). Left, right and midpoint sums are three particular choices of sample point.

The sample space, written \(\Omega\) (capital Greek omega) or \(S\), is the set of all possible outcomes of an experiment.

An event is a subset of the sample space: a collection of outcomes. We say "event \(A\) occurs" if the outcome that actually happens is a member of \(A\).

\(P(A)\), read "the probability of \(A\)", is the probability that event \(A\) occurs.

Examples. One die: \(\Omega = \{1, 2, 3, 4, 5, 6\}\); "roll is even" is the event \(\{2, 4, 6\}\). One coin tossed twice: \(\Omega = \{HH, HT, TH, TT\}\); "at least one head" is \(\{HH, HT, TH\}\).

\(s\) is the sample standard deviation, roughly the typical distance of a data point from the mean, in the data’s units. Greek letters (\(\mu\), \(\sigma\)) denote population quantities, the true values for the whole population or process; Latin letters (\(\bar x\), \(s\)) denote sample quantities computed from data.

Why \(n - 1\)? The deviations are measured from \(\bar x\), which was fitted to the same data and so sits closer to the data than the true mean \(\mu\) does. Dividing by \(n\) would systematically underestimate \(\sigma^2\); dividing by \(n - 1\) (Bessel’s correction) exactly fixes the average bias. For large \(n\) it hardly matters.

A scalar is an ordinary single number: a size with no direction (mass 4 kg, temperature 20 °C, speed 30 m/s). A vector is a quantity with both magnitude (size) and direction (displacement 30 m north-east, velocity 30 m/s due south). In the list view, a vector in \(n\) dimensions is an ordered list of \(n\) real numbers. The word "scalar" comes from the fact that multiplying a vector by a number scales it.

A number is in scientific notation (UK: standard form) when written as \(m \times 10^k\) with \(1 \le |m| < 10\) and \(k\) an integer. \(m\) is the significand (or mantissa or coefficient). Programming languages and calculators write \(5.97 \times 10^{24}\) as 5.97e24 (E notation). Engineering notation restricts \(k\) to multiples of 3, matching the prefixes kilo, mega, milli, micro.

Screen (or device, pixel) coordinates place the origin at the top-left, with \(x\) (column) rightward and \(y\) (row) downward. A pixel ("picture element") is one cell of the grid. World coordinates are the mathematical coordinates of the scene being drawn. A viewport transform converts world to screen coordinates.

A secant line (from Latin secare, to cut) is a line through two points on a curve. Its slope, \(\dfrac{f(a+h) - f(a)}{h}\), is the average rate of change of \(f\) between \(a\) and \(a+h\).

A tangent line (from Latin tangere, to touch) at a point \(P\) is the line through \(P\) whose direction matches the curve’s direction at \(P\). Zoom in far enough on a smooth curve at \(P\) and the curve and its tangent become indistinguishable.

Note: the tangent line is unrelated to the trig function \(\tan\) except by shared etymology.

Differentiating a partial derivative again gives a second partial derivative:

\(f_{xy}\) and \(f_{yx}\) are mixed partials. For every function you will meet in practice (any function whose second partials are continuous), Clairaut’s theorem says they are equal: \(f_{xy} = f_{yx}\). The order of differentiation does not matter.

A line segment \(\overline{AB}\) is the part of a line between endpoints \(A\) and \(B\). Its length is written \(AB\) (no bar). A ray \(\overrightarrow{AB}\) starts at \(A\) and continues forever through \(B\).

The midpoint of \(\overline{AB}\) is the point \(M\) on it with \(AM = MB\). Two segments with equal lengths are congruent, written \(\overline{AB} \cong \overline{CD}\).

A sequence is an ordered list of numbers \(a_1, a_2, a_3, \ldots\). Each number is a term. The subscript is the index (position): \(a_n\) is read "a sub n" and means "the term in position n". The whole sequence is written \((a_n)\) or \(\{a_n\}_{n \ge 1}\). A sequence may be finite (it stops) or infinite (it goes on forever). Formally, an infinite sequence is just a function whose domain is the positive integers: \(a_n\) is another way of writing \(a(n)\).

A series is a sum of the terms of a sequence. The nth partial sum is \(S_n = \sum_{k=1}^{n} a_k\). An arithmetic series sums an arithmetic sequence; a geometric series sums a geometric one. The partial sums \(S_1, S_2, S_3, \ldots\) themselves form a new sequence — this observation is the key to infinite series.

A set is a well-defined collection of distinct objects, called its elements or members. We write \(x \in A\) for "\(x\) is an element of \(A\)" and \(x \notin A\) for "\(x\) is not". Sets are written with braces: \(A = \{2, 3, 5, 7\}\). Order and repetition are irrelevant: \(\{1, 2, 2, 3\} = \{3, 2, 1\}\). Two sets are equal when they have exactly the same elements.

The cardinality \(|A|\) is the number of elements of a finite set: \(|\{2, 3, 5, 7\}| = 4\).

Set-builder notation describes a set by a rule rather than a list:

Read the bar \(\mid\) (sometimes a colon) as "such that". The left part says where elements come from and what shape they have; the right part is a predicate they must satisfy. You can also put an expression on the left: \(\{\, n^2 \mid n \in \mathbb{N} \,\} = \{0, 1, 4, 9, \ldots\}\). Python’s set comprehension {n*n for n in range(10)} is set-builder notation, almost symbol for symbol.

Set-builder notation \(\{x \mid x > 2\}\) is read "the set of all \(x\) such that \(x > 2\)". The vertical bar means "such that" (some books use a colon). \(x \in S\) means "\(x\) is an element of the set \(S\)"; for example \(3 \in (2, \infty)\). The union \(A \cup B\) is everything in \(A\) or \(B\) (or both); the intersection \(A \cap B\) is everything in both. These are developed properly in Chapter 21.

\(\displaystyle\sum_{i=m}^{n} f(i) = f(m) + f(m+1) + \cdots + f(n)\). The index \(i\) is a dummy variable; \(m\) and \(n\) are the lower and upper bounds (inclusive); \(f(i)\) is the summand. If \(n < m\) the sum is empty and equals 0. The number of terms is \(n - m + 1\).

The capital Greek letter \(\Sigma\) (sigma, for "sum") abbreviates a long addition:

Piece by piece: \(i\) is the index (a loop counter); \(i = 1\) underneath is where it starts; \(n\) on top is where it stops (inclusive); the expression to the right is the term evaluated for each \(i\) and then added. In code it is sum(f(x_star[i]) * dx for i in range(1, n + 1)).

The symbol \(\Sigma\) (capital Greek sigma, for "sum") abbreviates long sums:

Read it as "the sum, for \(i\) from 1 to \(n\), of \(a_i\)". The index \(i\) is a dummy variable (like a loop variable: it has no meaning outside the sum), the numbers below and above \(\Sigma\) are its starting and ending values, and \(a_i\) is the term, an expression in \(i\). For example \(\sum_{i=1}^{4} (2i - 1) = 1 + 3 + 5 + 7 = 16\). In Python: sum(2*i - 1 for i in range(1, n + 1)). A key identity for induction is "peel off the last term":

Sigma notation is covered in more depth in Chapter 14.

A floating-point number has the form \(\pm\, m\times\beta^{e}\), where \(\beta\) is the base (2 in hardware), \(m\) is the significand (also called the mantissa) holding a fixed number of significant digits, and \(e\) is the integer exponent. In base 2 the significand of a nonzero number can always be scaled so that it looks like \(1.f_1f_2f_3\ldots\) in binary; this is the normalised form, and the bits \(f_1f_2\ldots\) are the fraction.

The significant figures (s.f., or significant digits) of a measured number are the digits that carry real information about its precision. Rules: non-zero digits are significant; zeros between non-zero digits are significant (\(4007\): 4 s.f.); leading zeros are not (\(0.0072\): 2 s.f.); trailing zeros after a decimal point are (\(2.50\): 3 s.f.); trailing zeros in a whole number are ambiguous (\(1500\): 2, 3, or 4?) — scientific notation removes the ambiguity: \(1.5 \times 10^3\) vs \(1.500 \times 10^3\).

\(\triangle ABC \sim \triangle DEF\) ("is similar to") means corresponding angles are equal and corresponding sides are in a common ratio:

where \(k\) is the scale factor from the first triangle to the second. \(k > 1\) is an enlargement, \(0 < k < 1\) a reduction. Congruence is the special case \(k = 1\).

A proportion is an equation stating two ratios are equal, such as \(\tfrac{a}{b} = \tfrac{c}{d}\). Its solution method, cross-multiplying, gives \(ad = bc\).

Simple harmonic motion is oscillation described by \(x(t) = A\cos(\omega t + \varphi)\).

The period of SHM doesn’t depend on the amplitude. A small bounce and a big bounce take the same time.

To simplify an expression is to rewrite it in an equivalent form that is shorter or more useful, typically with brackets removed, like terms collected, and terms written in a standard order (usually decreasing powers: \(2x^2 + 7x - 4\)). "Simplest" depends on purpose: sometimes the factored form is the more useful one.

For an acute angle \(\theta\) in a right triangle:

The mnemonic is SOH-CAH-TOA: Sine = Opposite/Hypotenuse, Cosine = Adjacent/Hypotenuse, Tangent = Opposite/Adjacent.

Dividing the first by the second gives \(\tan\theta = \dfrac{\sin\theta}{\cos\theta}\).

Notation: \(\sin\theta\) means "sine of \(\theta\)". The brackets are optional for a single symbol but needed for anything longer: \(\sin(2\theta + 1)\). \(\sin^2\theta\) means \((\sin\theta)^2\), not \(\sin(\sin\theta)\).

Every real \(m\times n\) matrix can be written \(A = U\Sigma V^\mathsf{T}\) with \(U\), \(V\) orthogonal and \(\Sigma\) diagonal with non-negative entries \(\sigma_1 \ge \sigma_2 \ge \cdots\) (the singular values). The singular values are the square roots of the eigenvalues of the symmetric matrix \(A^\mathsf{T}A\); the columns of \(V\) are its eigenvectors. The number of nonzero singular values is the rank of \(A\). For a symmetric matrix with non-negative eigenvalues, the SVD and the eigendecomposition coincide.

For \(y = A\sin(B(x - C)) + D\) (with \(A, B > 0\)):

The slope (or gradient) \(m\) of a non-vertical line is \(\frac{\Delta y}{\Delta x}\) between any two of its points — the change in \(y\) per unit increase in \(x\). Positive slope rises left-to-right, negative slope falls, zero slope is horizontal, and a vertical line has undefined slope (\(\Delta x = 0\), so you would divide by zero).

In \(y = mx + b\) (the slope–intercept form):

The slope has units: units of \(y\) per unit of \(x\).

The inverse of a product reverses the order: \((AB)^{-1} = B^{-1}A^{-1}\). To undo "socks, then shoes" you remove shoes first, then socks. Check: \((AB)(B^{-1}A^{-1}) = A(BB^{-1})A^{-1} = AIA^{-1} = AA^{-1} = I\). Also \((A^{-1})^{-1} = A\) and \(\det(A^{-1}) = 1/\det A\) (if \(A\) multiplies areas by 3, undoing it divides them by 3).

A solution of a system is an assignment of values to all the unknowns that makes every equation true. For two unknowns \(x, y\) a solution is an ordered pair \((x, y)\); for three it’s an ordered triple \((x, y, z)\). The solution set is the set of all solutions. It may contain exactly one element, none, or infinitely many.

A solution of an equation is a value of the unknown that makes it true. For an equation of the form "expression \(= 0\)", a solution is also called a root. The solution set is the collection of all solutions, written with braces: the solution set of \(x^2 = 9\) is \(\{-3, 3\}\). An equation may have one solution, several, infinitely many, or none (solution set \(\varnothing\), the empty set).

A solution of an equation is a value of the unknown that makes the equation a true statement. For equations of the form \(f(x) = 0\), a solution is also called a root. The solution set is the collection of all solutions. For a linear equation it contains exactly one number, no numbers, or every number (Section 5.4). For an inequality it is usually a whole interval.

The \(\pm\) ("plus or minus") and \(\mp\) ("minus or plus") signs mean: read all top signs together, or all bottom signs together.

If \(A\) is a real symmetric \(n\times n\) matrix, then:

Geometrically: every symmetric matrix is a pure stretch along some set of perpendicular axes. Rotate to those axes (\(Q^\mathsf{T}\)), stretch (\(\Lambda\)), rotate back (\(Q\)). No shear, no rotation.

A square root of \(a\) is a number whose square is \(a\). Every positive number has two (\(3\) and \(-3\) are both square roots of 9). The symbol \(\sqrt{a}\) — the radical sign over the radicand \(a\) — means the principal (non-negative) square root: \(\sqrt{9} = 3\). A cube root \(\sqrt[3]{a}\) satisfies \((\sqrt[3]{a})^3 = a\); every real number has exactly one real cube root (\(\sqrt[3]{-8} = -2\)). In general \(\sqrt[n]{a}\) is the nth root; the small \(n\) is the index of the root.

\(a^2\) is read "\(a\) squared" because it is the area of a square with side \(a\); \(a^3\) is "\(a\) cubed", the volume of a cube with edge \(a\). A perfect square is the square of an integer (1, 4, 9, 16, 25, …); a perfect cube is the cube of one (1, 8, 27, 64, …).

Squeeze theorem (also sandwich or pinching theorem): if \(g(x) \le f(x) \le h(x)\) for all \(x\) near \(a\) (except possibly at \(a\)), and \(\lim_{x\to a} g(x) = \lim_{x\to a} h(x) = L\), then \(\lim_{x\to a} f(x) = L\) as well.

(Angles in radians. The \(\arctan\) entry is derived in the next section.)

The standard error (SE) of a statistic is the standard deviation of its sampling distribution: how much the statistic would vary from sample to sample.

Standard deviation describes the spread of individual data points; standard error describes the uncertainty of an estimate. Collecting more data doesn’t change the first and shrinks the second like \(1/\sqrt n\).

A polynomial is in standard form when like terms are combined and the terms are written in decreasing order of degree: \(3x^4 - x^2 + 5x - 8\), not \(5x - 8 + 3x^4 - x^2\).

\(\mathbb{N}\) natural numbers \(\{0, 1, 2, \ldots\}\) (some authors start at 1; computer scientists usually start at 0); \(\mathbb{Z}\) integers (from German Zahlen, "numbers"); \(\mathbb{Q}\) rationals (quotient); \(\mathbb{R}\) reals; \(\mathbb{C}\) complex numbers. See Chapter 1.

The subject of a formula is the variable standing alone on one side, usually the left: \(F\) is the subject of \(F = \tfrac{9}{5}C + 32\). Changing the subject (also rearranging, transposing, or solving for a variable) means producing an equivalent formula with a different subject. An inverse operation undoes another: addition and subtraction are inverses, as are multiplication and division (by a non-zero number), and squaring and taking the square root (with care about signs).

The result of an addition is a sum; its inputs are terms (or addends). The result of a subtraction is a difference. The result of a multiplication is a product; its inputs are factors. The result of a division is a quotient: in \(a \div b\), \(a\) is the dividend and \(b\) the divisor. Multiplication may be written \(a \times b\), \(a \cdot b\), or simply \(ab\) when at least one factor is a letter.

Sum rule. If a choice can be made from pool \(A\) or from pool \(B\), and no option is in both (the pools are disjoint, \(A \cap B = \varnothing\)), the number of choices is \(|A| + |B|\).

Product rule. If a process has two stages, the first can be done in \(m\) ways, and for each of those the second can be done in \(n\) ways, the whole process can be done in \(m \times n\) ways. It extends to any number of stages: \(n_1 \times n_2 \times \cdots \times n_k\).

Rule of thumb: "or" between disjoint cases → add; "and then" between stages → multiply.

A surd (UK usage) or radical (US usage) is an irrational root such as \(\sqrt{2}\) or \(\sqrt[3]{5}\), left in exact form rather than as a decimal. Simplest radical form means: no perfect-square factor under a square root, no root in a denominator, and no fraction under a root. Like surds have the same radicand (\(3\sqrt{2}\) and \(5\sqrt{2}\)) and can be added like like terms.

The graph (or surface) of \(f(x, y)\) is the set of points \((x, y, z)\) with \(z = f(x, y)\).

A level curve of \(f\) at level \(c\) is the set of points in the plane where \(f(x, y) = c\). A contour plot draws several level curves at evenly spaced levels. With three inputs, the same idea gives level surfaces \(f(x, y, z) = c\).

A system of equations is a collection of equations that must all hold at the same time, for the same values of the unknowns. A brace on the left is the usual way to show they belong together:

British courses call these simultaneous equations, a name that stresses the "at the same time" part. US courses say systems of equations. They mean the same thing.

The degree-\(n\) Taylor polynomial of \(f\) about \(a\) is \(T_n(x) = \sum_{k=0}^{n}\frac{f^{(k)}(a)}{k!}(x - a)^k\), the unique polynomial of degree at most \(n\) matching \(f\) and its first \(n\) derivatives at \(a\). Notation: \(f^{(k)}\) is the \(k\)-th derivative (brackets distinguish it from a power), with \(f^{(0)} = f\). Letting \(n \to \infty\) gives the Taylor series.

The terms of an expression are the pieces that are added together. In \(7x^2 - 3x + 4\) the terms are \(7x^2\), \(-3x\), and \(4\). Subtraction is read as adding a negative, so the minus sign travels with the term that follows it. Inside brackets things are grouped: \(3(x + 2) - 4y\) has two terms at the top level, \(3(x + 2)\) and \(-4y\).

A term is one of the pieces being added in an expression. In \(7x^2 - 3x + 4\) the terms are \(7x^2\), \(-3x\) and \(4\). The sign belongs to the term: the middle term is \(-3x\), not \(3x\).

3D Cartesian coordinates \((x, y, z)\) locate a point in space by signed distances along three mutually perpendicular axes. The system is right-handed when x, y, z follow the right-hand rule (the standard convention in mathematics and physics). Space, thought of as all such triples, is written \(\mathbb{R}^3\) ("R three"); the plane is \(\mathbb{R}^2\).

A transformation of the plane is a rule that sends every point to a new point. The image of \(A\) is written \(A'\) ("A prime"). An isometry or rigid motion preserves all distances (and hence angles and areas): translations, reflections and rotations are isometries, and they produce congruent figures. A dilation scales all distances by the same factor and produces similar figures.

A tree is a connected graph with no cycles. A forest is a graph with no cycles (a disjoint union of trees). A leaf is a vertex of degree 1. A rooted tree singles out one vertex as the root; then every other vertex has a unique parent (its neighbour on the path to the root) and possibly children. The depth of a vertex is its distance from the root. A spanning tree of a graph \(G\) is a tree that includes every vertex of \(G\) and uses only edges of \(G\).

The numbers \(T_n = 1 + 2 + \cdots + n = \frac{n(n+1)}{2}\) (1, 3, 6, 10, 15, …) are the triangular numbers, because that many dots can be arranged in a triangle. Note \(T_n = \binom{n+1}{2}\), the number of ways to choose 2 items from \(n+1\) (Chapter 24).

An unsigned \(n\)-bit integer represents \(0\) to \(2^n - 1\). A signed \(n\)-bit integer in two’s complement represents \(-2^{n-1}\) to \(2^{n-1} - 1\); the top bit is the sign bit (1 = negative). Overflow is when the true mathematical result of an operation lies outside the representable range, so the stored result wraps around. For 32 bits the signed range is \(-2\,147\,483\,648\) to \(2\,147\,483\,647\).

Power increases with larger samples, larger true effects, less noisy data, and a larger \(\alpha\). For a fixed sample size, lowering \(\alpha\) (fewer false alarms) raises \(\beta\) (more misses).

An ulp (unit in the last place) of a floating-point number \(x\) is the gap between \(x\) and the next representable number of larger magnitude. \(\mathrm{ulp}(1)=2^{-52}\); \(\mathrm{ulp}(10^{16}) = 2\). Errors are often quoted in ulps: "correctly rounded" means within half an ulp.

For sets \(A, B\) inside a universal set \(U\):

Sets with no common element, \(A \cap B = \varnothing\), are disjoint.

The union \(A \cup B\) is the set of elements in \(A\) or \(B\) (or both). The intersection \(A \cap B\) is the set of elements in both. A Venn diagram draws sets as overlapping regions so that each region represents one combination of "in" and "out". See Chapter 21 for the full set vocabulary.

Rotate the point \((1,0)\) anticlockwise about the origin by angle \(\theta\) (clockwise if \(\theta\) is negative). The point lands at \(P\). Then

For an angle \(\theta\) in standard position, the initial side is the positive \(x\)-axis and the terminal side is the ray \(OP\). The four quadrants are numbered I to IV anticlockwise, starting from the top right.

An unknown is a variable whose value we are trying to find from an equation. In \(3x + 4 = 19\), \(x\) is the unknown, and the equation pins it down to \(x = 5\). "Unknown" describes a variable’s job in a problem rather than a different kind of object.

Substitution (or u-substitution, or change of variables) rewrites an integral in terms of a new variable \(u = g(x)\), replacing \(g'(x)\,dx\) by \(du\). It is the chain rule run backwards. It works when the integrand contains an inner function and (up to a constant factor) that inner function’s derivative.

A conditional whose hypothesis is false is vacuously true: true because there is no case that could break it. "Every unicorn in this room is purple" is vacuously true, since there are no unicorns to check. In code, all(x > 100 for x in []) returns True for the same reason.

A variable is a symbol, usually a letter, that stands for a number we either do not know yet or want to leave unspecified. In \(2n + 10\), \(n\) is the variable. The word comes from Latin variabilis, "changeable".

The variance of \(X\), with \(\mu = E[X]\), is the expected squared deviation from the mean:

It is often written \(\sigma^2\) (sigma squared). The standard deviation is its square root, \(\sigma = \sqrt{\operatorname{Var}(X)}\), which has the same units as \(X\) (milliseconds rather than milliseconds squared), so it’s the one to quote.

Rough meaning: \(\sigma\) is a typical distance between \(X\) and its mean.

For vectors \(\mathbf{a}, \mathbf{b}, \mathbf{c}\) in \(\mathbb{R}^n\) and scalars \(s, t\):

Each follows immediately from the matching law for ordinary numbers, applied one component at a time. Any collection of objects obeying these rules is called a vector space; that abstraction is why polynomials, audio signals and images can all be treated as vectors.

A vector in \(\mathbb{R}^2\) is an ordered pair \(\mathbf{v} = (v_1, v_2)\), pictured as an arrow from the origin to the point \((v_1, v_2)\). It has a direction and a length. The same idea works with \(n\) components.

Its length (or magnitude, or norm) is \(|\mathbf{v}| = \sqrt{v_1^2 + v_2^2}\) (Pythagoras; see Chapter 10).

A unit vector has length 1 and is used purely to indicate a direction. Any nonzero vector can be turned into one by dividing by its length: \(\mathbf{u} = \mathbf{v}/|\mathbf{v}|\). For example, \((3, 4)\) has length 5, so \((0.6,\, 0.8)\) is the unit vector in that direction.

In this book, vectors are bold. We write them as tuples in running text to save space and as columns inside calculations, especially once matrices arrive in Chapter 27.

The vertex of a parabola is its turning point: the lowest point if \(a > 0\) (a minimum), the highest if \(a < 0\) (a maximum). For \(y = ax^2 + bx + c\) it is at \(x = -\dfrac{b}{2a}\). The \(y\)-value there is the minimum value or maximum value of the quadratic.

For \(ax^2 + bx + c = 0\) with roots \(r_1, r_2\) (real or complex, counted with multiplicity):

For a monic quadratic \(x^2 + bx + c\): the roots sum to \(-b\) and multiply to \(c\). This is exactly why factoring \(x^2 + bx + c\) means "find two numbers with sum \(b\) and product \(c\)" (the numbers are the negatives of the roots).

For any \(a \neq 0\): \(a^0 = 1\) (the empty product), and \(a^{-n} = \dfrac{1}{a^n}\). A negative exponent means "reciprocal", not "negative": \(2^{-3} = \tfrac{1}{8}\), which is positive. A negative exponent on a fraction flips it: \(\left(\tfrac{a}{b}\right)^{-n} = \left(\tfrac{b}{a}\right)^n\).

If \(AB = 0\) then \(A = 0\) or \(B = 0\) (or both). This is why factoring solves equations. It works only with zero on the right: \(AB = 6\) tells you nothing about \(A\) or \(B\) individually.