Appendix C: Notation and Symbols
Mathematical notation is a compressed language. This appendix is its phrasebook: for each symbol, how to say it aloud, what it means, a tiny example, and the chapter where it is properly introduced. Saying a formula aloud is half of understanding it, so the "Read aloud" column matters more than it looks.
Where a symbol has more than one common meaning, all are listed; context decides. The section Section C.13 collects the worst offenders, and Section C.14 shows how to take a dense formula apart.
C.1. Arithmetic, comparison, and approximation
| Symbol | Read aloud | Meaning | Example | Ch. |
|---|---|---|---|---|
\(=\) |
"equals" |
Both sides name the same value |
\(2 + 3 = 5\) |
|
\(\ne\) |
"is not equal to" |
The two sides differ |
\(0.1 + 0.2 \ne 0.3\) in floating point |
|
\(<,\ >\) |
"is less than", "is greater than" |
Strict order; the small end points at the smaller number |
\(-3 < 2\) |
|
\(\le,\ \ge\) |
"is less than or equal to", "is greater than or equal to" |
Order allowing equality |
\(x \ge 0\) |
|
\(\approx\) |
"is approximately equal to" |
Close in value; how close depends on context |
\(\pi \approx 3.1416\) |
|
\(\equiv\) |
"is identically equal to"; "is congruent to"; "is logically equivalent to" |
(1) true for every value of the variable; (2) same remainder modulo \(m\); (3) same truth value in every case |
\((x+1)^2 \equiv x^2 + 2x + 1\); \(17 \equiv 2 \pmod{5}\) |
|
\(:=\) |
"is defined to be" |
Defines the left side; not a claim to check |
\(f(x) := x^2\) |
|
\(\propto\) |
"is proportional to" |
Equal up to a constant multiplier |
\(A \propto r^2\) for circles (constant \(\pi\)) |
|
\(\ll,\ \gg\) |
"is much less than", "is much greater than" |
Smaller (larger) by orders of magnitude |
\(10^{-16} \ll 1\) |
|
\(\pm,\ \mp\) |
"plus or minus", "minus or plus" |
Two cases at once; \(\mp\) takes the opposite sign to a \(\pm\) in the same formula |
\(x = 3 \pm 2\) means \(x = 5\) or \(x = 1\) |
|
\(\times,\ \cdot\) |
"times" |
Multiplication (also cross product and dot product for vectors, Cartesian product for sets) |
\(3 \times 4 = 3 \cdot 4 = 12\) |
|
\(a / b,\ \tfrac{a}{b},\ a \div b\) |
"a over b", "a divided by b" |
Division; a fraction |
\(\tfrac{6}{4} = 1.5\) |
|
\(a : b\) |
"a to b" |
Ratio |
\(3 : 2\) |
|
\(\%\) |
"percent" |
Per hundred: \(p\% = p/100\) |
\(15\% = 0.15\) |
|
\(a^n\) |
"a to the n", "a to the power n" (\(a^2\) "a squared", \(a^3\) "a cubed") |
Repeated multiplication, extended to all real \(n\) |
\(2^{-1} = \tfrac{1}{2}\) |
|
\(\sqrt{a},\ \sqrt[n]{a}\) |
"root a", "the n-th root of a" |
The non-negative number whose square (\(n\)-th power) is \(a\) |
\(\sqrt[3]{8} = 2\) |
|
\(\lvert x \rvert\) |
"the absolute value of x", "mod x" |
Distance from 0 |
\(\lvert -7 \rvert = 7\) |
|
\(\lfloor x \rfloor\) |
"floor of x" |
Largest integer \(\le x\) |
\(\lfloor 2.7 \rfloor = 2\), \(\lfloor -2.7 \rfloor = -3\) |
|
\(\lceil x \rceil\) |
"ceiling of x" |
Smallest integer \(\ge x\) |
\(\lceil 2.1 \rceil = 3\) |
|
\(n!\) |
"n factorial" |
\(1 \cdot 2 \cdots n\), with \(0! = 1\) |
\(4! = 24\) |
|
\(\infty\) |
"infinity" |
Not a number: shorthand for "without bound" |
\(x \to \infty\), \((0, \infty)\) |
|
\(\ldots,\ \cdots\) |
"dot dot dot", "and so on" |
Continue the evident pattern |
\(1 + 2 + \cdots + n\) |
|
\(\therefore\) |
"therefore" |
Introduces a conclusion |
\(2x = 6 \therefore x = 3\) |
|
\(\blacksquare\) or \(\square\) |
"QED", "end of proof" |
The proof is complete |
C.2. Number systems and intervals
| Symbol | Read aloud | Meaning | Example | Ch. |
|---|---|---|---|---|
\(\mathbb{N}\) |
"the naturals", "N" |
Natural numbers \(\{0, 1, 2, \ldots\}\) (some authors start at 1) |
\(5 \in \mathbb{N}\) |
|
\(\mathbb{Z}\) |
"the integers", "Z" (from German Zahlen) |
\(\{\ldots, -2, -1, 0, 1, 2, \ldots\}\) |
\(-4 \in \mathbb{Z}\) |
|
\(\mathbb{Q}\) |
"the rationals", "Q" (for quotient) |
Fractions \(p/q\) with \(p, q\) integers, \(q \ne 0\) |
\(0.75 = \tfrac{3}{4} \in \mathbb{Q}\) |
|
\(\mathbb{R}\) |
"the reals", "R" |
All points on the number line |
\(\sqrt{2} \in \mathbb{R}\), \(\sqrt{2} \notin \mathbb{Q}\) |
|
\(\mathbb{C}\) |
"the complex numbers", "C" |
Numbers \(a + bi\) with \(a, b \in \mathbb{R}\) |
\(3 - 2i \in \mathbb{C}\) |
|
\(\mathbb{R}^n\) |
"R n" |
Lists (vectors) of \(n\) real numbers |
\((1, -2, 0.5) \in \mathbb{R}^3\) |
|
\(\mathbb{Z}_m\) or \(\mathbb{Z}/m\mathbb{Z}\) |
"the integers mod m" |
The remainders \(\{0, 1, \ldots, m - 1\}\) with modular arithmetic |
In \(\mathbb{Z}_{12}\), \(9 + 5 = 2\) |
| Notation | Means \(x\) satisfies | Endpoints |
|---|---|---|
\((a, b)\) |
\(a < x < b\) |
both excluded ("open") |
\(\lbrack a, b \rbrack\) |
\(a \le x \le b\) |
both included ("closed") |
\(\lbrack a, b)\) |
\(a \le x < b\) |
left included, right excluded ("half-open"; Python’s |
\((a, b \rbrack\) |
\(a < x \le b\) |
left excluded, right included |
\((a, \infty)\) |
\(x > a\) |
\(\infty\) is never included |
\((-\infty, b \rbrack\) |
\(x \le b\) |
|
\((-\infty, \infty)\) |
any real \(x\) |
the whole of \(\mathbb{R}\) |
A round bracket means "up to but not including"; a square bracket means "including". Some European texts write \(\rbrack a, b \lbrack\) for \((a, b)\). Do not confuse the open interval \((a, b)\) with the point \((a, b)\) in the plane; context tells them apart. See Chapter 5.
C.3. Sets
| Symbol | Read aloud | Meaning | Example | Ch. |
|---|---|---|---|---|
\(\{\ \}\) |
"the set containing …" |
Unordered collection, no repeats |
\(\{1, 2, 3\} = \{3, 1, 2\}\) |
|
\(\{x \mid P(x)\}\) or \(\{x : P(x)\}\) |
"the set of x such that P of x" |
Set-builder notation: everything satisfying a condition |
\(\{x \in \mathbb{Z} \mid x^2 < 5\} = \{-2, \ldots, 2\}\) |
|
\(\in\) |
"is in", "is an element of", "belongs to" |
Membership |
\(3 \in \{1, 2, 3\}\) |
|
\(\notin\) |
"is not in" |
Non-membership |
\(4 \notin \{1, 2, 3\}\) |
|
\(\subseteq\) |
"is a subset of" |
Every element of the left is in the right |
\(\{1, 2\} \subseteq \{1, 2, 3\}\) |
|
\(\subset\) or \(\subsetneq\) |
"is a proper subset of" |
Subset but not equal (some books use \(\subset\) for \(\subseteq\); check the convention) |
\(\mathbb{N} \subset \mathbb{Z}\) |
|
\(\cup\) |
"union", "A cup B" |
In either set (or both) |
\(\{1, 2\} \cup \{2, 3\} = \{1, 2, 3\}\) |
|
\(\cap\) |
"intersection", "A cap B" |
In both sets |
\(\{1, 2\} \cap \{2, 3\} = \{2\}\) |
|
\(A \setminus B\) or \(A - B\) |
"A minus B", "A without B" |
In \(A\) but not in \(B\) |
\(\{1, 2, 3\} \setminus \{2\} = \{1, 3\}\) |
|
\(A^c\), \(\overline{A}\), \(A'\) |
"A complement" |
Everything (in the universe) not in \(A\) |
In \(\{1, \ldots, 6\}\), \(\{1, 2\}^c = \{3, 4, 5, 6\}\) |
|
\(\emptyset\) or \(\varnothing\) |
"the empty set" |
The set with no elements |
\(\{1\} \cap \{2\} = \emptyset\) |
|
\(\lvert A \rvert\) or \(\#A\) |
"the size of A", "the cardinality of A" |
Number of elements |
\(\lvert \{a, b, c\} \rvert = 3\) |
|
\(A \times B\) |
"A cross B" |
Cartesian product: all ordered pairs \((a, b)\) |
\(\{1, 2\} \times \{x\} = \{(1, x), (2, x)\}\) |
|
\(\mathcal{P}(A)\) |
"the power set of A" |
The set of all subsets of \(A\) |
\(\mathcal{P}(\{1, 2\}) = \{\emptyset, \{1\}, \{2\}, \{1, 2\}\}\) |
C.4. Logic
| Symbol | Read aloud | Meaning | Example | Ch. |
|---|---|---|---|---|
\(\neg p\) |
"not p" |
True exactly when \(p\) is false (code: |
\(\neg(2 > 3)\) is true |
|
\(p \land q\) |
"p and q" |
True when both are true (code: |
\(x > 0 \land x < 1\) |
|
\(p \lor q\) |
"p or q" |
True when at least one is true; inclusive or (code: |
\(x < 0 \lor x > 1\) |
|
\(p \oplus q\) |
"p xor q", "exclusive or" |
True when exactly one is true (code: |
\(\mathbf{T} \oplus \mathbf{T}\) is false |
|
\(p \to q\) |
"if p then q", "p implies q" |
The conditional: false only when \(p\) is true and \(q\) false |
\(x > 2 \to x > 1\) |
|
\(p \leftrightarrow q\) |
"p if and only if q", "p iff q" |
Same truth value |
\(n \text{ even} \leftrightarrow n^2 \text{ even}\) |
|
\(\Rightarrow\) |
"implies" |
Used between statements in an argument: "from this, it follows that". Many authors use \(\Rightarrow\) and \(\to\) interchangeably |
\(2x = 6 \Rightarrow x = 3\) |
|
\(\Leftrightarrow\) |
"if and only if", "is equivalent to" |
Each side implies the other; an equation-solving step that can be reversed |
\(x + 1 = 3 \Leftrightarrow x = 2\) |
|
\(\forall\) |
"for all", "for every" |
Universal quantifier |
\(\forall x \in \mathbb{R},\ x^2 \ge 0\) |
|
\(\exists\) |
"there exists", "for some" |
Existential quantifier: at least one |
\(\exists x \in \mathbb{R},\ x^2 = 2\) |
|
\(\exists!\) |
"there exists a unique" |
Exactly one |
\(\exists! x,\ 2x = 6\) |
|
\(\mathbf{T},\ \mathbf{F}\) |
"true", "false" |
Truth values (also \(1, 0\) or \(\top, \bot\)) |
C.5. Functions
| Symbol | Read aloud | Meaning | Example | Ch. |
|---|---|---|---|---|
\(f(x)\) |
"f of x" |
The output of function \(f\) at input \(x\) (not \(f\) times \(x\)) |
\(f(x) = x^2,\ f(3) = 9\) |
|
\(f : A \to B\) |
"f from A to B" |
\(f\) takes inputs in \(A\) (domain) to outputs in \(B\) (codomain); a type signature |
\(\sqrt{\cdot} : \lbrack 0, \infty) \to \mathbb{R}\) |
|
\(x \mapsto f(x)\) |
"x maps to f of x" |
Names a function by its rule (like a lambda) |
\(x \mapsto 2x + 1\) |
|
\(f \circ g\) |
"f composed with g", "f after g", "f of g" |
\((f \circ g)(x) = f(g(x))\): apply \(g\) first |
\(f(x) = x^2,\ g(x) = x + 1 \Rightarrow (f \circ g)(2) = 9\) |
|
\(f^{-1}\) |
"f inverse" |
The function that undoes \(f\). Not \(1/f\) |
\(f(x) = 2x \Rightarrow f^{-1}(x) = x/2\) |
|
\(\sin^{-1} x\), \(\arcsin x\) |
"inverse sine of x", "arc sine of x" |
The angle whose sine is \(x\). But \(\sin^2 x\) means \((\sin x)^2\) |
\(\arcsin \tfrac{1}{2} = \tfrac{\pi}{6}\) |
|
\(\log_b x\), \(\ln x\), \(\log x\) |
"log base b of x", "natural log of x" ("ell-en x", "lin x" or "log x") |
The exponent that gives \(x\). \(\ln\) is base \(e\); bare \(\log\) is base 10 in school, base \(e\) in research papers and in Python, often base 2 in CS |
\(\log_2 8 = 3\) |
|
\(\exp(x)\) |
"exp of x" |
\(e^x\), written so the exponent can be large |
\(\exp(0) = 1\) |
|
\(\Delta x\) |
"delta x" |
A change (difference) in \(x\) |
\(\Delta x = x_2 - x_1\) |
|
\(\left. F(x) \right\rvert_a^b\) |
"F of x evaluated from a to b" |
\(F(b) - F(a)\) |
\(\left. x^2 \right\rvert_1^3 = 9 - 1 = 8\) |
|
\(O(g)\), \(\Theta(g)\), \(\Omega(g)\) |
"big O of g", "big Theta", "big Omega" |
Grows at most / exactly / at least as fast as \(g\), up to a constant factor |
Binary search is \(O(\log n)\) |
C.6. Sequences and sums
| Symbol | Read aloud | Meaning | Example | Ch. |
|---|---|---|---|---|
\(a_n\) |
"a sub n", "a n" |
The \(n\)-th term of a sequence (code: |
\(a_n = 2n \Rightarrow a_3 = 6\) |
|
\((a_n)_{n \ge 1}\) |
"the sequence a n" |
The whole sequence, not one term |
\((1/n)_{n \ge 1}\) |
|
\(\displaystyle\sum_{i=1}^{n} a_i\) |
"the sum from i equals 1 to n of a sub i" |
\(a_1 + a_2 + \cdots + a_n\); a |
\(\sum_{i=1}^{4} i = 10\) |
|
\(\displaystyle\prod_{i=1}^{n} a_i\) |
"the product from i equals 1 to n of a sub i" |
\(a_1 \cdot a_2 \cdots a_n\); the same loop accumulating with \(\times\) |
\(\prod_{i=1}^{4} i = 24\) |
|
\(\displaystyle\binom{n}{k}\) |
"n choose k" |
Number of \(k\)-element subsets of an \(n\)-element set; also written \(C(n, k)\), \({}^nC_k\), \({}_nC_k\), \(nCr\) |
\(\binom{5}{2} = 10\) |
|
\(P(n, r)\) |
"the number of permutations of n things r at a time" |
Ordered selections: \(n!/(n - r)!\); also \({}^nP_r\), \(nPr\) |
\(P(5, 2) = 20\) |
C.7. Calculus
| Symbol | Read aloud | Meaning | Example | Ch. |
|---|---|---|---|---|
\(\displaystyle\lim_{x \to a} f(x)\) |
"the limit as x tends to a of f of x" |
The value \(f(x)\) approaches as \(x\) gets close to (but not equal to) \(a\) |
\(\lim_{x \to 0} \tfrac{\sin x}{x} = 1\) |
|
\(x \to a^+\), \(x \to a^-\) |
"x tends to a from above / below" |
One-sided limits |
\(\lim_{x \to 0^+} \tfrac{1}{x} = \infty\) |
|
\(\varepsilon,\ \delta\) |
"epsilon", "delta" |
Small positive tolerances in the formal definition of a limit |
||
\(f'(x)\) |
"f prime of x" |
The derivative of \(f\) at \(x\) (Lagrange notation) |
\(f(x) = x^3 \Rightarrow f'(x) = 3x^2\) |
|
\(f''(x)\), \(f^{(n)}(x)\) |
"f double prime of x", "the n-th derivative of f" |
Second and \(n\)-th derivatives; brackets show it is not a power |
\((x^3)'' = 6x\) |
|
\(\dfrac{dy}{dx}\) |
"d y d x", "the derivative of y with respect to x" |
Leibniz notation: rate of change of \(y\) per unit \(x\). Not a fraction, though it often behaves like one |
\(y = x^2 \Rightarrow \tfrac{dy}{dx} = 2x\) |
|
\(\dfrac{d}{dx}\) |
"d by d x" |
The operation "differentiate with respect to \(x\)" |
\(\tfrac{d}{dx}\sin x = \cos x\) |
|
\(\dot{x}\) |
"x dot" |
Newton notation: derivative with respect to time (physics, mechanics) |
\(\dot{x} = v\) |
|
\(\dfrac{\partial f}{\partial x}\), \(f_x\), \(\partial_x f\) |
"partial f by partial x", "del f del x", "f sub x" |
Partial derivative: differentiate in \(x\), holding the other variables fixed |
\(f = x^2 y \Rightarrow f_x = 2xy\) |
|
\(\nabla f\) |
"grad f", "del f", "nabla f" |
The gradient: the vector of all partial derivatives |
\(\nabla(x^2 + y^2) = (2x, 2y)\) |
|
\(\displaystyle\int f(x)\,dx\) |
"the integral of f of x d x" |
Indefinite integral: the family of antiderivatives |
\(\int 2x\,dx = x^2 + C\) |
|
\(\displaystyle\int_a^b f(x)\,dx\) |
"the integral from a to b of f of x d x" |
Definite integral: signed area under \(f\) between \(a\) and \(b\) (a number). The \(\int\) is an elongated S, for "sum" |
\(\int_0^1 x\,dx = \tfrac{1}{2}\) |
|
\(dx\) |
"d x" |
Marks the variable of integration; the infinitesimal width of a strip |
C.8. Vectors and matrices
| Symbol | Read aloud | Meaning | Example | Ch. |
|---|---|---|---|---|
\(\mathbf{v}\), \(\vec{v}\), \(\underline{v}\) |
"vector v", "v" |
A vector. Print uses bold; handwriting uses an arrow or underline (the UK schools' squiggle) |
\(\mathbf{v} = (3, 4)\) |
|
\(v_i\) |
"v sub i" |
The \(i\)-th component (code: |
\(v_2 = 4\) |
|
\(\lVert \mathbf{v} \rVert\) |
"the norm of v", "the length of v" |
Length (Euclidean norm unless a subscript says otherwise) |
\(\lVert (3, 4) \rVert = 5\) |
|
\(\hat{\mathbf{v}}\) |
"v hat" |
Unit vector in the direction of \(\mathbf{v}\); \(\hat{\mathbf{i}}, \hat{\mathbf{j}}, \hat{\mathbf{k}}\) are the axis directions |
\(\hat{\mathbf{v}} = (0.6, 0.8)\) |
|
\(\mathbf{u} \cdot \mathbf{v}\) |
"u dot v" |
Dot (scalar, inner) product; also \(\langle \mathbf{u}, \mathbf{v} \rangle\) |
\((1, 2) \cdot (3, 4) = 11\) |
|
\(\mathbf{u} \times \mathbf{v}\) |
"u cross v" |
Cross product (3D only): a vector perpendicular to both |
\(\hat{\mathbf{i}} \times \hat{\mathbf{j}} = \hat{\mathbf{k}}\) |
|
\(\perp\) |
"is perpendicular to", "is orthogonal to" |
At right angles (dot product 0) |
\((1, 0) \perp (0, 1)\) |
|
\(A\), \(a_{ij}\) |
"matrix A", "a i j" |
Matrices are capital letters; \(a_{ij}\) is the entry in row \(i\), column \(j\) |
||
\(A^T\), \(A^\top\), \(A'\) |
"A transpose" |
Rows become columns: \((A^T)_{ij} = a_{ji}\) |
\(\begin{pmatrix} 1 & 2 \end{pmatrix}^T = \begin{pmatrix} 1 \\ 2 \end{pmatrix}\) |
|
\(A^{-1}\) |
"A inverse" |
The matrix with \(AA^{-1} = I\), when it exists |
||
\(I\), \(I_n\) |
"the identity", "I n" |
Square matrix with 1s on the diagonal, 0s elsewhere |
\(I\mathbf{x} = \mathbf{x}\) |
|
\(\det A\), \(\lvert A \rvert\) |
"the determinant of A", "det A" |
Signed area/volume scale factor of \(A\) |
\(\det\begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix} = 6\) |
|
\(\operatorname{tr} A\) |
"the trace of A" |
Sum of the diagonal entries |
\(\operatorname{tr} I_3 = 3\) |
|
\(\lambda\) |
"lambda" |
An eigenvalue: \(A\mathbf{v} = \lambda\mathbf{v}\) |
C.9. Probability and statistics
| Symbol | Read aloud | Meaning | Example | Ch. |
|---|---|---|---|---|
\(\Omega\) |
"omega", "the sample space" |
The set of all possible outcomes |
Coin: \(\Omega = \{H, T\}\) |
|
\(P(A)\), \(\Pr(A)\) |
"the probability of A" |
A number in \(\lbrack 0, 1 \rbrack\) |
\(P(\text{even die}) = \tfrac{1}{2}\) |
|
\(P(A \mid B)\) |
"the probability of A given B" |
Probability of \(A\) once you know \(B\) happened |
\(P(\text{six} \mid \text{even}) = \tfrac{1}{3}\) |
|
\(X\), \(x\) |
"capital X", "little x" |
Capital: a random variable; lower case: a particular value it takes |
\(P(X = x)\) |
|
\(X \sim D\) |
"X is distributed as D", "X follows D" |
Names the distribution of \(X\) |
\(X \sim N(0, 1)\) |
|
\(E[X]\), \(\mathbb{E}[X]\) |
"the expected value of X", "the expectation of X" |
Probability-weighted average |
Fair die: \(E[X] = 3.5\) |
|
\(\operatorname{Var}(X)\) |
"the variance of X" |
\(E[(X - \mu)^2]\): average squared distance from the mean |
Fair coin (0/1): \(0.25\) |
|
\(\mu\) |
"mu" |
Population (true) mean |
||
\(\sigma\), \(\sigma^2\) |
"sigma", "sigma squared" |
Population standard deviation, variance |
||
\(\operatorname{Cov}(X, Y)\), \(\rho\) |
"the covariance of X and Y", "rho" |
How two variables vary together; \(\rho\) is the correlation |
||
\(\bar{x}\) |
"x bar" |
Sample mean |
\(\overline{\{2, 4, 9\}} = 5\) |
|
\(s\), \(s^2\) |
"s", "s squared" |
Sample standard deviation, sample variance |
||
\(\hat{p}\), \(\hat{y}\), \(\hat{\theta}\) |
"p hat", "y hat", "theta hat" |
An estimate or prediction of the un-hatted quantity |
45 of 100 say yes: \(\hat{p} = 0.45\) |
|
\(H_0\), \(H_1\) |
"H nought", "H one" |
Null and alternative hypotheses |
\(H_0: \mu = 100\) |
|
\(\alpha\) |
"alpha" |
Significance level of a test |
\(\alpha = 0.05\) |
|
\(z^*\), \(t^*\) |
"z star", "t star" |
Critical values for confidence intervals |
\(z^* = 1.96\) for 95% |
C.10. Number theory and complex numbers
| Symbol | Read aloud | Meaning | Example | Ch. |
|---|---|---|---|---|
\(a \mid b\) |
"a divides b" |
\(b\) is a multiple of \(a\) (a statement, not a number) |
\(3 \mid 12\), \(5 \nmid 12\) |
|
\(a \bmod m\) |
"a mod m" |
The remainder on dividing \(a\) by \(m\) (code: |
\(17 \bmod 5 = 2\) |
|
\(a \equiv b \pmod{m}\) |
"a is congruent to b mod m" |
\(m \mid (a - b)\): same remainder |
\(38 \equiv 2 \pmod{12}\) |
|
\(\gcd(a, b)\), \(\operatorname{lcm}(a, b)\) |
"g c d of a and b", "l c m" |
Greatest common divisor (UK: HCF), least common multiple |
\(\gcd(12, 18) = 6\) |
|
\(\varphi(n)\) |
"phi of n", "Euler’s totient of n" |
How many of \(1, \ldots, n\) are coprime to \(n\) |
\(\varphi(10) = 4\) |
|
\(i\) |
"i" |
\(\sqrt{-1}\); engineers write \(j\), as does Python ( |
\(i^2 = -1\) |
|
\(\bar{z}\), \(z^*\) |
"z bar", "the conjugate of z" |
Flip the sign of the imaginary part |
\(\overline{3 + 4i} = 3 - 4i\) |
|
\(\lvert z \rvert\), \(\arg z\) |
"mod z", "arg z" |
Modulus (length) and argument (angle) of \(z\) |
\(\lvert 3 + 4i \rvert = 5\) |
|
\(\operatorname{Re} z\), \(\operatorname{Im} z\) |
"real part of z", "imaginary part of z" |
\(a\) and \(b\) in \(z = a + bi\) |
\(\operatorname{Im}(3 + 4i) = 4\) |
C.11. Optimisation and machine learning
| Symbol | Read aloud | Meaning | Example | Ch. |
|---|---|---|---|---|
\(\min_x f(x)\), \(\max_x f(x)\) |
"the minimum over x of f of x" |
The smallest (largest) value of \(f\) |
\(\min_x (x - 3)^2 + 1 = 1\) |
|
\(\operatorname{argmin}_x f(x)\), \(\operatorname{argmax}_x f(x)\) |
"the arg min over x of f of x" |
The input \(x\) that achieves the minimum (maximum) |
\(\operatorname{argmin}_x (x - 3)^2 + 1 = 3\) |
|
\(\theta^*\), \(x^*\) |
"theta star" |
The optimal value of a parameter |
||
\(\eta\) |
"eta" |
Learning rate (step size) in gradient descent |
\(\eta = 0.01\) |
|
\(\leftarrow\) |
"becomes", "gets" |
Assignment in pseudocode (code: |
\(x \leftarrow x - \eta f'(x)\) |
C.12. The Greek alphabet
Greek letters are used because the Latin ones run out, and certain letters have settled into conventional jobs. Knowing the conventions lets you guess what a symbol means before you’re told.
| Lower | Upper | Name | Common uses |
|---|---|---|---|
\(\alpha\) |
\(A\) |
alpha |
Angles; significance level in statistics; coefficients; sometimes learning rate |
\(\beta\) |
\(B\) |
beta |
Angles; regression coefficients; Beta distribution; momentum in optimisation |
\(\gamma\) |
\(\Gamma\) |
gamma |
Angles; Euler-Mascheroni constant \(\gamma \approx 0.5772\); discount factor in reinforcement learning. \(\Gamma\): gamma function (\(\Gamma(n) = (n - 1)!\)) |
\(\delta\) |
\(\Delta\) |
delta |
\(\delta\): small quantity (\(\varepsilon\)-\(\delta\) limits), Kronecker delta. \(\Delta\): change in (\(\Delta x\)), discriminant |
\(\varepsilon\), \(\epsilon\) |
\(E\) |
epsilon |
Small positive quantity; error or noise term; machine epsilon |
\(\zeta\) |
\(Z\) |
zeta |
Riemann zeta function \(\zeta(s) = \sum 1/n^s\); damping ratio |
\(\eta\) |
\(H\) |
eta |
Learning rate; efficiency |
\(\theta\), \(\vartheta\) |
\(\Theta\) |
theta |
Angles; model parameters. \(\Theta\): asymptotically tight bound in algorithm analysis |
\(\iota\) |
\(I\) |
iota |
Rare in mathematics; integer ranges in APL and Go |
\(\kappa\) |
\(K\) |
kappa |
Curvature; condition number of a matrix; Cohen’s kappa |
\(\lambda\) |
\(\Lambda\) |
lambda |
Eigenvalues; rate of a Poisson or exponential distribution; Lagrange multiplier; wavelength; anonymous functions (lambda calculus). \(\Lambda\): diagonal matrix of eigenvalues |
\(\mu\) |
\(M\) |
mu |
Mean; micro- (\(10^{-6}\)); coefficient of friction |
\(\nu\) |
\(N\) |
nu |
Degrees of freedom; frequency |
\(\xi\) |
\(\Xi\) |
xi |
Auxiliary or random variables; the point "somewhere in between" in error formulas |
\(o\) |
\(O\) |
omicron |
Almost never used (indistinguishable from Latin o) |
\(\pi\) |
\(\Pi\) |
pi |
\(3.14159\ldots\); permutations; policy in reinforcement learning; prime-counting function. \(\Pi\): product |
\(\rho\) |
\(P\) |
rho |
Correlation; density; radius in polar coordinates |
\(\sigma\) |
\(\Sigma\) |
sigma |
Standard deviation; sigmoid function; singular values; permutations. \(\Sigma\): summation; covariance matrix; diagonal matrix of singular values in the SVD |
\(\tau\) |
\(T\) |
tau |
Time constant; torque; \(\tau = 2\pi\); Kendall’s rank correlation |
\(\upsilon\) |
\(\Upsilon\) |
upsilon |
Rare |
\(\varphi\), \(\phi\) |
\(\Phi\) |
phi |
Angles (especially in spherical coordinates); Euler’s totient; golden ratio \(\varphi \approx 1.618\); standard normal density. \(\Phi\): standard normal cumulative distribution function |
\(\chi\) |
\(X\) |
chi |
Chi-squared distribution \(\chi^2\); characteristic (indicator) function |
\(\psi\) |
\(\Psi\) |
psi |
Wave functions in physics; auxiliary angles and functions |
\(\omega\) |
\(\Omega\) |
omega |
Angular frequency; complex cube root of unity; an outcome. \(\Omega\): sample space; asymptotic lower bound; ohms |
Pronunciation: most English speakers say "pie" (\(\pi\)), "fie" or "fee" (\(\varphi\)), "kye" (\(\chi\)), "zay-ta" or "zee-ta" (\(\zeta\)), "mew" (\(\mu\)), "new" (\(\nu\)), "ksee", "zye" or "sigh" (\(\xi\)). US and UK speakers differ on "zee-ta"/"zay-ta" and "bay-ta"/"bee-ta"; either is fine.
C.13. Same symbol, different meanings
Notation is reused shamelessly. When a symbol seems to make no sense, check whether it has switched jobs.
| Symbol | Meanings |
|---|---|
\(\lvert \cdot \rvert\) (vertical bars) |
Absolute value \(\lvert x \rvert\); modulus of a complex number \(\lvert z \rvert\); size of a set \(\lvert A \rvert\); determinant \(\lvert A \rvert\). A single bar \(\mid\) means "divides" (\(3 \mid 12\)), "such that" (\(\{x \mid x > 0\}\)) or "given" (\(P(A \mid B)\)) |
Prime \('\) |
Derivative (\(f'\)); a related but different object (\(A'\), "A prime"); complement (\(A'\)); transpose in statistics (\(X'\)); feet and minutes (\(5'\)) |
Bar \(\bar{x}\) |
Mean (\(\bar{x}\)); complex conjugate (\(\bar{z}\)); complement (\(\overline{A}\)); repeating decimal (\(0.\overline{3}\)); logical NOT in circuit diagrams |
Hat \(\hat{x}\) |
Estimate (\(\hat{\theta}\)); prediction (\(\hat{y}\)); unit vector (\(\hat{\mathbf{v}}\)) |
Superscript \(-1\) |
Inverse function (\(f^{-1}\)); reciprocal (\(x^{-1} = 1/x\)); inverse matrix (\(A^{-1}\)); modular inverse (\(3^{-1} \equiv 5 \pmod 7\)) |
Superscript \(2\) on a function |
\(\sin^2 x = (\sin x)^2\), but \(f^2(x)\) sometimes means \(f(f(x))\); read the author’s conventions |
\(\Sigma\) |
Summation; covariance matrix; matrix of singular values |
\(\pi\) |
The constant; a permutation; a policy; the prime-counting function |
\(e\) |
Euler’s number \(2.71828\ldots\); an event; an edge in a graph; an identity element; scientific notation in code ( |
\(\times\) |
Multiplication; cross product; Cartesian product; dimensions (a \(3 \times 4\) matrix) |
\(\equiv\) |
Identity; congruence; logical equivalence |
\((a, b)\) |
An open interval; a point in the plane; an ordered pair; \(\gcd(a, b)\) in older number-theory texts |
C.14. How to read a formula
A dense formula is code with one-letter variable names and no comments. Read it the way you would read unfamiliar code: find the overall shape, identify every name, then trace the evaluation from the inside out.
Input: a formula you don’t immediately understand.
Steps:
-
Find the top-level shape. What is being defined or claimed? Is it "name = expression", an equation to solve, an inequality, an update rule?
-
Name every symbol. For each letter, decide: constant, variable, index, function, or operator? Use this appendix and the surrounding text. Which symbols are bound (indices of a \(\sum\), the variable of an \(\int\), the \(x\) under an argmin) and which are free (inputs the result depends on)?
-
Read from the inside out, as an interpreter would evaluate it.
-
Say it aloud in words, then in plain English.
-
Translate it into code.
-
Try a tiny example small enough to do by hand.
Output: a sentence you could say to a colleague, and a few lines of code that compute it.
C.14.1. Decoding 1: sample variance
Shape. A definition: the name \(s^2\) equals (a fraction) times (a sum).
Symbols. \(n\) is the number of data values (free). \(x_i\) is the \(i\)-th data value. \(\bar{x}\) ("x bar") is their mean. \(i\) is a bound index: it exists only inside the \(\sum\), which runs it from 1 to \(n\). \(s^2\) is a single name, "s squared": it is the square of the standard deviation \(s\).
Inside out. \(x_i - \bar{x}\) is how far value \(i\) is from the mean (its deviation). Squaring makes it non-negative and punishes large deviations more. The \(\sum\) adds the squared deviations for every data value. Finally multiply by \(\tfrac{1}{n-1}\): an average, except dividing by \(n - 1\) rather than \(n\) (the reason is in Chapter 31).
Aloud. "s squared equals one over n minus one, times the sum from i equals 1 to n of x sub i minus x bar, all squared."
Plain English. "The sample variance is (almost) the average squared distance of the data from its mean."
Code and tiny example. For the data 2, 4, 6: the mean is 4, the deviations are \(-2, 0, 2\), their squares are \(4, 0, 4\), the sum is 8, and \(8 / (3 - 1) = 4\).
xs = [2, 4, 6]
n = len(xs)
xbar = sum(xs) / n
s2 = sum((x - xbar) ** 2 for x in xs) / (n - 1)
print(s2)4.0
C.14.2. Decoding 2: fitting a model by least squares
Shape. A definition: \(\hat{\boldsymbol{\theta}}\) is the result of an argmin, that is, the input that makes some expression smallest.
Symbols. The hat says \(\hat{\boldsymbol{\theta}}\) is an estimate. Bold says \(\boldsymbol{\theta}\) and \(\mathbf{x}_i\) are vectors. \(\boldsymbol{\theta} \in \mathbb{R}^d\) says \(\boldsymbol{\theta}\) is a list of \(d\) real numbers (the model’s parameters) and that the search ranges over all such lists; \(\boldsymbol{\theta}\) is bound by the argmin. \(\mathbf{x}_i\) is the \(i\)-th input (a feature vector) and \(y_i\) the matching observed output. \(\boldsymbol{\theta}^T \mathbf{x}_i\) is a transpose times a vector, which is just the dot product \(\boldsymbol{\theta} \cdot \mathbf{x}_i\): the model’s prediction for example \(i\).
Inside out. \(y_i - \boldsymbol{\theta}^T \mathbf{x}_i\) is the residual, the truth minus the prediction. Square it, sum over all \(n\) examples, divide by \(n\): the mean squared error of the model with parameters \(\boldsymbol{\theta}\). The argmin then asks for the parameter vector that makes this error smallest.
Aloud. "Theta hat equals the arg min, over theta in R d, of one over n times the sum from i equals 1 to n of y sub i minus theta transpose x sub i, squared."
Plain English. "Our fitted parameters are whichever parameters make the average squared prediction error on the data as small as possible." This is linear regression (Chapter 31); the argmin is found either exactly by linear algebra (Section 33.2) or by gradient descent (Chapter 20, Chapter 33).
Code and tiny example. With \(d = 1\) and two data points \((x, y) = (1, 2)\) and \((2, 4)\), the loss is \(\tfrac{1}{2}\left((2 - \theta)^2 + (4 - 2\theta)^2\right)\), which is zero at \(\theta = 2\), so \(\hat{\theta} = 2\). A crude argmin by search confirms it:
X = [[1.0], [2.0]]
Y = [2.0, 4.0]
def loss(theta):
n = len(X)
return sum((y - sum(t * xj for t, xj in zip(theta, x))) ** 2
for x, y in zip(X, Y)) / n
candidates = [[k / 100] for k in range(0, 501)]
theta_hat = min(candidates, key=loss) # argmin by brute force
print(theta_hat, loss(theta_hat))[2.0] 0.0
Notice that Python’s min(…, key=loss) returns the argument, not the minimum value: it is an argmin. loss(theta_hat) is the min.