Exploration ·
Follow a query from the box you type it into to the ten blue links: how text becomes terms, how an inverted index stores and compresses them, how BM25 scores a match, how we know a ranking is any good, and how the whole thing is split across thousands of machines — with the tools running live between the paragraphs.
Tool ·
Inside an inverted index: build one from documents and look at docIDs, frequencies and positions; compress the postings with VByte, Elias γ/δ, Golomb/Rice, Simple-9/16, FOR and PForDelta blocks of 128, Lucene's PForUtil and Roaring bitmaps (bits per posting, bit by bit); intersect with skips and galloping; and step through MaxScore, WAND and Block-Max WAND against exhaustive BM25.
Tool ·
Drop in text, Markdown, HTML or PDF and chunk it five ways — fixed token windows with overlap, sentences, LangChain's recursive splitter, Markdown headings and semantic breakpoints — see every boundary and overlap on the text, then compare BM25, embedding and hybrid retrieval across the chunkers with recall@k and MRR against passages you mark.
Tool ·
See why documents rank where they do: BM25 term by term like Elasticsearch's _explain, with k1 and b sliders that re-rank live, classic TF-IDF side by side, multi-field strategies and boosts, function_score (field_value_factor, decay curves, a recency script), and hybrid lexical + vector ranking with reciprocal rank fusion or a normalised linear blend.
Tool ·
The mathematics under machine learning, hands on — least squares with draggable points (residual squares, outliers, L1 vs L2, normal equations vs gradient descent and SGD), logistic regression and its cross-entropy bowl, a computation graph with forward and backward passes against dual numbers and finite differences, TF-IDF and BM25 search, entropy with a Huffman tree builder and KL divergence, and softmax temperature with an attention matrix.