Enterprise search, information retrieval, ranking, and content platforms.
Exploration ·
Follow a query from the box you type it into to the ten blue links: how text becomes terms, how an inverted index stores and compresses them, how BM25 scores a match, how we know a ranking is any good, and how the whole thing is split across thousands of machines — with the tools running live between the paragraphs.
Tool ·
Load a query and click log (or simulate one from known relevance and a known user), see CTR by rank, abandonment and reformulations, fit the click models of Chuklin, Markov & de Rijke — PBM, cascade, DCM, UBM, SDBN, DBN — checked against PyClick, correct for position bias with propensities from PBM or swap experiments, and compare interleaving with A/B testing.
Tool ·
Find near-duplicate documents: character or word shingles, exact Jaccard, seeded MinHash signatures, LSH banding with its S-curve and a (b, r) tuner, 64-bit SimHash with the permuted-table search, precision and recall against an exact similarity join, and clusters — 10,000+ documents in a Web Worker.
Tool ·
Inside an inverted index: build one from documents and look at docIDs, frequencies and positions; compress the postings with VByte, Elias γ/δ, Golomb/Rice, Simple-9/16, FOR and PForDelta blocks of 128, Lucene's PForUtil and Roaring bitmaps (bits per posting, bit by bit); intersect with skips and galloping; and step through MaxScore, WAND and Block-Max WAND against exhaustive BM25.
Tool ·
Drop in text, Markdown, HTML or PDF and chunk it five ways — fixed token windows with overlap, sentences, LangChain's recursive splitter, Markdown headings and semantic breakpoints — see every boundary and overlap on the text, then compare BM25, embedding and hybrid retrieval across the chunkers with recall@k and MRR against passages you mark.
Tool ·
Build an Elasticsearch analyzer from char filters, a tokenizer and token filters and watch the token stream change stage by stage — terms, positions, offsets, types and position lengths, drawn as a token graph — then index documents to see the postings, field-length norms and phrase/slop matching, and copy the _analyze request and index settings.
Tool ·
An animated, step-by-step Elasticsearch (and OpenSearch) cluster in your browser: murmur3 routing, primaries and replicas, the indexing buffer, translog, refresh, flush and segment merging inside every shard, query-then-fetch search, node failures, master elections, split brain and shard sizing — with guided scenarios and a REST console.
Tool ·
Build Elasticsearch and OpenSearch queries in a form or as JSON — each edits the other — and see them explained in plain English (query vs filter context, minimum_should_match, the Lucene query they become), linted for slow or surprising patterns, and run against a sample index in your browser with hits, scores, explanations, highlights and aggregations; optionally send them read-only to your own cluster.
Tool ·
Paste TREC qrels and runs (or CSV) and get P@k, recall@k, MAP, MRR, nDCG@k with linear and exponential gain, ERR and R-precision per query — checked against trec_eval and gdeval — then compare two rankers with paired t, Wilcoxon, randomisation and bootstrap tests, and simulate team-draft interleaving.
Tool ·
See why documents rank where they do: BM25 term by term like Elasticsearch's _explain, with k1 and b sliders that re-rank live, classic TF-IDF side by side, multi-field strategies and boosts, function_score (field_value_factor, decay curves, a recency script), and hybrid lexical + vector ranking with reciprocal rank fusion or a normalised linear blend.
Tool ·
Edit distances with the dynamic-programming table filled cell by cell (Levenshtein, OSA vs true Damerau, LCS, Hamming), Jaro–Winkler, Levenshtein automata as Lucene's FuzzyQuery builds them and walks the term dictionary, Soundex, Metaphone, Double Metaphone and NYSIIS, pg_trgm trigram similarity, and a Norvig spelling corrector.
Tool ·
Real tokenizers side by side in your browser — GPT-4/GPT-4o byte-level BPE (cl100k_base, o200k_base), BERT WordPiece, T5 SentencePiece Unigram and Llama 2 SentencePiece BPE with byte fallback — with coloured token boundaries and ids, UTF-8 bytes and split characters, BPE merges step by step, Unicode normalisation, cost per 1k tokens, the common pitfalls, and a tiny BPE you train yourself.
Tool ·
Approximate nearest-neighbour search you can watch: build and search HNSW layer by layer, train k-means and probe IVF cells, quantize with PQ, int8 and binary codes, compare L2, cosine and dot product, and benchmark recall against work and memory in a Web Worker.
Article ·
What it takes to build a complete social network as one coherent app — feed ranking, social graph, search, messaging — with Spring Boot, React, PostgreSQL, OpenSearch, and a Rust graph cache.
Article ·
A standalone JavaFX desktop app for inspecting and diagnosing Lucene indexes — documents, analyzers, and vector fields — as a desktop alternative to Luke.
Project ·
A full-featured social network — feeds, profiles, friends, groups, pages, messages, search, and an admin dashboard — built on Spring Boot, React, PostgreSQL, OpenSearch, and my Rust graph cache.
Project ·
A multi-language code analysis platform that turns repositories into a searchable graph of classes, functions, calls, and dependencies — with a REST API, a web UI, and an MCP server for AI assistants.
Project ·
A local log search and analysis tool — Splunk-like search, facets, SQL, charts, and exports running entirely on your own machine, built on Java, Lucene, and Spring Boot.
Project ·
A horizontally scalable web crawler in Java: Kafka distributes URLs across crawler processes, Cassandra keeps the metadata, and S3 or MinIO stores the content.