Chris Collins

Tagged “lucene”

Tool ·

Postings lab

Inside an inverted index: build one from documents and look at docIDs, frequencies and positions; compress the postings with VByte, Elias γ/δ, Golomb/Rice, Simple-9/16, FOR and PForDelta blocks of 128, Lucene's PForUtil and Roaring bitmaps (bits per posting, bit by bit); intersect with skips and galloping; and step through MaxScore, WAND and Block-Max WAND against exhaustive BM25.

Search and retrieval

Tool ·

Analyzer and inverted-index playground

Build an Elasticsearch analyzer from char filters, a tokenizer and token filters and watch the token stream change stage by stage — terms, positions, offsets, types and position lengths, drawn as a token graph — then index documents to see the postings, field-length norms and phrase/slop matching, and copy the _analyze request and index settings.

Search and retrieval

Tool ·

Elasticsearch, simulated

An animated, step-by-step Elasticsearch (and OpenSearch) cluster in your browser: murmur3 routing, primaries and replicas, the indexing buffer, translog, refresh, flush and segment merging inside every shard, query-then-fetch search, node failures, master elections, split brain and shard sizing — with guided scenarios and a REST console.

Search and retrievalData and event platforms

Tool ·

Relevance and scoring explainer

See why documents rank where they do: BM25 term by term like Elasticsearch's _explain, with k1 and b sliders that re-rank live, classic TF-IDF side by side, multi-field strategies and boosts, function_score (field_value_factor, decay curves, a recency script), and hybrid lexical + vector ranking with reciprocal rank fusion or a normalised linear blend.

Search and retrieval

Tool ·

Vector search, simulated

Approximate nearest-neighbour search you can watch: build and search HNSW layer by layer, train k-means and probe IVF cells, quantize with PQ, int8 and binary codes, compare L2, cosine and dot product, and benchmark recall against work and memory in a Web Worker.

Search and retrievalApplied AI