Chris Collins

Tagged “compaction”

Exploration ·

How a database survives a crash

Pull the plug on a database halfway through a write and it comes back with every committed transaction and none of the half-finished ones. How? Follow a write through the log, the buffer pool and the disk, compare B+trees with LSM-trees, watch compaction, and then see the other half of the promise — what concurrent transactions are allowed to see — with real PostgreSQL and SQLite running between the paragraphs.

Data and event platformsSoftware engineering

Tool ·

NoSQL capacity and cost

Size NoSQL deployments with the working shown: Cassandra and ScyllaDB nodes from rows, replication and compaction headroom, with node-replacement and repair times; Redis memory per key by encoding (listpack, intset, hashtable, skiplist) with fragmentation, replication buffers and an eviction policy; MongoDB working set, WiredTiger cache and oplog window; Azure Cosmos DB request units, partitions and cost; and a Zipf hot-partition checker.

Data and event platforms

Tool ·

Offline-first sync and replication

How offline-first apps sync, animated: CouchDB/PouchDB-style replication between a phone, a laptop and a server that go offline — revision trees, the changes feed, checkpoints, _revs_diff and _bulk_docs, conflicts and the deterministic winning revision, three ways to resolve them, compaction and _revs_limit's false conflicts — against a Firestore-style single-leader cache with pending writes, server timestamps and rolled-back writes.

Data and event platforms

Tool ·

Time-series database lab

Inside a time-series database: validate InfluxDB line protocol and Prometheus exposition text and see the series keys; watch Gorilla, Prometheus XOR/XOR2 and InfluxDB TSM compression bit by bit on your data (byte-identical to Prometheus's and InfluxDB's own encoders); estimate cardinality and memory; downsample counters correctly; size retention tiers; and watch Prometheus cut and compact blocks.

Data and event platformsSoftware engineering

Tool ·

Cassandra lab

A Cassandra data-modelling lab and CQL client: a cqlsh-like console on a built-in Cassandra node (CQL, TTLs, tombstones, SSTables, lightweight transactions), a query-first checker that says which queries a table supports and why, partition sizes and hot partitions, the token ring with vnodes and replica placement, consistency levels with replica responses — and a native-protocol client for your own Cassandra through the gitwebsite app.

Data and event platforms

Tool ·

Storage engines: LSM-tree vs B-tree, simulated

A B+tree and an LSM-tree side by side in your browser, fed the same operations: page splits and merges, the WAL and buffer pool, memtable flushes, leveled, size-tiered and lazy-leveling compaction, Bloom filters, fence pointers and tombstones — with write, read and space amplification counted as you watch, guided scenarios and the textbook cost models.

Data and event platformsSoftware engineering