NoSQL data modelling
Model one application four ways — document (embed or reference, with the subset, bucket, extended-reference, computed and outlier patterns), wide column (one table per query), key-value (keys and hand-made indexes) and graph — and see for each which access patterns are one read, several or scatter-gather, how many copies every write touches and what can go out of step; then a "Which database?" questionnaire with a reasoned shortlist and a CAP/PACELC explainer, and exports as CQL, MongoDB $jsonSchema, Cypher and JSON.
Added
Works with
- MongoDB playground Databases and streamingtry the document model's queries and paste its $jsonSchema validators into a collection
- Cassandra lab Databases and streamingcreate the exported CQL tables and check which queries each one supports
- DynamoDB single-table modeller Databases and streamingturn the key-value model's keys and hand-made indexes into a single table with GSIs
- Graph database playground Databases and streamingrun the graph model's Cypher on a sample graph
- Row key design Databases and streamingwork out the bytes and hotspots of a partition or row key chosen here
- NoSQL capacity and cost Databases and streamingsize the cluster, memory and cost for the model you pick
- Consistency models lab Distributed systemssee what the consistency risks listed for each model look like when replicas disagree
How to use it
A NoSQL schema starts from the questions the application asks, not from the data. The same application — the same entities, relationships and access patterns — comes out very differently in a document store, a wide-column store, a key-value store and a graph, and each version makes some reads cheap by making some writes expensive. This page takes one application and models it all four ways side by side, says which rule produced every choice, and counts the cost.
How to use it
Application. Pick one of six examples — e-commerce orders, a chat app, IoT telemetry, a social
graph, a content management system or a multi-tenant SaaS product. The Application tab lists its
entities (fields, typical size, how many), its relationships (cardinality, how far they grow,
whether they are read together) and its access patterns with how often each runs. Change a
frequency and every model follows; Edit the application as JSON lets you describe your own
(the format is in the comment at the top of nosql-modeling-data.js), checked before it is
applied. Sharded / partitioned cluster decides whether the document model is sharded, which is
what turns some queries into scatter-gather.
Each model tab opens with five numbers: the share of read traffic answered by a single request, the mean number of round trips per read, how many reads are scatter-gather, the mean number of copies written per logical write, and the consistency risk. Below them:
-
Document — every relationship’s embed-or-reference decision with the reasons (cardinality, growth, read together, update frequency, size against the 16 MB document limit) and the pattern it uses; each collection as an example document with its shard key; the reads (as
findqueries), the writes (which documents each touches and whether it needs a multi-document transaction) and the risks. Export: onedb.createCollectionwith a$jsonSchemavalidator per collection. -
Wide column — one table per query: partition key, clustering columns with their order, the columns copied in, static columns, rows and bytes per partition (flagged past Cassandra’s guidance), plus counter, latest-value, rollup and fan-out timeline tables and a source-of-truth table for anything no query reads. Export: CQL.
-
Key-value — the key schema: one key per aggregate, list and set keys, and
idx:keys that are secondary indexes kept by hand; each read as the commands it takes. Export: JSON. -
Graph — nodes and relationships drawn (link entities such as an order line or a follow become relationships with properties, dashed); each read as a Cypher query. Export: uniqueness constraints.
-
Compare — every read pattern under each model, and the totals, in one table. Export: all four models as JSON.
-
Which database? — answer eight questions (the shape of the hot reads, how well the queries are known, consistency, scale, regions, latency, who runs it, memory). Each answer adds or takes away points from eight candidates with the reason shown, optionally with signals from the application itself (a multi-hop traversal favours a graph, a heavy time series a wide-column store). Answers for this example fills in a sensible set for the chosen application. Below the shortlist, CAP and PACELC are explained with a table of where common systems sit.
What it models
Document (MongoDB, Data Modeling; Building with Patterns):
-
One-to-one or one-to-few, read together and bounded → embed. One-to-many is embedded too while the worst case stays under 1 MB — this page’s rule of thumb, well inside MongoDB’s 16 MB BSON document limit, because a growing document is rewritten and sent whole.
-
Read together but too large or growing without bound → subset: the newest N in the parent (
$pushwith$slice), all of them in their own collection. -
Read together, usually small, but a few parents have vastly more → outlier: embed up to a threshold, flag the parent and put the rest in overflow documents.
-
Unbounded time series → bucket: one document per parent per hour or day (MongoDB 5.0+ time series collections do this internally).
-
Many-to-one with fields the hot read needs → extended reference: keep the id and copy those fields; updating them fans out to every copy unless the copy is a deliberate snapshot (an order line’s product name at the time of purchase).
-
Many-to-many, bounded → an array of ids (or extended references). Unbounded or not read together → reference: the child stores the parent’s id.
-
An aggregate read more often than its inputs change → computed: kept in the document and updated on every write.
-
An entity is embedded in at most one parent; a second relationship to it is a reference.
-
Shard keys: the top-level field of each collection’s busiest read; a read without it is scatter-gather.
Wide column (Apache Cassandra, Data Modeling):
a table per read pattern; the partition key is the pattern’s equality fields (plus a time bucket
under a bucketed parent), clustering columns its sort order and then the entity’s key; included
children of a single parent become rows of its partition with the parent’s columns STATIC;
other included data is copied in. Aggregates become counter tables (counters cannot share a table
with other columns), latest-value or rollup tables. A timeline through a link (posts of the people
I follow) becomes fan-out on write. Partitions are flagged past about 100 MB or 100 000 rows, and
low-cardinality partition keys are called out. Writes count every table that holds the data.
Key-value (Redis, or DynamoDB without indexes): the document model’s aggregates as values; a
read by the key is one GET; a parent’s children are a sorted set of ids, read and then MGET
(two round trips); any other lookup needs an idx: key maintained by hand; edges are sets; a time
series is a list or stream per bucket. Multi-key writes are atomic only within one node (Redis
hash tags) or a transaction (DynamoDB TransactWriteItems).
Graph (Neo4j, data modelling): entities are nodes, relationships edges, link entities edges with properties; every read is one Cypher query, and a traversal’s cost is the number of relationships walked. Huge time series and supernodes are flagged.
Which database? The scores are this page’s judgement, written down so you can disagree with each one. PACELC is from Daniel Abadi, Consistency Tradeoffs in Modern Distributed Database System Design (IEEE Computer, 2012); CAP from Eric Brewer’s PODC 2000 keynote, proved by Gilbert and Lynch (2002).
Limits
-
The models are derived by rules from a short description, not designed by hand: a real schema would also weigh things the description does not hold (which fields change together, privacy, retention, how the application handles stale data). Treat each model as a well-argued first draft.
-
Sizes and counts are illustrative orders of magnitude for a mid-sized service. Fan-out counts (how many copies an update must reach) are estimated as count(referencing) ÷ count(referenced).
-
"One read" means one request to one place; it says nothing about how many bytes come back or how hot the partition is — see row key design and NoSQL sizing for that.
-
MongoDB secondary indexes and Cypher property indexes are assumed to exist for every field a pattern looks up by; Cassandra secondary indexes and materialised views are not used — each query gets its own table, as the Cassandra guidance recommends.
-
The questionnaire ranks eight well-known systems and leaves out many good ones (Couchbase, HBase, Bigtable, FoundationDB, ArangoDB, TiDB…). Classes in the PACELC table describe default configurations.
Linking to an example
Links can be written by hand. Add a tab after #: app (the default), document, wide, kv,
graph, compare or choose. A link’s settings replace this page’s saved settings (a button
restores yours); an edited application travels packed in the link.
| Parameter | Meaning | Example |
|---|---|---|
|
An example application with its questionnaire answers (list below) |
|
|
The same, keeping your questionnaire answers |
|
|
Shard the document model ( |
|
|
Hot reads: |
|
|
Queries: |
|
|
Consistency: |
|
|
Writes and data: |
|
|
|
|
|
Read latency: |
|
|
Who runs it: |
|
|
Working data |
|
|
Score the application’s own patterns too ( |
|
example= |
Shows |
|---|---|
|
Embedded order lines, a snapshot copy of the product name, the subset pattern for reviews and a computed rating (open) |
|
Messages bucketed per channel per day, memberships embedded in users, and a display-name change that reaches 4 000 copies (open) |
|
50 000 readings a second: bucket documents, an hour in the partition key, latest-value and rollup tables (open) |
|
The home timeline: fan-out on write in a table store against two hops in a graph (open) |
|
The outlier pattern for comments and a tag table with one row per tag (open) |
|
Tenant-scoped queries, a shard key that makes "my tasks" scatter, and an audit log in day buckets (open) |
Feedback is a gift. Something to improve, a feature to add, or a bug? Tell me.
Runs entirely in your browser; nothing you type leaves this page.