Data-lake browser

Connection — choose one, or + New and ✎ Edit:

Browse — the tree on the left lists buckets (containers, the HDFS root) and the folders you have opened; the list pages through ListObjectsV2 continuation tokens (Azure markers, GCS page tokens, WebHDFS LISTSTATUS_BATCH) with Load more. Starts with… searches by key prefix on the server; Filter this page narrows what is listed; click a column to sort (folders first). Tick rows to delete several at once; + Folder creates a prefix; drop files on the browser or ⇪ Upload to upload into the open folder.

Click a file. Info shows the size, ETag (and what a multipart ETag means), storage class, content type, user metadata, versions, and for HDFS the owner, permissions, replication, block size, the blocks along the file with their datanodes (GETFILEBLOCKLOCATIONS), and a replication setter. Preview shows text, CSV (delimiter detected), JSON, NDJSON and images, read with a Range request (the first 256 KB) — gzip files are inflated when small. Schema reads only the footer of Parquet (FileMetaData: schema with logical types, every row group and column chunk with codec, encodings, compressed and uncompressed sizes, min/max statistics and null counts), the tail of ORC (types, stripes, column statistics; ZLIB and Snappy footers) and the header and first blocks of Avro (writer schema, codec, rows). Commands lists the CLI equivalents for the file and the last request as curl.

Actions: Download (8 MB Range reads with progress; big S3 files through a presigned URL), Presigned URL with your choice of expiry (1 s – 7 days), Copy, Move / rename (S3 copy
delete; WebHDFS RENAME), Query in SQL, Delete (always confirmed).

SQL — DuckDB-wasm in a worker. ▶ Query in SQL adds a file under its name; a query that names it fetches it (up to 200 MB) the first time. On a direct S3 connection, Parquet over 32 MB is read by Range through a presigned URL, so DuckDB fetches only the footer and the columns a query needs. Examples include parquet_metadata(), parquet_schema() and SUMMARIZE. Export the result as CSV, JSON or Parquet. The demo’s files can be queried by name (SELECT * FROM 'cities.csv').

Bucket — the bucket’s location, versioning, CORS, policy and lifecycle (S3 direct, GCS), each with its CLI command; for HDFS the folder’s content summary and quotas.

Generators — bucket CORS for Amazon S3, MinIO (one server-wide setting), Cloudflare R2, Backblaze B2 (native rules with s3_* operations), Wasabi, GCS, Azure and Hadoop, with the command that applies it; a least-privilege IAM policy (read-only, read-write or write-only, scoped to a prefix with an s3:prefix condition on ListBucket); bucket policies (deny non-TLS, public read of a prefix, read for one role); lifecycle rules (transition, expiry, old versions, abort incomplete multipart uploads).

Transfers — uploads and downloads with progress and speed. S3 files over 16 MB go up as multipart uploads (8 MiB parts, four at a time, raised so a file fits in 10,000 parts); Pause lets the running parts finish, Resume carries on, and an upload interrupted by a reload resumes when you choose the same file again (the upload id and part ETags are kept in this browser). Azure uploads over 16 MB use Put Block / Put Block List the same way.

Requests — every request the page made (signatures and tokens masked) and Copy as curl: SigV4 requests use curl --aws-sigv4 with $AWS_ACCESS_KEY_ID / $AWS_SECRET_ACCESS_KEY, the app’s requests $GW_COMPANION_TOKEN, Azure $AZURE_SAS_SIG — never the secret itself.