CSVView
CSVView
=======Streaming‑first CSV writer mirroring the Parquet view.
Responsibilities
FUN/VAR streaming: append live into buffered writers; keep an in‑memory buffer for a final rewrite
Final rewrite (optional): globally sort by
(K, correlation)
and rewrite both files alignedBEST →
(K asc, corr asc)
WORST →
(K asc, corr desc)
AVERAGE rows are one‑per‑K; effective order is
K asc
Topics in
-Var
and-Top
: base64‑packed masks as"B64:<base64>"
(no padding)Top solutions: replace‑batch semantics; header announces
TopicsB64
Set
-Dnbs.csv.top.live=false
to buffer all batches and write once on close
Filesystem
Files live under the per‑run CSV subdirectory returned by ViewPaths.ensureCsvDir
Toggles
-Dnbs.csv.top.live
(defaulttrue
): live write of-Top
on each batch-Dnbs.csv.finalRewrite
(defaulttrue
): global sort & rewrite of FUN/VAR on close-Dnbs.csv.flushEvery
(default256
): throttled flush frequency-Dnbs.csv.buffer
(default262144
): writer buffer size in bytes
Functions
Close writers, then (optionally) globally sort & rewrite FUN/VAR to keep them aligned.
Append one streamed FUN/VAR row, buffering for a possible final rewrite.
Merge/replace cached TOP blocks and optionally write the whole file live.
Open and cache large‑buffer appenders for FUN/VAR of the current run/target.
Write a full snapshot of FUN/VAR (and TOP when applicable) to CSV.