COMING SOON · PROPOSALproposals/yaza-data-media-web-modules.md

Status dokumen bukan klaim bahwa fiturnya sudah tersedia.

RFC dan proposal mencatat rancangan. Untuk kemampuan yang benar-benar dapat dipakai, lihat matriks kapabilitas runtime dan catatan rilis.

Proposal: YAZA-Owned Data, Media, Analytics, and Web Modules

Status: Accepted design direction — staged implementation in progress Scope: Standard-library design for data files, documents, media, statistics, and web applications
Decision: Accepted by the project owner on 2026-10-24. Option A is selected: YAZA-authored format and analytics implementations with narrow operating-system API adapters; no third-party library implementations linked, loaded, copied, or vendored.

1. Goal

Give YAZA applications a coherent, safe, streaming standard library for common business data and web workloads. The library should support spreadsheets, documents, tabular data, images, audio, video, statistical analysis, and future web delivery from one language and toolchain.

The implementation must be authored and maintained in the YAZA project. It must not wrap or link third-party format, codec, analytics, or web-framework implementations such as libpng, zlib, FFmpeg, PDFKit, Pandas, or NumPy. Specifications and public test files may be used to implement and verify interoperability; reference implementations may be used only as external test oracles, never as runtime dependencies or copied implementation code.

YAZA must still read and write established formats so its files work in existing software. “YAZA-owned” describes the implementation, not a new private file format. The initial code should live in the YAZA standard library and native runtime. Narrow operating-system adapters may provide platform facilities such as file descriptors, sockets, memory mapping, clocks, and secure randomness; they must not silently substitute another library's format or codec implementation.

Why build with the YAZA ecosystem

The goal is a connected stack in which the language, compiler, CLI, runtime, API framework, data modules, and YAZA UI share types, diagnostics, project conventions, and release policy. This should reduce glue code and duplicate schemas, make it easier to carry validated data from an API into a web or mobile view, and give projects one coherent way to build, test, inspect, and deploy. YAZA UI is intended to supply consistent responsive behavior and visual standards for web interfaces; future native renderers should share its component contract and tokens.

Performance and speed are design targets across the whole stack: bounded parsers and streams should control memory use; native code generation and selective module inclusion should avoid unnecessary runtime and package weight; typed shared contracts should reduce conversion and validation duplication; and common CLI workflows should shorten setup and feedback cycles. Every benefit remains a hypothesis until measured on working implementations. Benchmark comparable workloads for startup, throughput, tail latency, memory, binary or bundle size, build time, and implementation effort. Do not advertise superiority from architecture diagrams or isolated microbenchmarks.

2. Design rules

Standards-based interoperability. Implement published formats and protocols where practical. Record the exact supported specification version, profiles, and unsupported features.
YAZA-owned implementation. Write and review the production parsers, encoders, algorithms, and APIs in YAZA source. Do not vendor, translate, link, or dynamically load third-party implementations.
Streaming first. Expose readers and writers over bounded byte streams. Large files should not need to be copied wholly into memory.
Limits before allocation. Every parser has explicit byte, element, nesting, pixel, duration, decompression, and execution budgets. Invalid or oversized input returns a typed error.
Safe by default. Parsing a document must not run embedded scripts, macros, actions, external links, or media. Network access and file extraction are explicit capabilities.
Deterministic results where useful. CSV, statistics, document generation, and tests should be reproducible under documented settings. Randomized algorithms take an explicit seed; security randomness remains a separate API.
Prove claims. Do not claim “fastest,” “complete,” or “production-ready” until conformance, fuzzing, resource-bound, cross-platform, and repeatable benchmark gates pass.
Separate formats from frameworks. General-purpose parsers and algorithms belong in std; API CRUD, authentication, and deployment conveniences belong in higher-level web packages.

Initial implementation status

The first bounded profiles now include YAZA-authored CSV record parsing/encoding in std/csv.yz and deterministic single-page printable-ASCII PDF generation in std/pdf.yz. These are intentionally narrow initial slices, not complete implementations of CSV dialects, spreadsheets, or PDF 2.0. See the runtime capability matrix for current limits. All other modules below remain design scope until implemented and verified.

3. Module map

All module names below are proposed. Their APIs and syntax need a separate normative specification before implementation.

Module	Responsibility	Proposed first scope
`std.bytes`	Bounded byte values, slices, endian conversion, checked cursors	Safe length-aware binary access, ownership and limits
`std.stream`	Incremental readers and writers	Files, memory, sockets, cancellation, backpressure
`std.compression`	YAZA-authored compression and checksums	A streaming DEFLATE implementation with strict output limits and checksums needed by ZIP/PNG
`std.archive`	Container parsing and writing	ZIP central directory, CRC validation, path traversal prevention, duplicate-entry policy
`std.text`	UTF-8, Unicode, normalization and tokenization primitives	Explicit invalid-text policy and bounded conversions
`std.csv`	Tabular text interchange	RFC 4180-compatible reader/writer, configurable delimiters, streaming records, headers, typed conversion
`std.spreadsheet`	Spreadsheet workbooks	`.xlsx` read/write over YAZA ZIP/XML modules; cells, sheets, basic styles, formulas, cached values, and streaming row access
`std.pdf`	Portable document generation and extraction	Deterministic text/vector/table PDF generation first; bounded metadata and text extraction next
`std.image`	Raster image decode, encode, and transforms	PNG read/write, dimensions and metadata, safe resize/crop/color conversion
`std.audio`	Audio sample and file operations	PCM samples and WAV read/write, then FLAC after conformance groundwork
`std.video`	Timed media containers and processing	Bounded metadata/demux for a documented WebM/Matroska subset before any encoder or broad codec support
`std.stats`	Statistical analysis	Descriptive statistics, quantiles, distributions, intervals, resampling, regression, and time-series foundations
`std.web`	Web primitives for server, browser, and edge apps	HTTP client/server types, URL and form parsing, safe HTML output, static assets, uploads, and streaming responses
`std.web.wasm`	WebAssembly host bindings	Browser and edge host interop with capability-limited APIs and no hidden native dependency

XML is a shared parser/writer dependency for spreadsheet formats. It must disable external entity resolution and external resource fetching by default. JSON and PostgreSQL modules should remain distinct but use the shared bytes, text, error, and stream foundations.

4. Module scope and behavior

4.1 CSV and spreadsheets

std.csv should parse quoted fields, doubled quotes, embedded line breaks, CRLF, and UTF-8 incrementally. It should make header handling and ragged-row behavior explicit rather than guessing. Spreadsheet-oriented exports should offer an explicit formula-injection-safe text mode for values beginning with spreadsheet formula markers.

The first spreadsheet target is .xlsx, based on Office Open XML and Open Packaging Conventions. Support should be profile-based: initially values, formulas as text, cached formula values, dates, basic number formats, multiple sheets, and bounded streaming. Formula evaluation should be a separately staged capability with a documented function set; unsupported formulas must not be silently rewritten or given fabricated results. Legacy binary .xls, macros, external data links, and pivot-table authoring are out of the initial scope.

4.2 PDF and images

PDF generation should produce deterministic documents from text, simple vector shapes, and tables, with embedded fonts added after Unicode/font licensing and subset rules are specified. A later reader may extract metadata and text within strict object, nesting, and decompression limits. The parser must not execute JavaScript, launch actions, retrieve URLs, or interpret embedded files. Digital signatures, encryption, redaction guarantees, and PDF/A conformance require separate security and profile proposals.

Image support should be a set of documented decoder/encoder profiles, not a promise to read every file that has a familiar extension. Begin with PNG, including checksums, color type/depth validation, alpha, metadata bounds, and streaming where the format allows it. JPEG and WebP should be separately scoped and accepted only after conformance corpus, fuzzing, color handling, and resource limits are established. Image transforms should use explicit pixel formats and avoid implicit color-space or orientation changes.

4.3 Audio and video

WAV/PCM is the first audio file target because it enables a useful, testable sample pipeline. FLAC read/write is a subsequent lossless codec target with published conformance vectors. MP3, AAC, and Opus support requires independent format, patent/licensing, and conformance review before acceptance.

Video is a large independent effort. First define timestamps, rational time bases, bounded frame/sample queues, and container inspection. Then implement a narrowly specified WebM/Matroska container subset. A container parser does not imply support for every codec stored inside it. AV1, VP9, H.264, and other video codecs each require a separate proposal, test corpus, licensing review, and performance plan. Browser-native decode/encode may be exposed as an optional platform adapter, but must be clearly labeled as host-provided functionality and cannot count as the YAZA-owned codec implementation.

4.4 Statistics

The statistics API should specify numeric type, precision, missing-value and NaN behavior, sample-versus-population conventions, confidence assumptions, and reproducibility. Start with stable one-pass and mergeable descriptive statistics, quantiles, distributions, confidence intervals, bootstrap resampling, regression, and time-series summaries. Matrix factorization and large-scale columnar execution should follow only after generic collection and numeric foundations are stable.

Statistical functions should return structured errors for insufficient samples, invalid domains, non-convergence, and overflow. The module must distinguish exploratory summaries from methods that make inferential assumptions; assumptions should be visible in names or result metadata.

4.5 Web foundations

The web module should reuse YAZA's HTTP runtime while adding typed URL/query/form parsing, multipart streaming, content negotiation, safe HTML escaping, static-file metadata, server-rendered output, and response streaming. Uploads need explicit size and temporary-storage limits. Browser and Cloudflare/edge support must go through explicit target adapters, with capability checks at compile time or startup.

The web module must not silently turn file parsing into remote fetches or permit arbitrary filesystem access. SSRF protections, cookie defaults, CORS, CSP, authentication, and request-body limits remain security-reviewed framework features rather than conveniences that are enabled implicitly.

5. Suggested delivery sequence

This sequence is a dependency order, not a delivery-date commitment.

Foundation: checked byte cursors, byte/text conversions, streams, typed errors, cancellation, and per-operation resource budgets.
Small useful data layer: CSV, deterministic statistics primitives, and deterministic PDF generation for a restricted profile.
Packaging and spreadsheet base: XML and ZIP/DEFLATE written in YAZA, then bounded .xlsx reader/writer profiles.
Image base: PNG decoder/encoder and bounded transforms; JPEG and WebP proposals after PNG gates pass.
Audio base: WAV/PCM, then FLAC after conformance and fuzzing.
Web integration: streaming uploads/downloads, static assets, server rendering, and browser/edge bindings against the runtime's async I/O milestones.
Video: container metadata and demux subset; codec work proceeds only as independent accepted proposals.

Each stage can ship independently only when its own acceptance gates pass. A module with an unimplemented profile must report the unsupported feature rather than silently degrading or corrupting data.

6. Quality and security gates

Before a module is called implemented, it needs:

A normative API and a profile-by-profile support table.
Positive and negative fixtures, including malformed, truncated, ambiguous, and oversized inputs.
Round-trip and interoperability checks against independent tools. Those tools are test oracles only; they are not linked into YAZA.
Fuzzing under available memory and undefined-behavior sanitizers, with minimized regressions committed for every discovered crash or incorrect acceptance.
Explicit maximum input/output sizes, decompression expansion, recursion, collection counts, image dimensions/pixel count, audio duration/sample count, video duration/frame count, and CPU/time budgets.
No unbounded allocation or decompression before checking declared and actual limits. Archive extraction rejects absolute paths, traversal, links, and collisions by default.
Repeatable benchmarks that state hardware, compiler, workload, input corpus, warmup, sample count, and variance. Compare equivalent behavior and report throughput, latency, peak memory, and output size as applicable.
CI on supported macOS and Linux architectures, with WebAssembly/edge conformance added when those targets exist.

Cryptographic primitives, TLS, and signature verification remain governed by the accepted security RFCs. This proposal does not authorize YAZA to invent new cryptography or to treat a file-format checksum as authentication.

7. Standards and interoperability references

CSV behavior should be documented against RFC 4180, while acknowledging it is informational and real-world CSV variants exist.
Spreadsheet support should target the documented profiles of ECMA-376 Office Open XML and its ISO/IEC 29500 alignment.
PDF profiles should be selected against ISO 32000-2:2020.
PNG support should target the W3C PNG Third Edition Recommendation.
FLAC should follow RFC 9639 and its published conformance material.

8. Main trade-offs and accepted direction

Writing every implementation in YAZA can make the library coherent and auditable, but it significantly increases the engineering and verification burden. DEFLATE, OOXML, PDF, modern image codecs, and especially video codecs have very different complexity and risk. The accepted direction makes the implementations YAZA-owned while staging the supported profiles; it does not promise immediate support for every feature of every format.

Option A preserves a path to browser and native interoperability while keeping third-party library implementations out of YAZA. Acceptance records the design direction; implementation is proceeding in independently verified profiles. Each format profile still needs a normative API, implementation, interoperability evidence, and security gates before it can be called complete.

← Kembali ke semua dokumentasi Versi dan status rilis →