YAZA Database Driver Architecture

Status: Recommended architecture — implementation staged by API priority
Priority: API completion; PostgreSQL first
Implementation status: PostgreSQL protocol, transport, and CRUD foundations exist. Generated API builds use the pure-YAZA client for CRUD and lifecycle hooks and no longer link libpq; the current lifecycle path passed migration and full API HTTP smoke against disposable PostgreSQL 18.6. Remote TLS, CI/version coverage, and other database engines are not complete.

Goal

Give YAZA applications one safe, typed way to use several database families while keeping each provider’s behavior explicit. “YAZA-owned” means YAZA source/runtime implements the wire protocol or storage engine and its tests. It does not mean inventing incompatible database protocols or ignoring their specifications, trademarks, or licensing terms.

Database products do not all provide the same model. A relational SQL connection, an embedded SQL engine, a key-value server, and a document database must not be forced into one pretend-universal CRUD abstraction. YAZA should share common connection, cancellation, timeout, transaction, and error concepts where they genuinely match, then expose provider-specific capabilities.

Current state

• std/pg contains pure-YAZA PostgreSQL protocol, parameter, TLS, SCRAM, query, and transaction foundations. Generated API CRUD, startup/schema checks, and readiness call it; the current migration and database-configured API HTTP smoke passed on disposable PostgreSQL 18.6.
• std/db/core.yz now defines provider names, stable database error codes (including unavailable), an explicit readiness marker, a bounded ASCII plain-identifier syntax check, a tagged DbValue type that preserves SQL NULL, typed accessors, and a conservative capability descriptor. std/db/query.yz adds a bounded structured SELECT descriptor for a projection, one typed comparison, ordering, and pagination. std/db/postgres.yz lowers it to quoted identifiers, fixed operators, separately bound PostgreSQL values, and safe DbError categories; direct server text is not returned. The native compiler's query-block grammar and schema-snapshot resolution are still absent, so this runtime descriptor is only groundwork, not the final schema-checked query API. This is not yet a provider-neutral connection/transaction API or a schema-to-shape mapper.
• Generated API builds no longer link libpq. CRUD and lifecycle operations use the pure-YAZA client. The current lifecycle path has one PostgreSQL 18.6 live smoke; remote TLS, failure-path coverage, pooling, additional server versions, and repeatable live-database CI remain open.
• There is no shipped SQLite, MySQL, MariaDB, SQL Server, Oracle, Redis, or MongoDB driver/engine.
• The current language lacks a mature generic trait system, so the first driver contract should use explicit provider records/functions and compile-time module selection. It can evolve to traits after the language semantics exist.

Public package shape

Proposed names are provisional until the package format is stable:

std/db/core.yz             # shared values, errors, deadlines, capabilities
std/db/sql.yz              # typed SQL parameters, rows, result metadata
std/db/postgres/           # PostgreSQL wire protocol and provider
std/db/sqlite/             # embedded SQL engine and file format
std/db/mysql/              # MySQL protocol and provider
std/db/mariadb/            # MariaDB-specific capability layer
std/db/sqlserver/          # TDS provider
std/db/redis/              # RESP key-value client; not forced into SQL
std/db/document/           # future document-provider contracts

Provider selection is explicit in project configuration and code. A build should link only the provider modules used by that application. The initial engine must not dynamically load undocumented native libraries or silently fall back to a different provider.

Shared contracts

The common layer should define bounded values and stable error categories:

• DbError: connection, authentication, TLS, timeout, cancelled, constraint, serialization, unavailable, protocol, and internal categories, with a safe message and optional provider code.
• DbDeadline and cancellation token shared with the YAZA scheduler once cancellation is language-visible.
• Explicit DbCapabilities: transactions, savepoints, prepared statements, streaming results, returning rows, batch execution, and isolation modes.
• SQL DbValue types for NULL, Boolean, signed integer widths, Float64, Decimal, Text, Bytes, date/time, and JSON after those language value types exist.
• DbRow and column metadata preserve SQL NULL separately from empty text/zero. Conversion is checked and names the column on mismatch.
• DbTransaction is explicit initially. Scoped automatic rollback is added only when deterministic cleanup and cancellation semantics are implemented.
• Connection/session close is idempotent; timeout, cancellation, and protocol errors close or reset a session before it can re-enter a pool.

The shared contract is intentionally not a universal SQL grammar. Provider SQL dialect, placeholder rules, transaction behavior, data type mapping, and feature limits remain visible.

Engine roadmap

Managed services compatible with one of these protocols are targets for that provider; they are not extra engines unless their behavior requires a distinct implementation. CockroachDB/TiDB and compatible systems require their own protocol and SQL behavior conformance profiles.

SQLite requires an engine, not merely a reader/writer for its file format. Its storage format and locking/journaling details are part of correctness; the YAZA engine must be tested against independent fixtures and recovery cases before it claims compatibility.

Security requirements

• Verify TLS peer identity and certificates by default for remote connections; any insecure mode must be explicit, conspicuous, and unavailable by default in release builds.
• Never concatenate request values into SQL. Encode parameters separately and keep SQL templates static or validated by the query API.
• Bound frame sizes, row counts, column counts, nesting, string/byte lengths, connection counts, pool waits, retries, and total deadlines.
• Treat server error fields and database values as untrusted input. Logs omit credentials, query values, connection strings, and sensitive row data.
• Credentials should come from environment/secret capabilities and be redacted in diagnostics. Avoid copying secrets into long-lived values where possible; zeroize temporary buffers.
• Test authentication downgrade handling, malformed frames, truncated reads, cancellation, TLS name mismatch, invalid certificates, and pool return after failure.
• Publish supported server versions and authentication mechanisms. Fail clearly for unsupported negotiation.

Conformance and quality gates

Each provider needs:

1. Unit tests for codecs, bounds, malformed frames, NULL handling, types, and error mapping.
2. A local service/container integration matrix covering supported server versions, auth, TLS, DDL, CRUD, transactions, cancellation, and reconnect behavior.
3. Differential tests against the provider’s official client or reference test vectors where legally and technically appropriate.
4. Sanitizer/fuzz coverage for native parser/runtime boundaries and deterministic YAZA-level protocol tests.
5. A compatibility suite for migrations and schema snapshots; portable schemas use provider-neutral types plus explicit provider extensions.
6. Benchmarks measuring connection setup, prepared query latency, throughput, p99, RSS, and binary size, without unsupported superiority claims.
7. Release notes that list unsupported SQL/types/features and prove the integration matrix ran.

Standards and reference material

Implementation should follow published protocol and file-format specifications, with tests authored for YAZA and legal review for redistributable code and documentation:

• PostgreSQL frontend/backend protocol describes PostgreSQL startup, messages, authentication, queries, and transactions.
• MySQL client/server protocol is the primary MySQL wire-protocol reference.
• MariaDB client/server protocol documents MariaDB-specific and compatible protocol behavior.
• Microsoft TDS specification defines the SQL Server communication protocol.
• SQLite file format documents on-disk structures, but does not by itself define a complete SQL engine.
• Redis RESP specification documents the Redis serialization protocol.
• MongoDB wire protocol is a future reference only; its documentation page includes a non-commercial/share-alike license notice, so legal review is required before using its text or examples in a commercial implementation.

This proposal does not copy code from these projects. Reading a specification is not a substitute for checking its license, patent, trademark, and redistribution terms for each implementation and release.

First implementation slice

1. Implemented foundation: add the initial std/db provider/error vocabulary, checked shared DbValue/DbCapabilities contracts, stable error codes, PostgreSQL parameter/result adapters, and the first structured SELECT intermediate representation without claiming support for unimplemented language generics. tests/database_core.test.yz checks typed values, SQL NULL versus empty text, mismatched accessor rejection, capability dependency rules, PostgreSQL OID mapping, binary parameter preservation, known scalar decoding, and rejection of malformed/out-of-profile values. tests/database_query.test.yz checks bounded query generation, closed operators, SQL NULL handling, parameter separation, identifier injection rejection, and safe error mapping; the full suite passes 34/34, and the structured query test passes under ASan/UBSan. Connection lifecycle, cancellation, provider-neutral transactions, native query-block parsing, schema-to-shape mapping, multiple predicates/joins, and full conformance remain to be implemented.
2. Implemented: route Product CRUD through the YAZA repository adapter and pure PostgreSQL client; the earlier migration, CRUD, and restart-persistence smoke passed against local PostgreSQL 17.6.
3. Partially implemented: generated API startup/schema/readiness hooks use the pure-YAZA client, and generated builds have no libpq dependency. One live PostgreSQL 18.6 lifecycle smoke passes; typed row mapping, complete transaction cleanup, TLS/failure matrices, a broader server matrix, and automated live PostgreSQL CI remain required.
4. Add repeatable live PostgreSQL CI and require it as a release gate for the pure-YAZA driver and API lifecycle path.
5. Add SQLite only after the API profile has a stable provider contract; implement the embedded engine as its own staged product with an explicit SQL conformance target.

References

Related YAZA work: API completion plan, Appendix G implementation status, PostgreSQL schema snapshot format, and RFC 0004.

← Kembali ke semua dokumentasiVersi dan status rilis →