# Compiler implementation in YAZA

The compiler supports product `shape` declarations with built-in, other shape, or tagged-variant fields, positional constructors, field access, type-checked updates through mutable `var` values, and use as function parameter and return types. It emits dependency-ordered native C structs and checks constructor arity, field types, and field names. Cyclic direct by-value layouts, derives, and source-level ownership checking remain unimplemented. See [`examples/shapes.yz`](../examples/shapes.yz).

User-defined tagged `variant` declarations support unit cases and named typed payload fields. Construct unit cases as `Type::Case` and payload cases as `Type::Case(...)`; indented `match` blocks must cover every case exactly once and bind each payload field to an identifier or `_`. The compiler checks case names, constructor arity, payload types, match exhaustiveness, and binding scope. Variants can be used as function parameters/returns, shape fields, and `Option[T]`/`Result[T, E]` payloads. Payloads are copied into the bounded runtime arena. See [`examples/variants.yz`](../examples/variants.yz) and [`tests/variants.test.yz`](../tests/variants.test.yz).

The compiler supports limited `Option[T]` values over built-in, shape, and variant payloads. Use `some(value)` to construct a value, context-typed `none` for absence, `option_is_some` to check its tag, and `option_unwrap` to retrieve a present payload. Exhaustive indented block matches use `some(name):` and `none:`; duplicate or missing arms fail compilation. Nested options up to 16 levels are supported; shape and variant payloads are copied into the bounded runtime arena and validated by type tag when unwrapped. See [`examples/option.yz`](../examples/option.yz), [`examples/match.yz`](../examples/match.yz), and [`tests/nested_shapes.test.yz`](../tests/nested_shapes.test.yz).

It also supports bounded `Result[T, E]` values over built-in, shape, and variant payloads through `ok`, `err`, `result_is_ok`, `result_unwrap`, and `result_unwrap_error`. Type parameters must come from a concrete context; reading the wrong variant terminates with a runtime error. Exhaustive indented block matches use `ok(name):` and `err(name):`. Postfix `?` propagates errors through `let`, `var`, `set`, and `return` when the error types match or one explicit direct conversion is declared as `impl From[Source] for Target` with a single `fn from(value: Source) -> Target` method. Duplicate conversions for one source/target pair fail compilation. General traits/generics, conversion chaining, and arbitrary nested `?` expressions remain unimplemented. See [`examples/result.yz`](../examples/result.yz), [`examples/match.yz`](../examples/match.yz), [`examples/result_propagation.yz`](../examples/result_propagation.yz), and [`tests/from_conversion.test.yz`](../tests/from_conversion.test.yz).

The native runtime provides `log_event(level, event, key_value_pairs) -> Number` for bounded JSON-line application logs. `level` accepts `debug`, `info`, `warn`, or `error`; fields are alternating key/value strings in a `TextList`. Duplicate names, invalid UTF-8, oversized fields, and invalid levels are rejected. Sensitive field names such as `password`, `token`, `authorization`, `email`, and `database_url` are emitted as `[REDACTED]`. See [`examples/structured_logging.yz`](../examples/structured_logging.yz). This initial API has no level filtering, tracing context, or asynchronous sink.

`use "../std/fsroot"` provides a rooted filesystem capability through `fs_root_open`, `fs_root_close`, `fs_root_create_directory`, `fs_root_list_directory`, `fs_root_metadata`, `fs_root_read_bytes`, and `fs_root_replace_bytes`. Relative paths are capped at 4096 bytes and 128 components, and reject traversal, malformed UTF-8, control bytes, backslashes, and symlinks on every traversed component. Recursive directory creation uses owner-only permissions for new directories; directory listings are bytewise sorted and fail above 4,096 names or 512 KiB of combined filename bytes. `fs_root_metadata` returns a typed `FsMetadata` shape for regular files/directories, with file size, low nine permission bits, and modification time; it rejects symlinks and other special files. Reads and replacements are bounded to 64 MiB, and reads accept regular files only. Replacements write a private same-directory file and atomically rename it over the destination; this guarantees atomic visibility, not power-loss durability. Root handles are generation-checked, limited to 64 per process, and closed at runtime shutdown. See [`tests/fsroot.test.yz`](../tests/fsroot.test.yz), [`tests/fsroot_directory_smoke.yz`](../tests/fsroot_directory_smoke.yz), and [`tests/fsroot.c`](../tests/fsroot.c). Permission-changing APIs, broader owner/identity metadata, streams, and cross-platform conformance are incomplete; this is not a complete filesystem API or an OS sandbox.

The compiler source is being moved into `.yz` files:

- `lexer.yz` tokenizes identifiers, integer and scientific-notation decimal literals, strings, operators, punctuation, and comments.
- `expression_parser.yz` contains Pratt expression parsing and line/indentation handling.
- `native_cli.yz` is the self-hosted native compiler and CLI for the documented language slice. It handles typed functions and modules, block and single-line `if`/`elif`/`else`, mutable `var` plus `set`, `while`, `for` over `TextList`, supported concrete `List[T]` values, and half-open `Number` ranges, loop-scoped `break`/`continue`, nested early returns with a top-level fallback return, exhaustive numeric `return match` guards, local state, text and list operations, files, directories, command-line arguments, and child processes. A collection expression is evaluated once and the loop uses its starting length; values appended during iteration are visited in a later loop. See [`examples/for_loop.yz`](../examples/for_loop.yz), [`tests/for_loop.test.yz`](../tests/for_loop.test.yz), and [`tests/typed_list.test.yz`](../tests/typed_list.test.yz).

The native CLI also renders API, web, fullstack, and mobile starter templates, checks API interface/implementation signatures, compiles the API server subset, generates Postman collections and OpenAPI 3.1 documents from routes and controller request declarations, runs `.test.yz` or `_test.yz` tests with Boolean assertions, benchmarks repeated native process runs, compiles programs with length-carrying `Bytes`, OS-backed `secure_random_bytes`, and binary file/stdin operations, and captures bounded child-process stdout/stderr with timeouts and an optional explicit environment allowlist.

## Build the self-hosted macOS CLI

On Apple Silicon, install Apple's Command Line Tools so `clang` is available, then run:

```sh
scripts/bootstrap-macos-cli.sh
```

The script starts from an existing native YAZA compiler (`dist/yaza` or `YAZA_BOOTSTRAP`), rebuilds successive native stages, checks that the final two stages emit identical C, and copies the self-hosted result to `dist/yaza`. It does not use Python or another language as a bootstrap runtime.

```sh
dist/yaza --version
dist/yaza check examples/native_compiler_flow.yz
dist/yaza run examples/native_compiler_arguments.yz -- hello
dist/yaza new --api catalog-api
cd catalog-api
../dist/yaza generate
../dist/yaza run
```

`dist/yaza` is a standalone arm64 Mach-O executable. It links only to macOS `libSystem`; when it builds or runs YAZA programs, it invokes the system Clang compiler to compile generated C. Clang is therefore a development requirement for `yaza build` and `yaza run`, but not for CLI commands such as `new` and `generate`.

## API starter and current limits

`yaza new --api <name>` creates controller, service, repository, model, interface, route, and test folders with a Product Catalog example. From the API project root, `yaza check src/main.yz`, `yaza run`, and `yaza generate` compile and launch its native HTTP server or create its Postman collection. The current API compiler handles static JSON responses and executable controller-to-service-to-repository chains. Zero-argument route handlers may return `Text`, `Option[Text]`, `Shape`, `Option[Shape]`, `Result[Text, E]`, or `Result[Shape, E]`; returned text is validated as JSON, `none` maps to a safe 404, and errors are never serialized to clients. The template's known `ApiError` cases map to 400, 401, 403, 404, 409, 422, 429, 500, or 503 responses. `InvalidInput(field: Text)` includes bounded safe dotted paths and canonical array indexes from 0 through 63; invalid field text is omitted and OpenAPI publishes the same grammar. 401 includes `WWW-Authenticate`, and 429/503 include `Retry-After`. Unknown cases and other error types use a generic 500. OpenAPI emits the same known response codes, schemas, and headers. HTTP and generation smoke checks cover these contracts normally, and the HTTP smoke also passes under ASan/UBSan. The greeting route reads Text path/query/header values; a typed demonstration route reads `Number`, `Int32`, `Float64`, and `Boolean` values from path and query components as `Option[T]`. The runtime validates declared metadata before dispatch, converts defaults to their declared type, and rejects malformed or out-of-range input. The compiler checks interface method signatures against implementations.

API templates declare their module layers and permitted dependency edges in `[architecture]` within `yaza.toml`. Run `yaza arch check` to validate relative `use` imports in `src`; compilation from the project root applies the same check. The checker reports source locations for forbidden edges. It accepts single-line and multi-line arrays, normalizes `.`/`..` lexically inside `src`, does not inspect symlinks, and does not implement package dependencies or aliases.

The compiler supports `Float64` decimal and scientific-notation literals plus checked arithmetic. Explicit conversion helpers reject precision loss; this binary floating-point type is not suitable for money. The current macOS `dist/yaza` was rebuilt from this source through three native stages, and the final two stages emitted identical C. `Decimal`, `Money`, and temporal types remain future language/runtime work.

The HTTP server is an early implementation: executable GET and POST controller actions can read declared path/query values, scalar JSON body fields, and nested shape request bodies through typed `Option[T]` accessors. A controller can declare `body payload: CreateProduct` plus an `example json` payload; the handler reads it with `let payload: Option[CreateProduct] = http_body_shape("payload")`. The example is documentation/generator metadata, not a request default. Nested shape fields may use Text, Number, Int32, Float64, Boolean, `TextList` (a JSON array of at most 64 strings, each at most 1023 UTF-8 bytes), optional scalar fields such as `Option[Text]`, optional lists, or optional nested shapes. Omitted or JSON `null` optional properties decode as `none`; unknown, missing-required, malformed, or wrong-typed values are rejected before dispatch. Shape fields may declare numeric `min`/`max`, Text `min_length`/`max_length`, or TextList `min_items`/`max_items` rules; text lengths count Unicode code points, and list limits accept integer values from 0 through 64. Each Text value has a separate 1023-byte decoder cap, exposed as `x-yaza-maxUtf8Bytes`; OpenAPI emits the declared list minimum and maximum while retaining the runtime cap. Constraints run during body decoding. OpenAPI output marks optional fields nullable and does not list them as required. Tagged variants and map fields remain unsupported. The Product Catalog template executes controller → service → repository functions. Product `create`, `replace`, `patch`, and `find` return `Result[ProductRecord, ApiError]` after strict `json_shape(Text)` decoding; `list` still returns JSON `Text`; delete returns a typed `ProductDeleteOutcome` while HTTP 204 suppresses its body. Storage is adapter-backed and this is not generalized typed repository dispatch. The server supports an optional shared Bearer token through `YAZA_API_TOKEN` and exact-origin CORS through `YAZA_CORS_ORIGIN`; it supports route-scoped JWT HS256 verification and verified string-claim access, but not token issuance, centralized roles, arbitrary user-defined middleware chains, HTTPS, reusable typed SQL drivers, production-grade routing limits, or web/mobile rendering. Route/group auth and rate-limit policies are built-in and reflected in OpenAPI where applicable.

## macOS installer

Build the `.pkg` after the CLI:

```sh
scripts/build-macos-pkg.sh
```

It installs the executable at `/usr/local/bin/yaza`, support source at `/usr/local/share/yaza/runtime`, and starter templates at `/usr/local/share/yaza/templates`. The package is unsigned. This managed build host adds AppleDouble provenance records despite stripping metadata, so the generated package is marked provisional and should be rebuilt and install-tested on a clean Mac before distribution. Linux and Windows need separate builds and installers.

The shared system-call implementation lives in [`../runtime/yaza_runtime.c.inc`](../runtime/yaza_runtime.c.inc). It is native C support linked into generated programs; it is not an interpreter or bundled foreign-language runtime.

## Bounded TTL cache

Import `std/cache/ttl` and use `ttl_cache_get(key) -> Option[Text]`, `ttl_cache_set(key, value, ttl_ms) -> Boolean`, `ttl_cache_remove(key) -> Boolean`, or `ttl_cache_clear() -> Number`. The cache is local to one process and volatile. It supports up to 128 entries, keys up to 255 bytes, values up to 4095 bytes, and TTLs from 1 ms through seven days; it expires by monotonic time and evicts the least-recently-used live entry when full. Do not store secrets or authorization decisions in this cache. It does not synchronize entries between processes or machines.

## JWT route-authentication profile

Declare `auth jwt` on a route or route group. Configure `YAZA_JWT_SECRET`, `YAZA_JWT_ISSUER`, and `YAZA_JWT_AUDIENCE`; generate the secret with a CSPRNG, for example `openssl rand -hex 32`, rather than choosing a human password. The verifier accepts compact JWS with HS256 only, requires `typ: JWT`, exact `iss` and an `aud` string or non-empty string array containing the configured value, and integer `exp`; optional integer `nbf` and `iat` must not be in the future. It verifies the signature before claim parsing, rejects padded/non-canonical base64url and unsupported JOSE headers, and returns the safe `token_expired` code only for a correctly signed, otherwise valid expired token. JWT-only non-loopback binding requires at least one JWT route and explicit auth policy on every declared route. Inside a handler for a matched JWT-protected route, `http_auth_claim_text("sub") -> Option[Text]` reads a verified top-level string claim; any other top-level string claim can be requested by name. Missing or non-string claims and calls outside a verified JWT handler return `none`; decoded values are limited to 1023 UTF-8 bytes, and the request-scoped verified payload is wiped after dispatch. Shared Bearer and public routes cannot supply an identity through request headers. This is claim access, not a typed principal or an authorization decision: YAZA does not issue tokens, support OIDC/JWKS or asymmetric keys, provide centralized roles or ownership policy, or implement sessions, refresh, revocation, or key rotation. See the [Appendix H security status](../docs/appendix-h-security-status.md) for remaining gaps.