RFC 0005: Structured Concurrency and HTTP Runtime

Status: Accepted Areas: Appendix G.14, G.15, G.17, G.18; Appendix H.7, H.8 Depends on: RFCs 0001, 0002, and 0004

Decision: Accepted by the project owner on 2026-10-23. This approves the design direction; implementation and conformance evidence remain outstanding.

Implementation status: Partial. The native scheduler has a deterministic FIFO ready queue, 256 generational task slots, indexed deadline-heap timers, wakeups, child joins, cancellation/failure propagation, and per-task bounded arenas. The HTTP server uses it for one request-reader task per connection. A separate YAZA-owned bounded handler pool now executes generated API controller/service/repository work and PostgreSQL-backed readiness: 1–32 OS threads (default 4), at most one in-flight job per accepted connection, copied request context, per-job request arenas, bounded owned results, and a pipe-based completion handoff. YAZA_API_MAX_IN_FLIGHT independently lowers the combined queued, executing, and completed-but-undelivered job cap from the connection capacity; it defaults to that capacity, rejects invalid configuration before listen, and returns a metered 503 on saturation. The event loop remains the sole owner of sockets and response writes. API work observes cancellation and absolute deadlines at dispatch and after the handler returns; http_request_cancelled() -> Boolean also exposes the per-job cancellation flag so long-running YAZA code can poll cooperatively; expired jobs map to HTTP 504 and increment the timeout metric. PostgreSQL pool lease waits now check the same request cancellation flag at bounded intervals of at most 50 ms; queries and in-progress socket calls still rely on their own timeouts. The loopback concurrency harness holds two route handlers at once and a third in the bounded queue; it checks the active/queued/in-flight gauges, rejection of another handler, operational health/metrics availability, readiness isolation, timeout responses and metrics, and shutdown. The concurrency harness and PostgreSQL pool cancellation harness pass normal, ASan/UBSan, and ThreadSanitizer runs on macOS. This proves the current bounded API worker slice on macOS, not language-level async or production performance. The scheduler still has one execution thread: YAZA source has no async/await, task scopes, or work stealing; PostgreSQL and other blocking calls occupy a worker until they return, and arbitrary handlers cannot be forcibly preempted. Linux epoll remains source-only. HTTP/1.1 has strict bounded chunked framing, keep-alive and ordered pipelining; responses are capped and flushed through writable readiness. Outbound HTTP, interoperable inbound TLS, HTTP/2, Linux end-to-end evidence, and the RFC's full structured-concurrency acceptance criteria remain incomplete. The local scheduler microbenchmark varied by more than the 5% evidence threshold and does not support competitor performance claims.

2026-10-28 HTTP method handling update: Runtime 405 responses now include an Allow header listing the registered methods for the matched path; built-in health, readiness, and metrics routes report GET, HEAD. Every GET route accepts HEAD through the GET handler and inherits its route policies, preserves GET representation headers including Content-Length, and sends no body bytes. OpenAPI and Postman include these implicit HEAD operations. Raw TCP, route middleware, generator, normal, and ASan/UBSan smoke checks pass on macOS. Linux verification and maximal header-boundary tests remain open. This does not close the RFC: handlers and database operations are synchronous within bounded worker threads, while language-visible async tasks, multi-worker scheduling, TLS, HTTP/2, and Linux evidence remain outstanding.

2026-10-28 keep-alive dispatch latency update: The event loop now skips its 100 ms fallback wait when the scheduler has already parsed a request and marked it ready. Before this change, repeated keep-alive requests could be delayed despite having no I/O left to wait for. Two local macOS/arm64 YAZA-only runs at concurrency 32 reduced /healthz p50 from 104.6–106.7 ms to 2.05–2.07 ms (1,000 requests per run); all requests succeeded. This targeted regression measurement does not establish cross-runtime performance or close the RFC's async implementation gaps.

2026-10-28 API cancellation-aware I/O update: PostgreSQL pool lease waits and native TCP/TLS readiness waits now observe the active API request cancellation flag in bounded slices of at most 50 ms. The pool and socket transport harnesses pass normally and under ASan/UBSan; both pass ThreadSanitizer on macOS. A cancelled blocking socket read returns promptly instead of consuming its full configured timeout. This does not turn these operations into suspendable tasks: DNS resolution, PostgreSQL protocol calls, and handlers that do not cooperate can still occupy a worker. Linux behavior and remote TLS cancellation remain unverified.

2026-10-28 HTTP EOF framing update: Raw TCP integration now verifies normal completion followed by client SHUT_WR, plus EOF during incomplete request headers and a declared request body. The complete request still receives 200; partial requests receive 400. The harness checks the exact 400 response metric, corresponding access-log records, and connection cleanup, passing normally, under ASan/UBSan, and under ThreadSanitizer on macOS. Linux behavior remains unverified.

Scheduler decision: The current poll loop is a bounded first step, not YAZA's final async engine. The target is a YAZA-owned structured-concurrency, M:N task runtime: many lightweight YAZA tasks share a bounded number of OS worker threads. Async I/O completion/readiness wakes suspended tasks; it does not dedicate a thread to each socket. Each worker has a runnable queue, with bounded cross-worker injection and work stealing after the single-worker scheduler is correct. A task runs cooperatively until it awaits, yields, completes, or reaches its execution budget; the scheduler returns to I/O and timer processing between budgets. Blocking work uses a separate bounded blocking pool. The internal deterministic scheduler and readiness-to-task C bridge are implemented, and the bridge now drives HTTP request-reader tasks. The API handler/readiness worker pool and completion bridge are now implemented. Language-visible async syntax, async route handlers, multi-worker task scheduling, and remaining nonblocking I/O integration are outstanding.

Summary

Define bounded asynchronous work and a production-oriented HTTP/1.1 runtime. Prefer structured task lifetimes and explicit cancellation over detached background work. Begin with one event loop per process and a bounded worker pool for blocking operations; do not promise unlimited concurrency.

Motivation

Before the bounded readiness loop, one slow client blocked other requests. The current API adds nonblocking reads/writes, cooperative cancellation at task and worker boundaries, and a bounded API worker pool; source-level async tasks, preemption of arbitrary handlers, TLS, HTTP/2, and general task APIs remain absent.

Proposal

1. An async task belongs to a parent scope. Child tasks must complete or be cancelled before the scope returns; detached tasks require an explicit privileged API and lifecycle owner.
2. Cancellation is cooperative at defined suspension points and is represented as a typed error/result. Cleanup and destructors run on cancellation.
3. The server uses a platform backend (kqueue on macOS/BSD, epoll on Linux, IOCP on Windows) behind one event API. Backend choice does not change observable ordering guarantees.
4. The scheduler has bounded runnable queues, timer storage, connection counts, request queues, request/body sizes, idle and total deadlines, and blocking-pool queues. Queue saturation applies backpressure or returns a defined overload result; it never creates unbounded tasks or threads.
5. The public model is structured async functions and await over typed Task[T]/Result[T, E], with task scopes, join, select, deadlines, and cooperative cancellation. Child tasks must finish or be cancelled and joined before the scope returns. Detached tasks require an explicit lifecycle owner. Cancellation is observable as a typed outcome; cleanup and destructors run on cancellation.
6. Readiness backends use nonblocking sockets and drain readable/writable work under a fairness budget. Linux starts with level-triggered epoll; edge-triggered mode is deferred until drain-to-EAGAIN behavior has dedicated conformance tests. macOS/BSD use kqueue. Windows uses overlapped operations and IOCP completions, represented through the same task-wakeup contract. Backends report readiness/completion; they do not expose OS-specific handles in ordinary YAZA APIs.
7. One binary min-heap stores task deadlines and timers in the initial scheduler. Replace it with a hierarchical timer wheel only if benchmark evidence shows timer management is a bottleneck at target timer counts.
8. Work stealing is added only after deterministic single-worker scheduling, cancellation, and resource-lifetime tests pass. Task execution is cooperative, but a task that does not await/yield is bounded by an instruction/time budget and diagnosed; CPU-heavy operations use the bounded blocking/compute pool so they cannot starve socket progress.
9. Blocking filesystem, DNS, and unavoidable blocking foreign calls use a separate bounded pool with admission limits, deadlines, and cancellation semantics. YAZA's PostgreSQL wire transport and outbound HTTP are designed as nonblocking state machines instead of occupying an async worker while waiting on sockets. A cancelled blocking call may not be forcibly stopped; its resources remain owned until it returns.
10. Request/task memory is scoped: a request arena cannot be reset while any child task can access it. Cross-task sharing uses immutable values or explicitly safe reference-counted handles; mutable state requires synchronization. This is a prerequisite for parallel scheduling.
11. HTTP/1.1 supports correct keep-alive framing and chunked transfer decoding, with strict rejection of request-smuggling ambiguity. Per-connection and per-request timeouts prevent slowloris resource pinning.
12. HTTP/2 is a later optional module with its own flow-control and header-compression review. TLS must use a maintained, reviewed implementation or OS facility; YAZA will not implement cryptographic primitives or TLS itself.
13. The server drains accepted work on shutdown up to a configured deadline, then cancels remaining tasks, closes sockets, and releases database resources in a documented order.
14. Outbound HTTP is a separate client with connect/read/total timeouts, redirect policy, DNS/IP checks, request/response size limits, and cancellation. SSRF checks must account for DNS rebinding and IPv4/IPv6 representations.
15. A request boundary catches only defined recoverable panics/failures, logs a redacted correlation ID, and returns a generic 500 response. Process-corrupting faults are not recovered. Recovery must not convert a partially committed transaction or unknown state into a success response.

Illustrative task syntax, not final grammar:

async fn refresh_catalog() -> Result[Unit, AppError]:
    task group:
        let price_task = spawn load_prices()
        let stock_task = spawn load_stock()
        update_catalog(await price_task?, await stock_task?)?

Security considerations

Concurrency makes shared mutable state and resource exhaustion more likely. Safe shared state must obey RFC 0002. Limits must be enforced before allocation and across the entire request lifecycle. TLS and certificate validation must use a supported trust store and secure defaults.

Platform references

• Linux epoll documents level-triggered and edge-triggered readiness; the initial backend deliberately chooses level-triggered behavior, while edge-triggered use must drain nonblocking I/O to EAGAIN correctly: epoll(7).
• Windows I/O completion ports queue asynchronous completions and provide a concurrency-controlled worker model, which maps to the same YAZA task-wakeup contract: I/O Completion Ports.

Staged implementation

1. The bounded single-worker scheduler core is implemented and has deterministic tests plus a native microbenchmark harness. Task handles encode a slot and generation, so active lookup and slot allocation/reuse are O(1) and old handles are rejected after reuse. Each task owns a bounded arena that survives suspension and is reclaimed on handle reaping. Next define typed language-level Task, scope, cancellation, and deadline semantics and safe cross-task ownership before connecting the scheduler to generated code.
2. The server uses a platform I/O driver and one bounded scheduler reader task per connection; readiness events wake tasks before they continue incremental reads. The keep-alive fixture verifies that a partial pipelined request yields while another client receives a health response, then resumes when more bytes arrive. Route dispatch now runs on the bounded API worker pool; the event loop retains socket ownership and bounded nonblocking response writes. Add Linux epoll build/runtime CI next.
3. The first thread-isolation slice is implemented: arena selection and ambient handler state are thread-local, shared program-arena list/accounting mutations are locked, request context and verified claims are copied into bounded jobs, and the event loop retains socket/connection/write ownership. Continue auditing every shared metric, cache, rate-limit table, and database resource before expanding worker access.
4. The API-facing bounded worker-pool slice is implemented: 1–32 workers (default 4), a queue capped by connection capacity and optionally lowered by YAZA_API_MAX_IN_FLIGHT, deadline/cancellation metadata, metered 503 overload behavior, an owned completion queue and event-loop wakeup. Invalid in-flight caps fail before listen. Existing synchronous controller/service/repository and PostgreSQL operations run in workers, which do not touch connection state or client sockets. Pool lease waits and native TCP/TLS readiness waits observe request cancellation on bounded intervals of at most 50 ms. Normal, ASan/UBSan, and ThreadSanitizer evidence covers two simultaneously blocked handlers plus one queued job, rejection of the next job, operational endpoint responsiveness, active/queued/in-flight and rejection gauges, health/readiness isolation, 504 timeout accounting, shutdown, and cancelled PostgreSQL-pool and socket waits. The http_request_cancelled() -> Boolean builtin lets long-running handler loops poll the per-job flag; cancellation is opt-in and cannot preempt a loop, synchronous DNS lookup, or foreign/blocking call that does not cooperate. Remaining work includes source-level async/await, automatic cancellation safepoints, live PostgreSQL query overlap and load behavior, soak behavior, and Linux behavior.
5. Define and expose typed language Task[T]/Result[T,E], structured request scopes, async/await, cancellation, deadlines, joins, and cleanup. Generated API handlers can then suspend and resume without blocking a worker while waiting on YAZA async operations. Cross-task memory ownership and compiler diagnostics are required before general task spawning.
6. Convert the YAZA PostgreSQL transport and outbound HTTP connect/read/write waits into readiness-driven state machines. Retain the blocking pool for filesystem, DNS, and unavoidable blocking calls. Add bounded queue, slow database, cancellation, overload, shutdown, race, sanitizer, load, and soak evidence.
7. Add multi-worker scheduler execution and work stealing only after deterministic single-worker semantics, cancellation, race checks, task-local memory safety, fairness, and bounded memory are proven. Keep each connection and its response state owned by one event loop; worker completion must cross an explicit synchronized handoff.
8. macOS kqueue and Linux level-triggered epoll implementations sit behind the internal readiness API; only the macOS path has been exercised. Add Linux build/runtime CI, then Windows IOCP behind the same async API. Keep platform-specific code in the native runtime; YAZA applications use one portable interface.
9. Add TLS integration, observability, and eventually optional HTTP/2 after HTTP/1.1 conformance and resource limits are independently reviewed.

API priority update (2026-10-28): Asynchronous execution and multi-threading are part of the API completion path. Safe concurrent execution of existing handlers through bounded workers is implemented and has normal, ASan/UBSan, and ThreadSanitizer evidence on macOS. The next API concurrency work is queue/load behavior, live PostgreSQL overlap, cancellation/shutdown limits, Linux verification, then language-visible structured async and multi-worker scheduling with explicit ownership. Unbounded threads/tasks and worker access to event-loop-owned sockets remain forbidden.

Acceptance criteria

• A slow-client test proves one stalled connection does not stop unrelated requests.
• Scheduler stress tests prove runnable-queue, timer, connection, blocking-pool, and memory bounds under saturation; fairness tests prove a continuously runnable task cannot indefinitely starve I/O or timers.
• The internal scheduler core has deterministic coverage for FIFO fairness, ordered timers, wakeups, child join and auto-join, failure propagation, cancellation delivery/cleanup, capacity bounds, and safe handle reaping; normal and ASan/UBSan runs pass.
• Scheduler performance reports include FIFO dispatch, task spawn/completion/reaping, timer wakeups, and external wakeups. They report operations per second with the target/toolchain and workload parameters; they are not cross-runtime superiority claims.
• Structured cancellation tests prove child tasks finish or release sockets, request arenas, and database leases on success, error, timeout, and shutdown.
• Multi-worker tests pass a race detector or equivalent concurrency instrumentation and show no task can outlive its memory scope.
• Load tests verify fixed upper bounds on connections, queued work, memory, and shutdown time.
• An API integration workload holds one controller/database operation behind a controllable delay while health and unrelated routes continue to meet their response deadline; it also proves the configured worker and queue limits and the documented overload response under saturation.
• Cancellation, timeout, and shutdown tests prove that worker completion cannot access a reused connection slot, request arena, JWT claims, response buffer, or released database lease.
• HTTP parser differential fuzzing covers framing, keep-alive, chunking, and request-smuggling cases.
• Cancellation tests prove no leaked sockets, tasks, or database leases on success, error, timeout, and shutdown.
• TLS tests verify certificate and hostname validation against a maintained test suite.

Remaining tuning questions

• Exact connection, ready-queue, worker, and timer defaults must be selected from repeatable load tests for each runtime profile; configuration must have safe upper bounds.
• Select the maintained TLS provider or OS TLS facility per supported platform after compatibility and update-policy review. Cryptographic code remains outside the YAZA runtime.

← Kembali ke semua dokumentasiVersi dan status rilis →