Changelog

All notable changes to SQL Anywhere are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

0.11.0 - 2026-10-30

Vector search becomes something to tune with numbers instead of guesses. search_l is set per query, sqlanywhere::recall measures what it buys against an exact search over the table's own rows, and tune_compression finds the smallest index that still reaches a recall target; hybrid_top_k weights one ranking over the other. sqld gets a Grafana dashboard with every metric it exports described, and the storage contracts a Python client. One fix matters in production: two sqld can no longer serve the same data directory, which made the replication log's frame numbers diverge.

No file-format changes. vector_top_k and hybrid_top_k called as before return the rows they did, except that a vector_top_k asking for more rows than the index's search_l now gets k of them instead of search_l + 1. A second sqld on a --db-path that is in use exits at startup; on a filesystem without flock the server warns and runs unlocked, as before. The sqlanywhere_server_user_http_fault metric's label is renamed from protcol to protocol, so a query that grouped by the old name needs the new one.

Added

• search_l per query, and the recall it buys. vector_top_k(index, vector, k, search_l) overrides the index's search_l for one search, so recall is traded for speed per query without rebuilding the index; hybrid_top_k takes it as its seventh argument. sqlanywhere::recall measures recall@k against an exact search over a sample of the table's own rows, and recall::tune returns the smallest search_l that reaches a target, with the recall at every step it tried. On 2000 vectors, recall@10 goes from 0.31 at search_l 10 to 0.97 at 400. See docs/VECTOR_SEARCH.md.
• Weighting one ranking over the other in hybrid_top_k. Its eighth and ninth arguments, vector_weight and fts_weight (both 1 unless given), scale each ranker's share of a row's score, weight/(rrf_k + rank): where exact terms matter more than meaning, weight the keywords, and the other way round. A weight of 0 leaves its ranker out, as a NULL vector or query does. The result is the RRF recipe's with the same weights, shown equal on a 300-row corpus. rrf_k now also refuses infinity. See docs/VECTOR_SEARCH.md.
• A Grafana dashboard for sqld, and every metric described. tools/observability has a dashboard (statements, errors, per-namespace load and size, WAL checkpoints, the tokio runtime) and a compose file that runs sqld, Prometheus and Grafana with it loaded: docker compose -f tools/observability/docker-compose.yml up. docs/OBSERVABILITY.md describes every metric, with its type, labels and unit. A test scrapes a real sqld and fails when the dashboard names a metric it does not export, or it exports one the page does not describe.
• How small a vector index can be, measured. recall::tune_compression(conn, index, k, target, sample, candidates) builds the index again once per compress_neighbors candidate, with the options its CREATE INDEX gave, measures each build's size and recall@k, and returns the smallest that reaches the target. The builds are made in a savepoint and rolled back, so the database is left as it was, inside a transaction of the caller's too. On 2000 vectors: 1.44 MB at recall 1.0 uncompressed, 0.80 MB at 0.999 with float8, 0.51 MB at 0.91 with float1bit. See docs/VECTOR_SEARCH.md.
• A Python client for the storage contracts. cargo xtask gen-contracts now also writes clients/python/sqlanywhere_contracts, the queue, cache and pub/sub contracts as typed Python functions (queue.claim(db, visibility=30, queue="mail")), on the standard library's sqlite3 or on sqld over HTTP, with no dependencies. An operation of several statements is one Hrana batch over HTTP, each step run only if the one before succeeded. Tested on Python 3.9 and 3.13 and against sqld; see clients/python.

Fixed

• vector_index_stats found no index after a ROLLBACK TO that undid a schema change. Rolling back a schema change (a CREATE INDEX, say) to a savepoint resets the connection's schema, and only a statement that reads a table loads it again; SELECT vector_index_stats('idx') reads none, so it said "not a vector index" about an index that was there, until some other statement ran. It now loads the schema itself.
• vector_top_k returned fewer than k rows when k exceeded search_l. The k best are picked from the search's candidates, of which there are search_l: an index with the default 200 asked for 300 neighbours returned 201, without a word. search_l is now raised to k for that search.
• Two sqld could serve the same data directory. Nothing stopped a second server started on the same --db-path: both appended to each namespace's replication log and wrote its WAL and bottomless backup, so the log's frame numbers diverged under replicas and backups. A server now takes an exclusive flock on <db-path>/.lock before it writes anything there, and a second one exits naming the process that holds it. The lock is the kernel's, so a crashed server leaves none behind.
• ATTACH in a second server in one process found the first server's namespaces. The function a connection uses to find the namespace it attaches was cached process-wide, holding the first NamespaceStore the process made, so every later server (tests, anything embedding sqld) resolved ATTACH against it: "Namespace x doesn't exist". It also kept that first store alive for good. Each store now makes its own, holding the store weakly.
• sqlanywhere_server_user_http_fault labelled its protocol protcol. It is protocol now, as on sqlanywhere_server_user_http_response; a query grouping by the old label needs the new name.

0.10.0 - 2026-10-29

Search, live results and branches reach the server. Hybrid retrieval is one table-valued function, hybrid_top_k, instead of a 15-line query; a client with no database file of its own waits for sqld to push the commits that change a live query's result; a sqld namespace is branched with one admin call and merged back three ways, row by row; and the storage contracts get a TypeScript client. Three fixes matter in production: sqld no longer aborts when a transaction writes to main and reads an attached namespace, commit announcements no longer go out before the commit is done, and a bottomless backup verifies on a machine of either byte order.

No file-format changes. A bottomless generation's .meta object gains four bytes after its checksum (the WAL's byte order); older versions read the first twelve and ignore them, and this one reads older .meta objects as before. vector_top_k and hybrid_top_k return exactly the rows they did; nothing in an existing database changes.

Added

• A TypeScript client for the contracts. cargo xtask gen-contracts now also writes clients/typescript/src: one module per contract (queue, cache, pubsub) with install, the statements as constants and one typed async function per operation, taking an object of the operation's parameters (queue.claim(db, { visibility, queue })). It has no runtime dependencies: operations run on a Driver, with adapters for @libsql/client (a sqld server over HTTP, or a file) and node:sqlite. Operations of several statements run in one transaction through Driver.batch, which works over HTTP where separate BEGIN and COMMIT requests would not; payloads are blobs, passed as bytes or strings and returned as Uint8Array. Tested on both adapters in CI on Node 22 and 24, and against a running sqld with SQLD_URL; CI fails when a contract changes without the client. See clients/typescript/README.md.

• Branches of a sqld namespace, merged three ways. POST /v1/namespaces/:ns/branch/:name forks the namespace twice: the branch, writable, and name@base, the same moment read-only, both attachable. sqlanywhere::merge3::Merge3 then compares, from a connection to the live namespace, what the branch changed since the base and what the live side changed meanwhile, and merges in one IMMEDIATE transaction: rows only one side changed are applied, updates to different columns of a row both stand, and rows changed differently on both sides are conflicts (refuse, keep main, or take the branch). Ordinary tables, no CRDT, no history; the rows are copied with SQL from the attached branch. Internal, virtual and shadow tables are left out, and skip leaves out more; schema changes refuse the merge. Works on local files too. See docs/BRANCHES.md.

• Live queries over sqld: LiveQuery::wait. A client connected to sqld over HTTP, with no database file of its own, waits for the server to announce a commit that changes its result and gets the diff, without polling. sqld pushes an event per commit on GET /beta/listen (server-sent events); a live query listens to sqlanywhere_live_tables, which the version triggers of its tracked tables write, so one stream covers all its tables and WITHOUT ROWID tables are seen too. Each announcement is followed by a refresh, which re-runs the query only when one of its own tables moved, and commits that leave the result alone are waited through. The first wait catches up on commits made before it listened; a dropped wait keeps the subscription. Every table the query reads must be live::tracked, and the connection must reach the primary. See docs/LIVE_QUERIES.md.

• Hybrid search in one call: hybrid_top_k. hybrid_top_k(vector_index, fts_table, vector, query, k [, rrf_k]) is a table-valued function that fuses a DiskANN vector ranking and an FTS5 keyword ranking with Reciprocal Rank Fusion: each ranker contributes its top k, and the k best fused scores come back with id, score, vector_rank and fts_rank. It replaces the 15-line CTE the docs recommended, and returns exactly what that query does (tested equal, scores included, on a 300-row corpus). rrf_k defaults to 60; a NULL vector or query leaves that ranker out; the vector and query can come from another table, a fused top-k per row; and the DiskANN nodes it visits count as rows read, as for vector_top_k. The local_rag example, README and docs/VECTOR_SEARCH.md use it.

Fixed

• A transaction that wrote to main and read an attached namespace aborted sqld. Each database a connection has open, its own and every attached one, has a WAL that goes through the connection's write slot under the connection's id. When the attached database ended its read transaction it gave back the slot that main's write was holding, and main's commit then found no slot to release: an unreachable! in a function that cannot unwind, so the process aborted. INSERT INTO main.t SELECT … FROM attached.t in a transaction was enough. Only the WAL that took the slot now gives it back.

• A bottomless backup could only be verified on a machine of the byte order that made it. SQLite writes a WAL's checksums in its writer's byte order, which the WAL's magic number records; restoring verified them in the restoring machine's order instead, so a backup from a big-endian machine failed verification on a little-endian one and the other way round. Each generation's .meta object now records the order (four bytes after the checksum), and a restore verifies in it; older .meta objects are read as before. The checksum also read a byte slice as u32s through an unsafe cast, undefined behaviour when the slice was not 4-byte aligned, and the restore buffer was uninitialised memory handed out as &mut [u8]; both are safe now. This was the last P0 in docs/TECH_DEBT.md.

• /beta/listen announced commits before they were done, and could miss the first one. The announcement went out from SQLite's commit hook, which runs before the commit completes, so a listener that read straight away could find nothing new; it is now sent once the committing statement has finished. And the subscription was made when the response body was first polled, after the response had gone out; it is now made before. A replica without a primary URL accepted listeners it could never tell about a commit (writes reach it as frames, past the hooks); it now refuses them with database is not a primary.

0.9.0 - 2026-10-29

Every idea from the review after 0.7.0, built and released. The present can be compared with the past, and an embedding says how far its meaning moved; a vector can wait for rows to come near it instead of being searched for; a large database forks from a copy of its file, and a branch can hold only the rows a lens selects, chosen by a predicate or by meaning; a DiskANN index can be looked inside and its recall measured in SQL; and the storage contracts get a PHP client. Two fixes matter in production: a primary encrypted at rest can restart after a crash that left frames in its WAL, and vector_top_k takes its query vector from another table, a top-k per row.

No file-format changes. New tables appear only when a feature is installed (sqlanywhere_interests, sqlanywhere_branch in a branch). The cache contract's Increment now names its column value; positional readers see no difference.

Added

• Partial branches: a lens chooses the rows a branch holds. A branch::Lens names, per tracked table, a SQL predicate over its rows, relational (project_id = ?) or semantic (id IN (SELECT id FROM vector_top_k(…))); fork_lens makes a branch with only those rows, each with its whole history, and pull_lens keeps the held rows in step with main (deletes included) and brings in rows that match now, whole. Writes on the branch diff and merge as on any branch, and a merge never deletes what the branch does not hold. The first step towards selective replication. See docs/BRANCHES.md.
• branch::fork_file(&main, path): fork a large database by copying the file. VACUUM INTO takes one consistent page-level copy, and the copy is given an identity of its own before cr-sqlite opens it: main's clock rows move from site ordinal 0 (which means "this database") to an ordinal for main's site id, ordinal 0 gets a new site id, and the branch bookkeeping records the version main was at in the snapshot. Without that, the first diff of a copied file would offer every row main ever wrote. diff, merge and pull treat it exactly like a fork; 20,000 rows fork in 16 ms instead of 309 ms. See docs/BRANCHES.md.
• A PHP client for the storage contracts. cargo xtask gen-contracts now also writes clients/php/src/: Queue, Cache and PubSub classes on PDO with install() and one typed static method per operation, rows as arrays by column name, multi-statement operations in one transaction (BEGIN IMMEDIATE where the contract says so) that rolls back on failure, and payloads bound as blobs. It is generated from the same documents as the Rust API, so generated_contracts_are_fresh fails CI when it falls behind, and a new workflow runs its checks on PHP 8.1 and 8.4. A Composer package (sqlanywhere/contracts, PSR-4). See clients/php.
• vector_index_stats('index'). A SQL function in the engine that describes a DiskANN index as JSON: its parameters with defaults filled in (metric, vector type, dimensions, compress_neighbors, block size, maximum neighbours, alpha, search_l, insert_l), how many nodes it holds, how many neighbours they have on average, how many have none or are full, and how many bytes its node blocks take. Works from every client and sqld. With rows_read (every node a search visits counts as a row read) and a recall@k query in plain SQL, this is the tuning view the roadmap asked for. See docs/VECTOR_SEARCH.md.
• sqlanywhere::compare: what changed between two versions of a table, and how far its embeddings moved. compare::table(&conn, table, before, after) compares a table with itself in an attached schema (a namespace at a moment from /at, an older copy of the file) and returns every row inserted, updated or deleted, the columns that differ with before and after, and the cosine distance for vector columns; TableDiff::moved(column) lists rows by how far their vector moved. Rows match by primary key or rowid, columns by name. No CRRs or history needed; works on a local file and, inside a transaction, on sqld. See docs/TIME_TRAVEL.md.
• Interests: standing vector queries. Interests::on(table, column) registers vectors with a cosine distance (subscribe, or subscribe_text through an embedder), and every write that puts a row's vector within reach of one publishes an event on its channel through the pub/sub contract: {"table", "column", "rowid", "interest", "distance"}, nearest first. Two triggers do the matching, so it works whoever writes the vector (the application, an embedding column worker, a branch merge, an embedded replica's primary), in the same transaction as the write; a value that is not a vector of the right size matches nothing rather than failing the write. matching(rowid) shows what a row matches now. See docs/INTERESTS.md.

Changed

• The cache contract's Increment names its column value. It returned CAST(value AS INTEGER) unaliased, which a client reading columns by name saw as that expression; the result is unchanged.

Fixed

• A primary encrypted at rest could not restart after a crash. The checkpoint sqld runs at startup, before recovery rebuilds the replication log and before bottomless starts, opened the database without its cipher and key; with frames left in the WAL it failed with "file is not a database". It now opens it like every other connection to an encrypted database.
• vector_top_k could not take its query vector from another table. FROM q, vector_top_k('idx', q.v, 10) and the same as a correlated subquery failed with "xBestIndex malfunction": the virtual table accepted a plan in which the vector argument was not available yet. It now refuses such a plan, so SQLite puts the other table first, and a top-k per row works.

0.8.0 - 2026-10-28

Branches and live queries, made safe to leave running on their own. Jobs a branch enqueues and events it publishes now wait for the merge and arrive on the parent exactly once, where before a merge silently left them behind; and merge_if lets a policy merge an agent's harmless changes by itself while holding the rest, with the reasons, for a person. A live query over tracked tables re-runs only for writes to those tables, on embedded replicas too, so a busy queue in the same file no longer makes every live query re-run.

No file-format changes. BranchDiff and MergeReport gain fields (the branch module is experimental), and a branch forked by 0.7.0 starts carrying jobs and events after its next merge.

Added

• Branches carry their jobs and events, and can merge on a policy. A job a branch enqueues or an event it publishes on the storage contracts is listed in BranchDiff.effects and enqueued or published on the parent by the merge, in the same transaction as the data and exactly once (the parent records how far each branch has been carried). Before, these tables could not be tracked at all (their AUTOINCREMENT keys rule out a CRR), so a merge silently left an agent's queued work behind. branch::merge_if(main, branch, &policy) merges only when the diff passes a MergePolicy (no_conflicts, read_only, no_deletes_in, max_rows, max_vector_distance, no_jobs_on, …), holding the parent's write lock from the check to the commit; otherwise it writes nothing and returns the diff with every Violation. See docs/BRANCHES.md. BranchDiff and MergeReport gain fields; a branch forked by 0.7.0 starts carrying jobs and events after its next merge.
• Live queries that watch only their own tables. live::track(&conn, table) gives a table triggers that bump its version in sqlanywhere_live_tables; a live query whose tables are all tracked compares those versions instead of the file's counters, so writes to other tables (a queue, a cache, embedding jobs) no longer make it re-run. The tables are read from the query's own EXPLAIN, through views, joins and covering indexes; anything that cannot be traced (an untracked or virtual table, another schema) keeps the old whole-file watch, and LiveQuery::watched_tables says which applies. The versions travel with the data, so other connections and embedded replicas are covered too. See docs/LIVE_QUERIES.md.

Fixed

• A live query noticed its own connection's schema changes only by chance. DROP TABLE or ALTER TABLE on the live query's own connection moves neither data_version nor total_changes; live queries now also compare schema_version.
• EXPLAIN SELECT … on an embedded replica went to the primary. It is now classified as a read and answered from the replica's own file.

0.7.0 - 2026-10-10

SQL Anywhere's pieces, composed into things they could not do apart, and a write-lock hang found behind the last flaky test. A database can be forked, diffed and merged back like a branch; a vector column keeps itself in step with its text through the durable queue; a query result keeps itself current and returns the rows that moved; a namespace can be read as it was at any moment in its backup, beside the live one; and the storage contracts compile into a typed API. test_many_concurrent, which hung on 8 of 12 CI runs, was a connection that gave up on the write lock stranding it for every writer after; it has run clean on all nine CI runs checked since the fix.

No file-format changes. Embedder gains a defaulted model_id, and the meta store's namespace configuration a new optional field; both are backward compatible.

Added

• Namespaces made by /at can expire. ttl_seconds in the request makes the primary delete the namespace, with its own backups, that long after it is made. The expiry is a new optional expires_at in the namespace configuration, stored through the meta store, so a restart does not forget it, and the primary sweeps for expired namespaces every five seconds. A plain fork does not inherit it. See docs/TIME_TRAVEL.md.
• A typed Rust API compiled from the storage contracts. Every statement in the queue, cache and pub/sub contracts now carries an sqlc-style annotation (-- name: Claim :opt), and every parameter is typed once in a table in docs/contracts/README.md. cargo xtask gen-contracts compiles the two into sqlanywhere::contracts, one function per operation with typed arguments (queue::claim(&conn, visibility, queue), cache::lock(&conn, key, owner, ttl), …), each contract's install, and the statements as constants; and into docs/contracts/contracts.json, the same operations as data for generating clients in other languages. The annotation is what separates alternatives from sequences: forget and flush are two operations, the dead-letter move is one, run in the IMMEDIATE transaction the contract spells out. generated_contracts_are_fresh fails CI when a document changes without the API following. A runtime such as Askr can now call the contracts instead of transcribing them. A row a :one or :opt function returns is copied out of its statement, which is run to completion, so holding a claimed job does not make the connection's next COMMIT fail with "SQL statements in progress", as holding a plain local Row would.
• Embedding columns: a vector column kept in step with a text column. sqlanywhere::embedding_column::EmbeddingColumn installs triggers that enqueue a job whenever the text changes, so writes never wait on a model, and process embeds in the background from any number of workers. The jobs live in the queue contract's own askr_jobs table and are claimed, acked, backed off and dead-lettered with the contract's statements word for word; a unit test compares them with the document. A worker writes a vector only if the row still holds the text it embedded, so a slow model cannot leave a stale vector behind an edit, and edits made before a worker arrives collapse into one job. Each column records the model it is embedded with, a worker running another model is refused, and reembed is the explicit migration. See docs/EMBEDDING_COLUMNS.md.
• Embedder::model_id, with a default, so existing embedders keep compiling. LexicalEmbedder reports lexical-fnv1a-<dims>d.
• Database branches (experimental): fork, diff, merge, pull. sqlanywhere::branch forks a database into an empty one, lets the fork be written freely, and merges it back through cr-sqlite, so the parent is never locked and keeps taking writes while the branch exists. diff is the review step: every row the branch changed beside the parent's current value, the conflicts a merge would hit, and for a vector column the cosine distance the embedding moved, which a byte diff of two blobs cannot show. merge never fails on a conflict and reports which side won each one; merging again sends only what was written since. The use it is shaped for is an AI agent that works on its own branch and is merged after review. Tables that are not CRRs are copied once and listed as untracked rather than silently dropped. See docs/BRANCHES.md; verified by sqlanywhere/tests/branch.rs, which the cr-sqlite workflow now runs.
• Live queries: a result that keeps itself current and says what changed. sqlanywhere::live::LiveQuery runs a SELECT once; each refresh returns a Diff of the rows inserted, updated and deleted, or None when the result is the same, and skips the query entirely when nothing was written since, which costs one local pragma. It follows writes on its own connection (total_changes, since SQLite leaves data_version alone for a connection's own commits), commits by other connections, and the frames an embedded replica pulls in on sync. Rows can be keyed so an edit is an update rather than a deletion plus an insertion. Verified against a real primary and embedded replica in the turmoil suite. See docs/LIVE_QUERIES.md.
• Querying the past: a namespace as it was at any moment, attachable to the live one. POST /v1/namespaces/:namespace/at with a UTC timestamp restores the namespace from its bottomless backup as of that moment into a read-only namespace named <namespace>@<YYYYMMDDTHHMMSS>Z, with attach allowed. ATTACH "orders@20261001T120000Z" AS past then puts history next to the live tables, so what changed is a join and a deleted row is an INSERT … SELECT. The name is deterministic, so repeating the request, or two clients making it at once, yields one namespace. See docs/TIME_TRAVEL.md. Built on forking at a timestamp, which had no end-to-end test; the new one, against a mock S3, covers both.

Fixed

• A connection that gave up waiting for the write lock could leave every later writer in the namespace waiting forever. A connection queued for the write slot returns SQLITE_BUSY when it wakes to find a checkpoint holding the slot, or a stale sync token after a WAL restart. Its queue entry stayed behind, since the queue cannot remove one. Once that connection had closed, the next release handed the slot to the dead entry as Notified, a state with no timeout that no other writer may steal, and every writer after it waited on it, spinning, for good. A waiter that gives up now marks its entry dead, so it is skipped, and passes the slot on if it had already been handed it. Reproduced deterministically by two new tests in connection_manager.rs, one for each half of the fix, each of which fails without it. This is the likely cause of test_many_concurrent hanging until its 60 s timeout in CI, which retries a busy BEGIN on a fresh connection, the sequence that strands the slot.

• The Docker images build again: base images moved from Debian 11 to 12. bullseye left LTS on 2026-06-30. deb.debian.org kept serving the bullseye-security index while the pool dropped the +deb11u packages it names, so apt-get install in every stage died on a 404 that no retry could fix; 12 of the last 13 image builds failed that way, the v0.6.2 release build among them. rust:slim-bookworm and debian:bookworm-slim now. The workflow gains a version input so the images of an already-tagged release can be published from main after a Dockerfile-only fix, tagged with that version rather than the branch.

0.6.2 - 2026-09-05

Changed

• The shutdown signal is now a level-triggered CancellationToken, and the Notify on Server.shutdown is deprecated. Notify::notify_waiters is edge-triggered: it reaches only the tasks already parked on notified() at that instant, and a task not yet polled misses it for good and runs on forever. That is the shutdown hang. #36 and #40 narrowed the window by polling early; this closes it, because a cancelled token stays cancelled and a listener that only starts afterwards still sees it. Server::shutdown_token() is the new initiator. The old field still works: start bridges it into the token, so existing callers keep the narrow window they have today rather than breaking. Server gains a public shutdown_token field. That is a compile-time break for anyone constructing it by struct literal without ..Default::default(); such callers add the field or the spread.

• The storage contracts are now the source of the SQL their tests run. contract_conformance.rs held its own copy of every contract statement, so "executable spec" meant a test that contained a transcription of the document rather than the document itself. Nothing stopped the two from drifting, and the queue's claim is a subtle atomic UPDATE that a consumer has to reproduce exactly. The tests now read the SQL out of docs/contracts/*.md and execute it verbatim, so editing a contract either proves the new semantics or turns the test red. All 18 operations the contracts publish with SQL are now executed from the document, so every one can be mutation-checked, and every_contract_operation_is_covered_or_listed fails if that slips.

Fixed

• The lock's release statement was transcribed into the test rather than read from the contract, and the coverage guard could not see it. The lock operation publishes two SQL blocks under one heading, acquire and release. The extractor reached only the first, so release was a hand copy in the test, and the guard counted headings rather than blocks, so a heading with an uncovered second block looked fully covered. Both are fixed: the extractor takes a block index, bounded to the heading's own section so an index past the end fails loudly instead of reading the next operation, and the guard now enumerates every block. Release is executed from the document, and mutating it either way, ignoring the owner or deleting nothing, turns cache_contract_v1 red. Every block the three contracts publish is now covered.

• The cache lock contract read as though acquired could be 0. The SETNX statement ends in RETURNING (value = :owner) AS acquired, which invites a consumer to branch on that value. It never is 0: RETURNING only fires for a row that was written, and a written row always carries :owner, so the comparison is redundant with the WHERE guard. The real signal is whether a row came back at all, and a consumer that treated an empty result as an error, or waited for a 0 that cannot arrive, would have been misled. The prose now says so plainly, and the SQL carries a comment; the statement itself is unchanged, so nothing built against it moves. Found by mutation testing: replacing the expression with a constant 1 changed nothing.

• Three further contract guarantees were documented but unverified, found the same way once the remaining operations were executed from the documents. Tag invalidation was never shown to be scoped to its tag, because the test had only one tag, so deleting every tagged key passed. The sweep was never shown to spare live rows, because the test flushed the table immediately after and asserted it was empty, which a sweep of DELETE FROM askr_cache also passes. The subscriber cursor was never shown to advance, because it was only ever saved once, which cannot tell an upsert from an insert that ignores the conflict. All three are asserted now.

• Two more contract guarantees were documented but unverified. Mutating the cache contract found them, the same way mutating the queue contract found the delay gap. The increment says it "treats a missing/expired entry as 0", but removing that reset left cache_contract_v1 green, because nothing ever incremented an expired counter. The upsert carries expires_at = excluded.expires_at so an overwrite moves the expiry, but removing that left the test green too, because nothing ever overwrote a key with a different TTL. Both are asserted now.

• The queue contract's delay semantics were not actually verified. Making the document the source made it possible to check the tests for teeth by mutating the contract, and one mutation slipped through: removing the available_at <= unixepoch() guard from the claim, which lets a delayed job be taken before its time, left queue_contract_v1 green. The test drained the queue and then asserted it was empty, which cannot distinguish "the delayed job was correctly skipped" from "the delayed job was claimed during the drain". It now asserts the delayed job is never among what is claimed.

0.6.1 - 2026-09-04

Three real bugs found by chasing tests that looked like noise. Every embedded-replica connection was being closed twice, the server library killed the host process on a failed shutdown, and shutdown itself could hang indefinitely. None of them were visible as failures; all three were sitting in test output nobody read because the tests around them were usually green.

No API or file-format changes.

Changed

• Shutdown now completes instead of hanging, at the cost of dropping connections that will not drain. sqld stopped accepting new connections when signalled and then waited for the in-flight ones with no deadline, so a client holding a connection open blocked shutdown for as long as it liked. In practice that meant shutdown hung until shutdown_timeout and then failed, which is a slow deploy, a container that will not die, and eventually a SIGKILL from whatever is supervising it. Each HTTP service now gets a bounded drain window after the signal, and anything still in flight when it closes is dropped. The window is derived from shutdown_timeout rather than configured separately, so there is still one knob and the outer timeout goes back to being a backstop rather than the mechanism. Reproduced at roughly one run in four on slow storage before, 40 out of 40 clean after, and the full server suite is green across repeated runs.

• The server library no longer calls std::process::exit on a failed shutdown. Server::start killed the host process when graceful shutdown failed or timed out. That is wrong for a library: the integration tests run the server in-process, and so does anything embedding sqld, so a shutdown problem took the whole host down with it and left nothing to diagnose. It now returns an error. The sqld binary is unaffected, since main propagates it and still exits non-zero with the same message.

• extension-keygen no longer writes the private key into the repository. It defaulted the whole key pair to sqlanywhere-sqlite3/ext/, so the trust root's private half landed in a working tree, where .gitignore is the only thing standing between it and a git add -f, a stray archive, or an rsync. The public half still goes to the repo, since CI reads it from a fresh checkout; the private half now goes to $XDG_CONFIG_HOME/sqlanywhere/ (falling back to ~/.config/sqlanywhere/), in a directory created mode 700 with the key mode 600. Writing it inside a working tree is refused outright, including via a relative path that climbs back in, rather than merely warned about. Both paths are overridable with --pubkey and --secret, and the command now prints the exact gh secret set line to run next.

Fixed

• Every embedded-replica connection was closed twice. SqlanywhereConnection's Drop called disconnect(), and then the Connection field it had just called it on was itself dropped, whose Drop called disconnect() again on the same value. The drop_ref count is identical at both calls, so both concluded they were the last owner and both called sqlite3_close_v2 on the same handle. The refcount guard answers "is anyone else still using this connection", not "have I already closed it". Closing an already-freed handle is undefined behaviour, survivable here only by luck. SQLite reported the second close as SQLITE_MISUSE, which had been sitting unread in the embedded-replica test output as sqlite error 21: API call with invalid database connection pointer, next to misuse at line 185183, the SQLITE_MISUSE_BKPT in sqlite3Close. disconnect() now clears the handle it closed, which makes it idempotent, and is pinned by disconnect_closes_once, a test that fails without the fix. Present in 0.6.0 and every earlier release; no database or file format is affected.

• Two flaky tests in sqlanywhere-server, with unrelated causes. test_many_concurrent asserted that opening a write transaction always succeeds. It does not: ConnectionManager::acquire returns SQLITE_BUSY on purpose while a checkpoint holds the slot, so that writers cannot starve the checkpointer, and the test crossed the auto-checkpoint threshold often enough to meet one on a loaded runner. It now retries on a busy error, bounded, with the error classification pinned by its own test. local_sync_with_writes exhausted a 120s simulated budget that in practice measured disk speed rather than protocol steps, because the test performs real file I/O inside a simulation whose clock advances independently of it: reproduced at 0/12 on overlayfs against 12/12 on tmpfs with the same binary, and raised to the 1000s the rest of these simulations use. The full server suite now runs 200/200 with no retries.

0.6.0 - 2026-09-03

Extensions you can trust. The loadable-extension ABI can now describe itself, so a version mismatch is something an extension detects rather than crashes on, and every extension this project publishes ships with a signed manifest saying exactly what it is.

This release changes the extension ABI. Prebuilt extensions from 0.5.2 and earlier must be re-downloaded; see below.

Breaking

• The extension thunk's layout changed: extensions built against 0.5.2 or earlier must be rebuilt or re-downloaded. iVersion is now the first member of sqlanywhere_api_routines, which moves close_hook. An extension compiled against an older sqlite3ext.h reads offset 0 expecting a function pointer, finds the interface-version integer instead, and calls it; loading one into 0.6.0 kills the process (verified: SIGBUS). This affects the prebuilt crsqlite-* archives attached to every release from v0.3.1 through v0.5.2. Use the v0.6.0 builds instead.

  Nothing else moves: the SQLite C API is unchanged and database files stay byte-compatible with stock SQLite, so only loadable extensions are affected. This is also the only time the layout can move. From 0.6.0 on, iVersion lets an extension detect a mismatch instead of crashing on it, and the release manifest records the interface version each artifact was built against, so a bad pairing is caught before the file is ever loaded.

Added

• Versioned loadable-extension ABI. sqlanywhere_api_routines — the SQL Anywhere thunk handed to extension entry points alongside the stock sqlite3_api_routines — now carries an iVersion as its first member, and sqlite3ext.h gained SQLANYWHERE_API_VERSION plus a SQLANYWHERE_API_ATLEAST(V) guard. An extension is compiled against one copy of the header and then loaded by whatever host library the user happens to have, so without a version field an extension built against a newer header would read past the end of an older host's structure, with no way to detect it. The structure holds one member today (close_hook), so this is the last moment the field can be added at all; the cost of adding it now is the one-time layout change described under Breaking below. Documented in sqlanywhere_extensions.md and verified by sqlanywhere-sqlite3/test/rust_suite/src/extension_abi.rs, which compiles a real out-of-tree loadable extension and checks that the host's advertised version agrees with its own header and that a probe for a not-yet-implemented version is declined rather than followed.

• Signed extension repository. A prebuilt extension is code you download and then run with the full privileges of your database process, but installing one meant fetching a shared object from a release page and hoping. Releases now carry SHA256SUMS for integrity, a MANIFEST.json describing every artifact (digest, size, and the extension interface version it was compiled against), and a detached Ed25519 signature over that manifest's exact bytes. Three new tasks drive it: cargo xtask extension-keygen (a one-time, deliberately manual step, since CI must not mint its own trust root), sign-extensions, and verify-extensions, which checks the signature, then each digest, then that the artifact's interface version is one the host implements. Keys carry short ids so rotation does not need a flag day, and an unsigned release cannot pass as a signed one: verify-extensions fails unless told --allow-unsigned. Wired into crsqlite.yml on every build, not just tags, so the tooling stays covered. New docs/EXTENSION_REPOSITORY.md, which also records what is deliberately not done: the loader still opens any file you point it at, and making it enforce signatures is a policy decision with open questions listed there.

Removed

• publish-crsqlite.yml. It attached its own unsigned crsqlite-linux-x86_64.zip on v* tags, built independently of the per-target workflow, so that archive sat outside MANIFEST.json and nobody could verify it. crsqlite.yml already builds the same Linux x86-64 target (plus Apple Silicon and Linux ARM), docs/CRDT.md already documents the crsqlite-<tag>-<target>.tar.gz naming rather than the zip, and every tag ever pushed matches the v*.*.* pattern the remaining workflow triggers on, so nothing is lost. The dead prebuild-test.* trigger goes with it.

Fixed

• Stale SQLite version in the README. The sample shell transcript claimed the fork is based on SQLite 3.43.0; the bundled amalgamation has been 3.47.0 for some time. Corrected while bumping the version marker on the same line.

• Null-thunk dereference in the vendored cr-sqlite extension. crsqlite.c called sqlanywhere_close_hook unconditionally, but a host built with SQLITE_OMIT_LOAD_EXTENSION passes no SQL Anywhere thunk at all. The call is now guarded by SQLANYWHERE_API_ATLEAST(1).

0.5.2 - 2026-07-16

The substrate half of the Redis-free stack, made provable: the queue, cache and pub/sub contracts are now executable, CI-verified specs the Askr runtime builds its L2 drivers against.

Added

• Conformance-tested storage contracts (substrate × Askr runtime). The queue, cache and pub/sub contracts in docs/contracts/ are now executable, CI-verified specs: sqlanywhere/tests/contract_conformance.rs runs the exact contract SQL and asserts the documented semantics (queue at-least-once claim / priority / delay / dead-letter / backlog; cache TTL / atomic increment / SETNX locks / tag invalidation; pub/sub monotonic tail / cursor / retention). This is the substrate half of the Redis-free stack (epic elyra-2); the Askr runtime (askr/docs/STORAGE_BACKEND.md) now builds its L2 drivers against a proven contract that cannot silently drift.

0.5.1 - 2026-07-16

Another chapter, not another product: the everyday storage primitives — a KV cache with TTL, a durable work queue, and pub/sub — shown to compose out of plain SQL over the replicated SQLite engine, with no Redis/SQS/Kafka alongside.

Added

• Storage primitives, as chapters not products. A KV cache with TTL, a durable work queue, and pub/sub all compose out of plain SQL over the replicated SQLite engine — no Redis/SQS/Kafka alongside. The cache is a table with an expiry column (lazy-filtering view + periodic sweep); the queue is a table with an atomic UPDATE … RETURNING claim and a visibility timeout (at-least-once); pub/sub is an append-only topic tailed by cursor, carried across nodes by the replication log. New docs/STORAGE_PRIMITIVES.md; demonstrated (sqlanywhere/examples/storage_primitives.rs) and verified (sqlanywhere/tests/storage_primitives.rs, 3 tests).

0.5.0 - 2026-07-16

Search, made whole and made honest. Full-text, faceted, vector and hybrid search are unified as one engine, one chapter — not a separate product; real semantic embeddings become first-class via a pluggable Embedder trait (with a worked neural example); and every published Docker image is now smoke-tested on both architectures.

Added

• Search, unified as one chapter (not a product). Full-text (FTS5 inverted index), faceted (GROUP BY over the matched set), vector (DiskANN) and hybrid (RRF) search are the same engine composed in plain SQL — no separate search service. New docs/SEARCH.md ties them together; faceted search is demonstrated (sqlanywhere/examples/faceted_search.rs) and verified (sqlanywhere/tests/faceted_search.rs, 3 tests: full-text match, facet counts, drill-down).
• Pluggable embeddings (Embedder trait). Bring your own semantic model (local ONNX/candle, or a hosted API) and feed it into the same vector32(...) storage and vector_top_k search path as the built-in embedder. The dependency-free embed() is now the LexicalEmbedder implementation of this trait; a to_vector_literal() helper formats any raw vector for vector32. Output of embed() is unchanged (back-compatible).
• Worked semantic-search example (examples/semantic-search) plugging a real local sentence-transformer (all-MiniLM-L6-v2 via candle) into the Embedder trait — kept out of the main workspace so its ML dependencies never touch the core build. Demonstrates true semantic matching (finds "the cat sat on the mat" for "a small feline rested on a rug").
• Docker release smoke test. scripts/smoke-test-docker.sh boots a published image and asserts the HTTP API serves a real vector search (vector_top_k over a DiskANN index). Wired into docker.yml to run on both amd64 and arm64 after the manifest is published, so a broken release image can never pass silently.

0.4.0 - 2026-07-08

The flagship collaborative, syncable vector index — CRDT offline merge × DiskANN vector search × inline embed() — plus multi-arch Docker images for Ubuntu Intel and ARM alongside the prebuilt binaries.

Added

• Collaborative, syncable vector index (experimental). The flagship combination of CRDT offline merge × DiskANN vector search × inline embed(): several devices build a semantic index offline and independently, then merge conflict-free — afterwards every device can vector-search over every device's documents (the index is maintained as cr-sqlite applies merged rows). Verified by sqlanywhere/tests/collab_vector.rs (a doc indexed only on node B becomes the nearest neighbour on node A after merge) and demonstrated by sqlanywhere/examples/collab_vector.rs. Guide: docs/COLLABORATIVE_VECTOR.md.
• Docker images. Multi-arch sqld container images (linux/amd64 and linux/arm64) are built and published to ghcr.io/kwhorne/sqlanywhere-server on each release, alongside the prebuilt binaries.

0.3.1 - 2026-07-02

Experimental CRDT offline merge via the vendored cr-sqlite extension — conflict-free multi-writer offline sync, the other half of a local-first stack alongside embedded replicas. Additive and opt-in.

Added

• CRDT offline merge (experimental). The vendored cr-sqlite extension (sqlanywhere-sqlite3/ext/crr) now builds into a loadable extension via scripts/build-crsqlite.sh, turning tables into conflict-free replicated relations with crsql_as_crr(...). Multiple databases can be edited offline and merged deterministically by exchanging crsql_changes rows. Verified end to end (two nodes converge; concurrent same-row edits resolve the same way on both sides). Guide: docs/CRDT.md; continuously built by the crsqlite.yml CI workflow. Not yet bundled into sqld or the prebuilt binaries.
• CRDT via the Rust API. sqlanywhere/examples/crdt_sync.rs demonstrates offline multi-writer merge driven entirely through the sqlanywhere client (load_extension + crsql_as_crr + crsql_changes), and sqlanywhere/tests/crdt.rs asserts it (gated on SQLANYWHERE_CRSQLITE, run in CI against a freshly built extension). Prebuilt extensions are attached to releases for macOS Apple Silicon and Ubuntu Intel/ARM.

0.3.0 - 2026-06-24

Theme: vector-native edge. SQL Anywhere is one of the few engines to ship native vector search and bi-directional edge replication in the same file-compatible SQLite fork. 0.3.0 leans into that: it turns the vector engine into a batteries-included toolkit for local-first / edge RAG — embed text inline, index it compactly, and retrieve with fused semantic + keyword ranking, all inside one embedded database.

Everything below is additive and opt-in; databases that don't use the new features remain byte-compatible with stock SQLite.

Added

• Hybrid search (vector + FTS5). Fuse DiskANN vector similarity with SQLite FTS5 full-text relevance in a single query using Reciprocal Rank Fusion (RRF), so documents strong in both signals rank highest — the state-of-the-art retrieval pattern for RAG. No new engine code: it composes primitives the engine already ships. Documented in the README; covered by sqlanywhere/tests/hybrid_search.rs (3 tests).

• Vector quantization for the edge. The DiskANN index now accepts compress_neighbors=float16|float8|float1bit to quantize the neighbour vectors stored in the graph. Measured on 800×32-dim cosine vectors the on-disk index shrinks 1.9× / 2.8× / 5.5× respectively while search keeps working — a large win on memory-constrained devices. Covered by sqlanywhere/tests/vector.rs (2 tests).

• embed() reference text embedder. sqlanywhere::embed(text, dims) turns text into an L2-normalized vector literal for inline vector32(embed(text, dims)), so you can build a vector column without a separate pre-compute step. Uses the hashing trick (FNV-1a bag-of-words), so it is deterministic and dependency-free. It is lexical, not semantic — for production semantic search, compute embeddings with a real model and store them the same way. Covered by src/embed.rs (5 unit tests + doctest) and sqlanywhere/tests/embed.rs (E2E).

• Local RAG capstone example. sqlanywhere/examples/local_rag.rs is a runnable, end-to-end retrieval pipeline in a single embedded database: embed() → quantized (float8) DiskANN index → FTS5 keyword index → hybrid RRF retrieval. Run it with cargo run -p sqlanywhere --example local_rag.

• docs/VECTOR_SEARCH.md — a why / how / examples guide for all of the above, and docs/ROADMAP.md — direction for the release.

Verification

• New automated tests for hybrid search, quantization and embed() all run in CI. Vector search and replication were re-verified end to end (see sqlanywhere/tests/vector.rs, sqlanywhere/tests/replication.rs, and the server embedded_replica suite).

Notes

• CRDT offline-merge (cr-sqlite) remains on the roadmap; it requires a pinned nightly toolchain plus build-std and C linking and is tracked as a separate effort.

0.2.0 - 2026-06-23

The first stabilization release after the initial fork. Focuses on build reproducibility, full independence from upstream packages, fixing rebrand-era bugs, and getting CI green across Linux and Windows.

Fixed

• WASM user-defined functions: corrected a truncated internal table name (sqlanywhere_wasm_func_table) caused by a hard-coded string length left over from the rename. WASM UDFs failed with no such table: sqlanywhere_wasm_func_ before this fix. Patched in both src/ and the bundled amalgamations.
• Native library name: the SQLite-compatible C library now builds consistently as libsqlanywhere.{a,la,dylib} / sqlanywhere.lib. The rename had accidentally produced an invalid sqlite3.la (no lib prefix, rejected by libtool) and a mangled sqlanywhereite3 target.
• Autotools/MSVC build: regenerated autoconf/Makefile.msc from Makefile.msc so srctree-check passes again.
• P0 safety issues:
  • Documented the Send/Sync soundness of local::Rows (SQLITE_THREADSAFE, Arc ownership, single-task RefCell access).
  • Hardened bottomless::Replicator::wait_until_snapshotted with explicit control flow and clear error semantics.
  • Removed a TOCTOU race in namespace fork (redundant existence check before the lock-guarded check).
• Applied cargo fmt across the workspace.

Changed

• Full independence from upstream crates.io packages: replaced the external libsql-client (dev) and libsql-wasmtime-bindings dependencies. The bottomless integration test now dogfoods the in-tree sqlanywhere client, and the WASM runtime uses the in-tree wasmtime-bindings crate.
• Renamed the C-binding crate sql-experimental → sqlanywhere-experimental (output libsqlanywhere_experimental.a), removing the last libsql-looking artifact.
• Modernized CI: actions/checkout v2/v3 → v4, actions/cache v3 → v4, replaced deprecated actions-rs/cargo@v1 with direct cargo commands; Windows builds skip the encryption feature (no Visual Studio CMake generator on runners).
• Added/normalized Cargo.toml descriptions and keywords for all crates.
• Documentation links now point to https://elyracode.com/docs/sqlanywhere; source links point to github.com/kwhorne/sql-anywhere.

Added

• CHANGELOG.md (this file).
• docs/TECH_DEBT.md — a categorized inventory of the inherited code markers (35 FIXME, 59 TODO, 1 HACK, 2 XXX) with recommended priorities.
• Build prerequisites table in the README (Rust, C compiler, libclang, protoc, cmake) with per-OS install commands; CI installs cmake where the encryption feature is built.
• workflow_dispatch triggers on the core CI workflows for manual runs.

CI status

Green on Linux and Windows for: Rust (fmt/check/test/encryption), C bindings, Extensions (vector, UDF, cr-sqlite), and the Makefile/WASM SQLite test suite.

0.1.0 - 2026-06-21

Initial release of SQL Anywhere — an embeddable, replication-ready SQL engine built on SQLite, maintained by Elyra.

Added

• Complete fork and rebrand to SQL Anywhere across the entire codebase: Rust crates, the SQLite C fork, FFI bindings, bundled amalgamations, and binary test fixtures (WASM modules and the DiskANN vector-index database).
• Embedded Rust API (sqlanywhere), server (sqld / sqlanywhere-server), Hrana remote protocol, replication primitives, and bottomless S3-backed WAL replication.
• Original project README, set the workspace and C-library version to 0.1.0, and published the v0.1.0 tag and GitHub release.