Skip to content

Ad4mModel Refactor#694

Open
jhweir wants to merge 146 commits intodevfrom
ad4m-model-refactor
Open

Ad4mModel Refactor#694
jhweir wants to merge 146 commits intodevfrom
ad4m-model-refactor

Conversation

@jhweir
Copy link
Copy Markdown
Contributor

@jhweir jhweir commented Feb 25, 2026

Ad4mModel Refactor — SHACL-native ORM, full test suite refactored

Summary

This PR completes the full Ad4mModel refactor started in the feat/shacl-sdna-migration base. The legacy Prolog-based subject class system is removed from the TypeScript ORM layer. Ad4mModel is now a clean, SHACL-native ORM backed entirely by SurrealDB, with a Prisma-inspired query/decorator API, atomic transactions, reactive subscriptions, and eager-loading via IncludeMap. The monolithic 3,917-line Ad4mModel.ts has been decomposed into focused modules and is now 759 lines. The full ad4m test suite has been refactored.


What changed

⚠️ Breaking changes

  • baseExpressionid on all Ad4mModel instances — get baseExpression() public getter removed; PerspectiveProxy and Rust keep baseExpression at the protocol level
  • writablereadOnly on PropertyOptions / SHACLPropertyShape (inverted semantics: readOnly: true means no setter)
  • collection*relation*setCollection* methods renamed to set*; CollectionSetter Rust enum variant renamed to RelationSetter
  • update() instance method removed — use save() for instance updates or the new static Ad4mModel.update()
  • run() alias removed on ModelQueryBuilder — use .get()
  • InstanceQuery decorator removed — use findAll() / ModelQueryBuilder
  • generateSDNA() removed from @ModelOptions
  • useSurrealDB flag and useSurrealDB() method removed from ModelQueryBuilder
  • isInstance, prologCondition, where.condition removed from the query API
  • Query.source and ModelQueryBuilder.source() removed — has_child source-scoping is gone
  • useLiveQuery(Model, perspective, options?)perspective is now arg 2, not inside options; useModel removed from both react and vue hooks
  • ParentScope is now a union: { model, id, field? } or { id, predicate } (raw form)

� Migration guide

1. baseExpressionid

// Before:
const post = await Post.findOne(perspective, { where: { title: "Hello" } });
console.log(post.baseExpression); // "ad4m://abc123"

// After:
console.log(post.id); // "ad4m://abc123"

2. Decorator names and options

// Before:
@ModelOptions({ name: "Post" })
class Post extends Ad4mModel {
  @Optional({
    through: "flux://has_title",
    resolveLanguage: "literal",
    writable: true,
  })
  title: string;

  @Property({
    through: "flux://has_body",
    resolveLanguage: "literal",
    writable: true,
  })
  body: string;

  @ReadOnly({ through: "flux://has_slug", resolveLanguage: "literal" })
  slug: string;

  @Collection({ through: "flux://has_child" })
  comments: string[];
}

// After:
@Model({ name: "Post" })
class Post extends Ad4mModel {
  @Property({ through: "flux://has_title" })
  title: string;

  @Property({ through: "flux://has_body", required: true })
  body: string;

  @Property({ through: "flux://has_slug", readOnly: true })
  slug: string;

  @HasMany(() => Comment, { through: "test://has_comment" })
  comments: Comment[] = [];
}

3. Static create / update / delete

// Before — manual three-step pattern:
const post = new Post();
post.title = "Hello";
await post.save(perspective);

const existing = new Post();
existing.id = "ad4m://abc123";
await existing.getData(perspective);
existing.title = "Updated";
await existing.save(perspective);

const toDelete = new Post();
toDelete.id = "ad4m://abc123";
await toDelete.deleteSubject(perspective);

// After — single-call static API:
const post = await Post.create(perspective, { title: "Hello" });

await Post.update(perspective, "ad4m://abc123", { title: "Updated" });

await Post.delete(perspective, "ad4m://abc123");

4. Transaction API

// Before (fragile — leaked batch if any save threw):
const batchId = await perspective.createBatch();
await a.save(batchId);
await b.save(batchId);
await perspective.commitBatch(batchId);

// After:
await Ad4mModel.transaction(perspective, async (tx) => {
  await a.save(tx.batchId);
  await b.save(tx.batchId);
});

5. useLiveQuery / useModel (ad4m-hooks)

// Before — useModel, perspective inside options:
const { entries: posts } = useModel(Post, {
  perspective,
  where: { published: true },
});

// After — useLiveQuery, perspective is arg 2:
const { data: posts } = useLiveQuery(Post, perspective, {
  where: { published: true },
});

6. Query builder — run()get(), subscribe()live()

// Before:
const results = await Post.query(perspective).where({ published: true }).run();
Post.query(perspective).subscribe((posts) => console.log(posts));

// After:
const results = await Post.query(perspective).where({ published: true }).get();
Post.query(perspective).live((posts) => console.log(posts));

7. ModelQueryBuilder.source() / Query.source removed

// Before — source-scoping via has_child:
Post.query(perspective).source(parentId).get();

// After — use parent option:
Post.query(perspective)
  .parent({ model: Parent, id: parentId, field: "posts" })
  .get();
// or equivalently via findAll:
Post.findAll(perspective, {
  parent: { model: Parent, id: parentId, field: "posts" },
});

�🗑️ Prolog removal & SHACL/SurrealDB migration

TypeScript — Prolog query paths removed:

  • Deleted Subject.ts — necessary bits inlined into PerspectiveProxy as a local class
  • Deleted queryToProlog(), instancesFromPrologResult(), countQueryToProlog(), and all build*Query helpers
  • Deleted @InstanceQuery decorator and InstanceQueryParams interface
  • Deleted generateSDNA() from @ModelOptions
  • Removed prologGetter / prologSetter from PropertyOptions, PropertyMetadata, Optional(), Property(), ReadOnly() interfaces
  • Removed useSurrealDB flag and useSurrealDB() method from ModelQueryBuilder
  • Renamed makeRandomPrologAtommakeRandomId

Rust — Prolog→SHACL generation removed:

  • Removed parse_prolog_sdna_to_shacl_links (328-line function) and backward-compat Prolog→SHACL code generation from add_sdna; SHACL→Prolog direction (shacl_to_prolog.rs) kept for compatibility
  • Extracted shacl_to_prolog.rs — SHACL→Prolog compat code moved into its own module with comprehensive unit tests
  • Fixed sdnaCode nullable in GraphQL schema — was String!, must be String since Prolog is now optional

SHACL/SurrealDB migration:

  • Migrated all flow methods to SHACL — rewrote sdnaFlows, availableFlows, startFlow, expressionsInFlowState, flowState, flowActions, runFlowAction in PerspectiveProxy.ts to use SHACLFlow/getFlow() instead of Prolog infer() calls
  • Removed subjectClassesFromSHACL GraphQL endpoint — replaced with client-side SHACL link queries via findClassByProperties()
  • Replaced buildQueryFromTemplate() / Prolog-based SubjectClassOption with client-side SHACL matching via findClassByProperties(); then replaced by a single SurrealDB query (two-pass client-side processing) to avoid N+1 round trips; subjectClassesByTemplate falls back to findClassByProperties() when class name lookup fails
  • Used SHACLShape.toJSON() in ensureSDNASubjectClass instead of manual JSON; getSubjectClassMetadataFromSDNA now uses getShacl()/SHACLShape.fromLinks()
  • Batched SHACL/Flow link writesaddShacl()/addFlow() use addLinks() batch API; getShacl()/getFlow() use a single querySurrealDB() call instead of individual get() calls
  • Converted ends_with filters to SurrealDB string::ends_with queries (5 functions in perspective_instance.rs that previously fetched all and filtered in memory)
  • Fixed SurrealQL function name: string::starts_with not starts::with; replaced SQL LIKE with string::starts_with in getFlow()
  • Executor commit ordering fixupdate_prolog_engines() (which spawns the pubsub link-added task) previously ran before persist_link_diff() (the SurrealDB write); any subscriber calling findAll() immediately on link-added would read stale data; order swapped so SurrealDB is committed before pubsub fires

�️ Ad4mModel decomposition

Ad4mModel.ts went from 3,917 → 759 lines across the following extractions:

File Contents
model/types.ts WhereCondition, Query, IncludeMap, ModelMetadata, subscription types
model/schema/metadata.ts getModelMetadata(), assignValuesToInstance(), ModelValueTuple
model/schema/fromJSONSchema.ts createModelFromJSONSchema, determinePredicate, determineNamespace
model/query/surrealCompiler.ts Pure SurrealQL builder helpers
model/query/operations.ts queryToSurrealQL, instancesFromSurrealResult, findAll, findOne, findAllAndCount, paginate, count
model/query/hydration.ts hydrateInstanceFromLinks(), evaluateCustomGetters() — shared between single-instance and bulk paths
model/query/fetchInstance.ts 6-stage single-instance hydration pipeline (getData() body)
model/query/snapshot.ts Snapshot-based dirty tracking
model/query/ModelQueryBuilder.ts Fluent query builder class
model/mutation.ts MutationContext, setProperty, setRelation*, saveInstance, innerUpdate, cleanCopy
model/transaction.ts runTransaction, TransactionContext
model/subscription.ts createSubscription(), shared registry, fingerprinting
model/parentUtils.ts normalizeParentQuery(), resolveParentPredicate()

Unified hydrationgetData() (single-instance) and instancesFromSurrealResult() (bulk) previously had divergent hydration implementations (one using latest-wins, the other first-wins); both now share hydrateInstanceFromLinks() from hydration.ts with consistent latest-wins ordering.

Transaction API:

// Before (fragile — leaked batch if any save threw):
const batchId = await perspective.createBatch();
await a.save(batchId);
await b.save(batchId);
await perspective.commitBatch(batchId);

// After:
await Ad4mModel.transaction(perspective, async (tx) => {
  await a.save(tx.batchId);
  await b.save(tx.batchId);
});

Include API / IncludeMap:

// Prisma-style eager loading — relations stay as bare IDs unless include is set
Recipe.findAll(perspective, {
  include: {
    comments: true,
    tags: { where: { active: true }, limit: 5 },
  },
});

Removed the old relations field and .relations() fluent method. Nested include maps are supported: setting a relation key to another IncludeMap object (rather than true) recursively hydrates that relation's own relations.


🎨 Decorator & relation API

  • Introduced @Model, @HasMany, @HasOne, @BelongsToOne, @BelongsToMany decorators
  • WeakMap metadata registry keyed on constructor (not prototype) — eliminates a silent inheritance/data-corruption bug for subclassed models
  • @BelongsToOne / @BelongsToMany decorators with direction='reverse'/maxCount; HasMany/HasOne accept an optional model factory argument
  • Renamed setCollection*set* throughout the public API; PerspectiveProxy.isCollectionSetter() now uses metadata instead of a string prefix check
  • @HasOne hydration fixgetData() and instancesFromSurrealResult() now apply a maxCount === 1 guard so @HasOne fields resolve to a scalar string, not an array
  • @Flag SHACL wiringgeneratePropertySetterAction() throws if metadata.flag is set; innerUpdate() skips flag fields; flags are immutable after creation; @Flag now automatically sets readOnly: true
  • Renamed writablereadOnly throughout the stack (inverted semantics): PropertyOptions.writable → readOnly, SHACLPropertyShape.writable → readOnly in toTurtle/toLinks/fromLinks/toJSON/fromJSON, Rust PropertyShape.writable → read_only with #[serde(rename = "readOnly")], PerspectiveProxy.writablePredicates → readOnlyPredicates with inverted guard logic for isSubjectInstance matching
  • @Property auto-derive initialgenerateSHACL() now auto-derives effectiveInitial = "literal://string:" for any writable non-flag property without an explicit initial; destructor actions always added for all writable non-flag properties; users only need initial: for semantic defaults (e.g. "todo://ready")
  • resolveLanguage: "literal" is now the implicit defaultdecorators.ts auto-injects it into the SHACL shape for all non-flag properties so the Rust executor always uses fn::parse_literal; all explicit resolveLanguage: "literal" annotations removed from decorators and fromJSONSchema calls
  • Model inheritance via SHACL sh:nodeSHACLShape gains parentShapes, addParentShape(), and emits sh:node in both toTurtle() and toLinks() serializers; child shape uses only own properties/relations and references the parent via sh:node instead of duplicating; runtime inheritance already worked via the WeakMap prototype-chain walk in getPropertiesMetadata/getRelationsMetadata
  • Relation decorator validation@HasMany, @HasOne, @BelongsToMany, and @BelongsToOne throw a clear error at class-definition time if the through predicate is missing, instead of silently producing broken SHACL shapes
  • RelationSetter backwards-compat alias — Rust CollectionSetter enum variant gains a #[serde(alias = "collectionSetter")] so existing serialised data written with the old name continues to deserialise correctly
  • Updated @we/models and the test app to the new decorator names

🚀 Query & mutation API

Static convenience methods:

  • Ad4mModel.create<T>(perspective, data, options?) — static factory that constructs, assigns, and saves in one call; options.parent links the new instance to a parent node atomically (predicate auto-inferred from @HasMany relations when field is omitted); options.batchId batches both the save() and the parent link write in the same transaction
  • Ad4mModel.update<T>(perspective, id, data, batchId?) — fetches the existing instance, merges the partial data, then saves; replaces the common three-step new/assign/save pattern
  • Ad4mModel.delete(perspective, id, batchId?) — constructs an instance by id, automatically removes all incoming links ({ target: id }) so parent collection queries stay consistent, then calls removeSubject(); replaces the new/delete two-step
  • Ad4mModel.register(perspective) — thin wrapper around ensureSDNASubjectClass() for a consistent static API

Query extensions:

  • Query.linkedFrom: { id, predicate } — parent-scoped graph-traversal filter; compiles to count(<-link[WHERE predicate='...' AND in.uri='...']) > 0 in SurrealQL; subscription layer watches linkedFrom.predicate so child additions/removals trigger live re-queries
  • Query.parent — higher-level alternative to linkedFrom accepting ParentQueryByPredicate | ParentQueryByModel; normalizeParentQuery() resolves the model-backed form to { id, predicate } by scanning @HasMany metadata; wired through surrealCompiler, operations.ts, subscription.ts, and ModelQueryBuilder
  • Where-filtering on relation fieldsbuildGraphTraversalWhereClause() now handles relation names in the where clause (was silently skipped); forward relations compile to ->link[WHERE predicate AND out.uri = value] > 0; reverse relations compile to <-link[WHERE predicate AND in.uri = value] > 0; IN, not, and not-IN variants all supported
  • Nested include supportoperations.ts now follows Query.include maps recursively (was hard-coded to false for sub-queries); multi-level depth guard prevents infinite recursion
  • Include sub-query order preservation — fixed include sub-query result ordering when the sub-entry has its own order clause
  • Remove has_child and source-scopingQuery.source, ModelQueryBuilder.source(), and has_child link write on create path removed; legacy perspectives had meaningless source → ad4m://has_child → baseExpression links

Internal fixes:

  • Snapshot-based dirty tracking (model/query/snapshot.ts) — captures a deep snapshot of all field values after hydration; innerUpdate() checks isDirty() before re-writing each property/relation, preventing duplicate link writes for unchanged relations (fixes channel-pinning duplication); snapshot captured in both fetchInstance.ts and operations.ts after hydration
  • Properties projection fixcleanCopy() now only deletes schema-declared fields, preserving internal ORM machinery (_id, _perspective, generated addX/removeX/setX methods)
  • TypeScript private instead of JS # fields — JS hard-private # fields use WeakMap semantics that fail when this is a Vue reactive Proxy; switched to TypeScript private (_) so Ad4mModel instances can be stored directly in Vue reactive state without toRaw() wrappers
  • Fix stableFingerprint — was always undefined because it referenced the deleted baseExpression field; now uses id; instanceToSerializable() helper extracted for DRYness and circular JSON safety
  • Fix multi-field sort dropping all but the first key in operations.ts
  • Fix count() N+1 — added hasJsFilterConditions() fast path; count() was hydrating all rows just to count them
  • Fix queryToSurrealQL SELECT — was ->link AS links (returns only id/uri); fixed with a correlated subquery returning predicate, out.uri as target, author, timestamp; removed phantom $perspective filter (silent no-op)
  • matchesCondition fix for combined operatorsnot and between folded into the allMet chain in surrealCompiler.ts so they compose correctly with other WhereCondition operators
  • normalizeTimestamp for numeric min/maxhydrateInstanceFromLinks now normalises timestamps before numeric min/max comparison, fixing latest-wins ordering for links with sub-second timestamps
  • resolveLanguage implicit default in compilersurrealCompiler.ts uses fn::parse_literal for both undefined and "literal" in WHERE comparisons

🔄 Subscription system

Client-side link-listener subscription (replaces server-push SurrealDB subscriptions):

  • New createSubscription<T>() in subscription.ts — fires immediately then on every relevant link-added/link-removed
  • Relevance check: only re-queries when a changed link's predicate belongs to the model (properties + relations); watches linkedFrom.predicate for parent-scoped queries
  • Shared registry: identical (model, query) pairs on the same perspective share one findAll() execution and one set of link listeners via a WeakMap-keyed registry
  • Result fingerprinting (stableFingerprint): callbacks only fire when the result set actually changed; includes @HasMany/@HasOne relation fields so relation-only mutations (e.g. addTags) trigger re-broadcast
  • Late subscriber fast-path: second subscriber receives cached lastResults immediately via microtask
  • Last-subscriber teardown: shared entry and listeners are fully cleaned up when listener count reaches zero
  • 50 ms coalesce window (SETTLE_MS) to batch rapid link events from a single save() into one re-query
  • .live(callback) fluent terminal on ModelQueryBuilder alongside .get()

Server-push SurrealDB subscription removal:

Deleted the legacy server-side SurrealDB subscription system made redundant by the above:

TypeScript (client):

  • Removed isSurrealDB field from QuerySubscriptionProxy and its if/else surreal branches in subscribe(), keepalive loop, and dispose()
  • Deleted subscribeSurrealDB() from PerspectiveProxy
  • Deleted perspectiveSubscribeSurrealQuery(), perspectiveKeepAliveSurrealQuery(), perspectiveDisposeSurrealQuerySubscription() from PerspectiveClient

Rust (server):

  • Deleted SurrealSubscribedQuery struct
  • Dropped trigger_surreal_subscription_check and surreal_subscribed_queries fields + Arc::new() initialisers from PerspectiveInstance
  • Removed surreal_subscription_cleanup_loop() from start_background_tasks()
  • Deleted 5 functions: subscribe_and_query_surreal, keepalive_surreal_query, dispose_surreal_query_subscription, surreal_subscription_cleanup_loop, check_surreal_subscribed_queries
  • Deleted 3 GraphQL mutation resolvers: perspective_subscribe_surreal_query, perspective_keep_alive_surreal_query, perspective_dispose_surreal_query_subscription
  • Removed two trigger_surreal_subscription_check trigger lines from the link-added/link-removed update paths

Net: –876 lines, +769 lines across 4 files.


🪝 ad4m-hooks — useLiveQuery API

  • Deleted helpers/src/ entirely — SubjectRepository, SubjectFactory, cache.ts, getProfile.ts and their index re-exports removed; ad4m-hooks/helpers package removed from pnpm workspace, package.json, and publish workflows
  • Deleted from both react and vueuseMe, usePerspectives, useAgent, usePerspective, register — app-level concerns already implemented in consumer repos
  • Replaced useModel with useLiveQuery in both react and vue packages:
    • useLiveQuery(Model, perspective, options?) — perspective is now arg 2, not inside options
    • Parent scopeparent: ParentScope restricts results to children of a parent node; ParentScope is { model, id, field? } (auto-resolves predicate) or { id, predicate } (raw form); bails out early when parent.id is falsy to handle web-component property-assignment races
    • Single-instance modeid option returns LiveInstanceResult<T> instead of a list
    • Load-more / infinite-scroll pagination via loadMore() / pageSize option
    • preserveReferences mode for stable object identity across re-renders
    • String model name override for generic class browsers
    • isRef() normalisation of the perspective prop in Vue hook so both raw values and refs are accepted
    • Active subscription torn down on perspective / query / page change and on onUnmounted/cleanup
    • Entries keyed on entry.id (was entry.baseExpression)
  • Net: –1,292 lines, +144 lines initial slim; further +649/–285 lines for useLiveQuery rewrite (21 files)

🔧 Executor — JS bundle Deno ESM compatibility

  • executor/esbuild.ts post-build buffer patchsafe-buffer@5.2.1 (inside @transmute/did-key-ed25519) was bundled as a __commonJS wrapper that emitted var buffer2 = __require("buffer"). Deno's ESM runtime cannot execute require() calls, so coreInit() threw Dynamic require of "buffer" is not supported, the executor started its HTTP server but silently failed all GraphQL operations, and integration tests hung until timeout. Fixed by adding a post-build regex replacement in esbuild.ts that rewrites all __require("buffer"|"node:buffer") calls to reference the existing top-level import buffer from "node:buffer" already present in the bundle.
  • Added // @ts-nocheck to executor/esbuild.ts — tsc never processes this Deno script; suppresses ~35 pre-existing URL-import and implicit-any errors.
  • rust-executor agent data directoryinit_global_instance() now creates {app_path}/ad4m/ before constructing AgentService; the nested subdirectory was never created so std::fs::write() panicked with Failed to write agent file: Os { code: 2, kind: NotFound } on fresh CI workdirs.

⚙️ CI

  • Replaced Docker executor (coasys/ad4m-ci-linux) with self-hosted machine runner coasys/marvin (AMD Ryzen 9 9950X, 60 GB RAM, Ubuntu 25.04; Rust 1.92, Node 18, Deno, Go, pnpm pre-installed)
  • Removed remote cacherestore/save_cache was uploading/downloading the full Rust target/ dir (~20 GB) causing 29-minute cache restore steps on every run; the persistent machine runner does incremental compilation instead (cleanup_working_directory: false)
  • Fixed nvm stdout corruption in cargo build scripts — nvm use 18 in $BASH_ENV printed Now using node v18.20.8 to stdout which corrupted the libffi link step via a bash syntax error in a subshell; fixed by adding the Node 18 bin dir directly to PATH
  • Isolated setup port ranges per concurrent jobports.ts now assigns non-overlapping port windows per CI job index so parallel integration-tests-core and integration-tests-multi-user jobs don't collide on port 12000; added waitForPortFree() pre-check
  • Added prepare-test step to integration-tests-js and integration-tests-multi-user jobs — ensures the executor binary and bootstrap seed are freshly prepared before each suite; bootstrap seed reset cleanly before each prepare-test invocation
  • Renamed CircleCI jobs for clarity: build-and-testbuild, integration-tests-jsintegration-tests-core, integration-tests-multi-user-simpleintegration-tests-multi-user; removed duplicate integration-tests-email-verification (covered by test-auth inside test:ci); bumped multi-user job timeout to 60 min

🧪 Test suite

Infrastructure:

  • Added helpers/ directory: ports.ts (dynamic port allocation), executor.ts (AgentHandle), assertions.ts (waitUntil), index.ts (barrel)
  • Added wipePerspective() export to utils/utils.ts
  • Centralized global.fetch polyfill into tests/setup.ts
  • Migrated auth and multi-user tests to a shared startAgent helper
  • Removed findAndKillProcess entirely — matched any process by name and could kill a live AD4M instance; teardown already uses tree-kill with the specific child PID

New model test suite (tests/model/):

  • models.tsTestPost, TestComment, TestTag, TestBaseModel, TestDerivedModel, TestChannel, TestReaction fixture models
  • model-core.test.ts — 20 CRUD + decorator coverage tests; Ad4mModel.update() and .delete() integration tests
  • model-query.test.ts — 16 sections covering where/order/limit/offset/count/paginate/findAllAndCount/include/parent queries/dirty tracking (~1,100 lines)
  • model-subscriptions.test.ts — live subscription tests using .live() API; parent-scoped subscription tests; @HasMany re-fire regression test
  • model-transactions.test.ts — batch/transaction pattern tests including rollback (7 tests) + 3 dirty-tracking tests
  • model-inheritance.test.ts — metadata isolation, SHACL, polymorphic findAll tests
  • model-where-operators.test.ts — 9 tests covering all WhereCondition operators (IN, not, contains, gt, gte, lt, lte, between)
  • model-prolog.test.ts — 5 pure-function tests for generatePrologFacts() + 2 executor infer() tests
  • Unit tests for normalizeParentQuery() and queryToSurrealQL() with parent and linkedFrom options in Ad4mModel.test.ts

Reorganisation:

  • Split monolithic multi-user-simple.test.ts into 8 focused test files
  • Reorganised tests into subfolders (auth/, model/, sdna/, multi-user/)
  • Renamed prolog-and-literals.test.tssdna.test.ts; removed duplicate model tests from it
  • Script renames across the monorepo: test-maintest, test-alltest-run, test-runtest:ci; removed legacy aliases; combined auth scripts; folded test-from-json-schema into test-model

Test runner fixes:

  • Replaced stale findAndKillProcess('holochain') / findAndKillProcess('lair-keystore') with findAndKillProcess('ad4m') — everything is a single ad4m-executor process since the Rust refactor
  • Switched from downloading stale pre-built language bundles (CJS, required ESM conversion) to the bootstrap seed (tests/js/bootstrapSeed.json); removed node-wget-js and unzipper dependencies
  • Shell EXIT trap added to the Mocha run script — guarantees executor cleanup after the last suite finishes or on unexpected failure
  • AgentHandle.stop() now clears and unrefs the fallback SIGKILL timer after a clean exit so Node doesn't linger
  • injectPublishingAgent.js retries with exponential backoff — prevents flaky failures when the executor is slow to become ready
  • @BelongsToMany CI stabilisation — wrapped findOne call in waitUntil (5 s timeout, 100 ms poll) to eliminate a read-cache race on slower CI machines
  • Triple-agent DHT gossip lag fix (triple-agent-test.suite.ts) — expected 16 to equal 20 flake caused by Holochain HolochainRetreiver: Could not find entry errors; fixed by calling makeAllThreeNodesKnown() mid-test before Jim's poll, adding a 10 s pre-sleep, increasing per-retry interval 1 s → 3 s, increasing retry count 20 → 40, bumping suite this.timeout 200 s → 420 s
  • Lazy-load PUBLISHING_AGENT in runtime.suite.ts — prevents startup-ordering crash when the agent is not yet initialised at module load time
  • Added process-level unhandledRejection guard that silently swallows WebSocket 1006 close errors escaping observable chains before per-client guards fire

Key fixes found during test rebuild:

  • register() calls added to beforeEach in all model test files — SHACL definitions are stored as links and don't survive wipePerspective()
  • Transaction tests rewritten to avoid create+delete within the same batch (the runtime doesn't handle constructor+destructor for the same entity in one committed batch)
  • Fixed same-batch double-save bug: _savedOnce flag passes an alreadyExists hint to saveInstance() to skip the SurrealDB existence-check on subsequent saves in the same uncommitted batch
  • Fixed @BelongsToMany include hydration

🐛 Bug fixes

ORM layer:

  • Fixed save() create/update routing: checks SurrealDB for existing links before branching; setProperty() now encodes raw values as literal:// URIs before passing to executeAction, mirroring Rust's resolve_property_value
  • Fixed innerUpdate emitting Property X has no metadata, skipping noise for generated relation methods and un-decorated fields; setProperty now throws for truly unknown direct calls
  • Fixed quote escape no-op in fetchInstance.ts (.replace(/'/g, "'") replaced with same character)
  • Circular JSON fix in createSubjectJSON.stringify call on the subject instance threw on circular references; fixed by using instanceToSerializable() which walks only declared field metadata
  • Literal.get() strict boolean parsing — rejects payloads that are not exactly "true" or "false" instead of silently coercing via Boolean()
  • Literal.toUrl() throw on unsupported type — throws a descriptive error instead of returning a malformed URL
  • Literal-guard fix (from origin/dev merge)hydration.ts (resolveValue()) already incorporates a strictly-superior guard (resolveLanguage === "literal" || resolveLanguage === undefined) covering both explicit literal properties and plain undecorated string fields

Networking / multi-user:

  • Fixed neighbourhood signal routing: send_signal() and send_broadcast() in perspective_instance.rs only searched list_user_emails() to decide if a recipient is local; the main agent has no email so signals to it fell through to the link language; fixed to also check the main agent's DID
  • Fixed broadcast loopback — broadcasts skip local delivery when owners is None or empty; owners=None/[] treated as implicit main-agent ownership for legacy perspectives
  • Neighbourhood Apollo signal subscription disposalNeighbourhoodClient stores the ZenObservable subscription in a #signalSubscriptions map; removeSignalHandler() calls .unsubscribe() and fully cleans up both maps when the last handler for a perspective is removed, preventing a leaked WebSocket stream
  • WebSocket 1006 isSocketCloseError guard — added isSocketCloseError(e) to core/src/utils.ts; applied across all six Apollo/graphql-ws subscription clients (PerspectiveClient, AgentClient, RuntimeClient, AIClient, NeighbourhoodClient, NeighbourhoodProxy — 16 handlers total); errors silently swallowed only for the expected shutdown case
  • Fixed infer() call in perspective_instance.rs not awaiting findAndKillProcess in error handlers
  • Multi-user test suite fixes: auth flow, error strings, URI validation, bootstrap timing

📄 Documentation

  • Added AD4M-MODEL-REFACTOR.md refactor plan (continuously updated through 2026-02-24); backlog items for future PRs consolidated into core/src/model/IMPROVEMENTS.md
  • Added core/src/model/README.md — architecture overview, full Recipe example, decorator reference table, query API cheatsheet, transaction pattern, fromJSONSchema examples, inheritance notes; corrected two method name errors after initial authoring (.run().get(), .subscribe().live() on the fluent builder)
  • Added Phase G deprecation plan — documents what can be removed from PerspectiveProxy / Rust once Flux and ad4m-hooks migrate off the Subject proxy API to Ad4mModel (getSubjectData, getSubjectProxy, Rust get_subject_data())
  • Added SUBSCRIPTION_STRATEGY.md — documents client-side subscription architecture, multi-user node compatibility, shared registry design
  • Trimmed JSDoc verbosity by 24–37% across Ad4mModel.ts, decorators.ts, transaction.ts, types.ts
  • Updated CHANGELOG with 21 unreleased entries since 0.11.1 (Fixed ×11, Added ×5, Changed ×5)

Comprehensive plan for the ad4m-model-refactor branch, covering:

- Phase 0: test app scaffold in we/apps/playgrounds/react/ad4m-model-testing
  using ad4m-connect, scenario-per-phase structure
- Phase 1: SHACL→Prolog fact generator (generatePrologFacts.ts), then
  removal of all dead Prolog auto-query paths and Subject.ts
- Phase 2: decorator cleanup — @Optional/@Property/@readonly deleted,
  @Collection→@hasmany, @modeloptions@model, @Flag unchanged;
  new @HasOne/@BelongsToOne/@BelongsToMany with sh:inversePath SHACL support;
  hard rename with no deprecated aliases (consumer footprint confirmed by grep)
- Phase 3: Ad4mModel.ts decomposition into query/, schema/, prolog/ modules;
  TransactionContext API replacing raw batchId strings; JSON-first Query<T>
  with fluent builder as pure sugar; include/eager loading; subscriptions
  with onError handling and Subscription.lastError
- Phase 4: WeakMap metadata registry fixing prototype mutation inheritance bug;
  fromJSONSchema second write path; true model inheritance with
  getModelMetadata() prototype chain merge and generateSHACL() sh:node emission
- Phase 5: CRDT ordering (deferred, depends on Phase 3)

Execution order documented with test scenario cross-references.
Flux decorator rename (~15 files in packages/api) tracked as step F,
separate PR after test app validates Phase 2.
- Delete queryToProlog(), instancesFromPrologResult(), countQueryToProlog()
- Delete all build*Query helper functions (only used by deleted methods)
- Strip useSurrealDB param + Prolog else branches from findAll, findAllAndCount,
  paginate, count, and all ModelQueryBuilder methods
- Remove useSurrealDBFlag field and useSurrealDB() method from ModelQueryBuilder
- Delete @InstanceQuery decorator and InstanceQueryParams interface
- Delete generateSDNA() method from @modeloptions
- Remove prologGetter/prologSetter from PropertyOptions, PropertyMetadata, Optional(),
  Property(), ReadOnly() interfaces and JSDoc
- Rename makeRandomPrologAtom -> makeRandomId
- Update Ad4mModel.test.ts: remove all Prolog-only tests, replace prologGetter
  with getter in assertions, delete useSurrealDB(false) test cases
- AllInstancesResult kept as 'any' alias (used by PerspectiveProxy subscription infra)
…filter

queryToSurrealQL SELECT used '->link AS links' which returns target node
records (id, uri only), not edge records (predicate, target, author,
timestamp). instancesFromSurrealResult could never hydrate properties or
collections. Fixed with a correlated subquery:

  (SELECT predicate, out.uri AS target, author, timestamp
   FROM link WHERE in = $parent.id ORDER BY timestamp ASC) AS links

Separately removed all 'perspective = $perspective AND' graph traversal
filters — the link table has no perspective field (each perspective is its
own isolated SurrealDB database) and $perspective was never substituted by
surreal_query(). Filters were silent no-ops.

All 4 scenario 08 tests now passing (save, findAll, field round-trip,
locations @hasmany collection).

Also updated AD4M-MODEL-REFACTOR.md: all active issues marked resolved,
scenario 08 status updated to 4/4 passing.
- Dual hydration implementations (getData vs instancesFromSurrealResult)
- WeakMap metadata fix promoted from Phase 4 to Phase 2 (correctness bug)
- String interpolation into SurrealQL → parameterized queries (Phase 3a)
- eval() in Subject.ts/PerspectiveProxy (CSP violation, JIT hazard, Phase 1c urgency)
- generateCollectionAction duplicates Rust SHACL logic (sync dependency)
- Execution Order updated to reflect WeakMap and hydration changes
- Ad4mModel.ts: generated method is now set${cap} not setCollection${cap}
- decorators.ts: @HasMany/@hasone stubs updated; HasManyMethods<K> type updated;
  JSDoc examples updated
- util.ts: collectionSetterToName slices at 3 not 13;
  collectionToSetterName emits set${...} not setCollection${...}
- PerspectiveProxy.ts: replace startsWith('setCollection') sentinel with
  __collections-metadata-based isCollectionSetter() helper — correct even
  when setter name happens to start with 'set' without the old prefix
…asOne, WeakMap fix

- decorators.ts: WeakMap metadata registry keyed on constructor (not prototype)
  eliminates silent inheritance/data-corruption bug for subclassed models
- decorators.ts: @BelongsToOne, @BelongsToMany, @hasone decorators with
  direction='reverse'/maxCount; HasMany/HasOne accept optional model factory arg
- decorators.ts: HasManyMethods<Keys extends string> utility type exported
- Ad4mModel.ts: getModelMetadata() propagates direction + maxCount from relation
  descriptors (root-cause fix for @BelongsToOne not populating reverse fields);
  constructor skips add/remove/set stubs for direction='reverse' relations;
  getData() + instancesFromSurrealResult() handle reverse-link queries
  (WHERE out.uri = ... / maps l.source instead of l.target)
- PerspectiveProxy.ts: buildQueryFromTemplate / isCollectionSetter use
  getPropertiesMetadata / getRelationsMetadata from decorator registry
  instead of __properties / __collections prototype mutation
- generatePrologFacts.ts: CollectionMetadata -> RelationMetadata, collections -> relations
- util.ts: formatting
- AD4M-MODEL-REFACTOR.md: status updated to 2026-02-23; WeakMap + BelongsToOne
  issues marked resolved; dual-hydration downgraded to yellow
getData() and instancesFromSurrealResult() both assigned the raw values[]
array to collName unconditionally. Added maxCount === 1 check (same as the
existing reverse-relation path) so @hasone fields get values[0] ?? null
instead of an array object.
- Rename collName/collMeta/collsToFetch variables in Ad4mModel.ts
- Update all JSDoc/inline comments: collection → relation terminology
- Rename buildSurrealSelectFields/WithAggregation parameter collections? → relations?
- Remove dead imports: singularToPlural, pluralToSingular, collectionToAdderName,
  collectionToRemoverName, collectionToSetterName
- Rename wire action "collectionSetter" → "relationSetter" full stack:
  - Rust: #[serde(rename)] + enum variant CollectionSetter → RelationSetter + match arm
  - Ad4mModel.ts: actionMap setter string
  - PerspectiveProxy.ts: JSDoc union, bullet description, isCollectionSetter → isRelationSetter
  - tests/js/sdna/subject.pl: all 3 collection_setter action string fixtures
…iteral encoding

- save() now checks SurrealDB for existing links before branching:
  - new instance → createSubject + has_child + innerUpdate(relations only)
  - existing instance → innerUpdate(props + relations), no duplicate links
- setProperty() encodes raw values as literal:// URIs before passing to
  executeAction, mirroring Rust's resolve_property_value in create_subject.
  Values already carrying a URI scheme are passed through unchanged.
- update() deprecated; delegates to save() for backwards compatibility
- Rust create_subject merge fix: *cmd = Command { ..setter_cmd.clone() }
  preserves SetSingleTarget action instead of keeping AddLink from constructor
- AD4M-MODEL-REFACTOR.md updated: completed table, pending items, two new
  resolved issue entries
- Add `flag?: boolean` to PropertyOptions interface in decorators.ts
- generatePropertySetterAction() throws if metadata.flag is set —
  prevents accidental setter generation for flag fields
- innerUpdate() skips flag fields in the setProperties pass —
  flags are written once by createSubject constructor action;
  re-writing them via setSingleTarget on every save() would corrupt
  the flag link
- Scenario 08 +1 test: flag value survives re-save and findAll()
  still returns the instance via the flag predicate+value filter
- Mark Phase 2 COMPLETE in AD4M-MODEL-REFACTOR.md
Split the 3,917-line monolith into four dedicated files:

- model/types.ts          — shared types (WhereCondition, Query, ModelMetadata, …)
- model/query/SurrealQueryBuilder.ts — pure SurrealQL query-building helpers
  (buildSurrealQuery, buildSurrealCountQuery, formatSurrealValue, matchesCondition, …)
- model/schema/fromJSONSchema.ts     — JSON Schema → dynamic model factory
  (createModelFromJSONSchema + determineNamespace, determinePredicate, …)
- model/query/QueryBuilder.ts        — ModelQueryBuilder<T> fluent query-builder class

Ad4mModel.ts is now 2,290 lines (was 3,917).  All extracted symbols are
re-exported from Ad4mModel.ts so consumers see no API change.

build: pnpm tsc --noEmit passes with 0 errors; pnpm build succeeds.
Create query/hydration.ts with two shared functions:
  hydrateInstanceFromLinks(instance, links, metadata, perspective)
  evaluateCustomGetters(instance, perspective, metadata)

Both getData() (single-instance) and instancesFromSurrealResult() (bulk)
previously implemented the same property/relation hydration logic
independently. This caused a real divergence: getData() used 'latest-wins'
(last ASC-ordered link) while instancesFromSurrealResult() used 'first-wins'.
Both now use latest-wins via the shared implementation.

Ad4mModel.ts: 2,290 → 1,881 lines (-409 lines)
query/hydration.ts: 259 lines (new)

Deleted: private static evaluateCustomGettersForInstance (~73 lines)
Replaced: getData() 307 → ~170 lines; instancesFromSurrealResult() per-row
  loop 430 → 26 lines (the rest of the bulk path unchanged).
…ts (Phase 3a Part 3)

Move the following 9 static methods out of Ad4mModel.ts into the new
core/src/model/query/operations.ts module as standalone functions that
take a model constructor (ctor: Ad4mModelCtor<T>) as their first arg:

  queryToSurrealQL, countQueryToSurrealQL,
  instancesFromSurrealResult (was ~320 lines alone),
  _findAllInternal, findAll, findOne,
  findAllAndCount, paginate, count

Ad4mModel.ts keeps thin static-method wrappers for full API compatibility.

Ad4mModel.ts: 1,885 → 1,506 lines (-379)
operations.ts: 434 lines (new)

All imports trimmed accordingly:
  - escapeSurrealString removed from Ad4mModel.ts (now in operations.ts)
  - Internal SurrealQueryBuilder import reduced to formatSurrealValue only
  - ops namespace import added

pnpm tsc --noEmit: 0 errors
…e 3a Part 4)

Move getModelMetadata() body and assignValuesToInstance() body out of
Ad4mModel.ts into the new core/src/model/schema/metadata.ts module:

  getModelMetadataForClass(ctor)     — decorator-registry + JSON-Schema fallback
  assignValuesToInstance(p, i, vals) — batch (name, value, resolve?) hydration
  ModelValueTuple                    — replaces file-private ValueTuple type

Ad4mModel.ts keeps public static wrappers (both are API surface).

Import changes in Ad4mModel.ts:
  - isArrayType, determinePredicate, determineNamespace removed (→ metadata.ts)
  - internal import type { JSONSchemaProperty } removed (re-export kept)
  - ValueTuple private type removed (replaced by ModelValueTuple from metadata.ts)

Ad4mModel.ts: 1,516 → 1,316 lines (-200)
schema/metadata.ts: 241 lines (new)

pnpm tsc --noEmit: 0 errors
…rClass

- Drop Ad4mModel.assignValuesToInstance() static wrapper — zero callers
  anywhere; assignValuesToInstance() remains available from schema/metadata
- Rename getModelMetadataForClass -> getModelMetadata in schema/metadata.ts
  so the standalone function name matches the static method it backs
- The static method now uses import alias _getModelMetadata to avoid shadowing
- ModelValueTuple export removed (had no external consumers)

Ad4mModel.ts: 1,316 -> 1,304 lines
pnpm tsc --noEmit: 0 errors
…a Part 5)

Move the 212-line single-instance hydration pipeline out of Ad4mModel.getData()
into fetchInstanceData(instance, perspective, baseExpression, metadata) in the
new query/fetchInstance.ts module.

  - 6-stage pipeline: forward links, post-filters, reverse relations,
    relatedModel eager hydration, custom getters, isInstance filtering
  - _findAllInternal calls updated to use standalone ops._findAllInternal(ctor,...)
  - Internal hydrateInstanceFromLinks / evaluateCustomGetters imports removed
    from Ad4mModel.ts (still re-exported for consumers)

Ad4mModel.ts: 1,304 -> 1,102 lines (-202)
fetchInstance.ts: 212 lines (new)
pnpm tsc --noEmit: 0 errors
Add Ad4mModel.transaction(perspective, async tx => { ... }) static method
that handles batch lifecycle automatically.

New files:
  core/src/model/transaction.ts
    - TransactionContext interface: { batchId, perspective }
    - runTransaction(): creates batch, commits on success, logs + rethrows on error
      (PerspectiveProxy has no abortBatch — uncommitted batch is discarded by runtime)

Ad4mModel changes:
  - import runTransaction; re-export TransactionContext
  - static transaction<T>(perspective, callback) delegates to runTransaction()

Usage:
  // Before (fragile — leaked batch if any save throws):
  const batchId = await perspective.createBatch();
  await a.save(batchId);
  await b.save(batchId);
  await perspective.commitBatch(batchId);

  // After (safe):
  await Ad4mModel.transaction(perspective, async (tx) => {
    await a.save(tx.batchId);
    await b.save(tx.batchId);
  });

pnpm tsc --noEmit: 0 errors
…d imports

New file: core/src/model/mutation.ts (~431 lines)
  - MutationContext interface
  - generatePropertySetterAction / generateRelationAction (pure builders)
  - setProperty / setRelationSetter / setRelationAdder / setRelationRemover
  - cleanCopy / innerUpdate / saveInstance

Ad4mModel.ts: 1,137 → 759 lines (-378)
  - Remove #subjectClassName (dead field — set but never read)
  - Remove 10 private methods; replace with #mutationContext() + delegation
  - save() / get() are now 1-liners
  - Drop unused imports: Link, Model, propertyRegistry, relationRegistry,
    propertyNameToSetterName, duplicate formatSurrealValue import,
    and 6 type-only imports only needed by the re-export block
Add `include?: string[]` to the Query type, replacing the TODO comment
on the old `relations` field. Implements opt-in, explicit eager loading:

  - When `include` is NOT set: legacy behaviour preserved — all relations
    with a `relatedModel` factory are batch-hydrated automatically.
  - When `include` IS set: ONLY the listed relations are hydrated, using
    whichever class source is available (relatedModel factory or
    where.isInstance). Gives callers precise control over which sub-graphs
    to load without N+1 queries.

Changes:
  - types.ts: add `include?: string[]`, deprecate `relations`
  - query/operations.ts: instancesFromSurrealResult step 3 honours include
  - query/QueryBuilder.ts: add .include() fluent method

Usage:
  Recipe.query(perspective).include(['author', 'comments']).get()
  Recipe.findAll(perspective, { include: ['author'] })
Breaking change: include is now a map, not a string array.
No auto-hydration — relations stay as bare IDs unless include is set.

New type:
  IncludeMap = { [relationName: string]: true | Query }

API:
  Recipe.findAll(perspective, {
    include: {
      comments: true,
      tags: { where: { active: true }, limit: 5 },
    },
  });

  Recipe.query(perspective)
    .include({ comments: true })
    .get();

  // Single-instance get() also accepts include:
  await recipe.get({ comments: true });

Changes:
  - types.ts: add IncludeMap, update Query.include, deprecate relations
  - operations.ts: step 3 only fires when include present; passes sub-query
  - fetchInstance.ts: forward + reverse relation hydration gated on include
  - QueryBuilder.ts: .include() now accepts IncludeMap
  - Ad4mModel.ts: get(include?) + getData(include?) wired through; re-export IncludeMap
- Add Phase 3a (file decomposition), 3b (Transaction API), 3c (IncludeMap)
  rows to completed status table
- Populate pending phases table: 3a/b/c done, 3d next, 4 and 5 pending
- Move 'save() batch lifecycle' and 'dual hydration' from Open to Resolved
- Mark Phase 3 heading as PARTIALLY COMPLETE (3a/3b/3c done; 3d next)
- Mark 3b and 3c sub-sections as COMPLETE with commit hashes
- Add IncludeMap design note to 3c (Prisma-style vs planned Include<T>[])
- Mark 3d as NEXT, update execution order with checkmarks
- Add core/src/model/subscription.ts:
  - createSubscription<T>() — fires callback immediately + on every relevant
    link-added / link-removed from the perspective
  - Relevance check: only re-queries when a changed link's predicate is one
    the model owns (properties + relations + ad4m://has_child); optional
    source filter further narrows the watch
  - Debounce support: rapid writes within the window trigger one re-query
  - Error handling: re-query errors and callback errors both route to
    onError (default: console.error); lastError exposed on Subscription handle

- Add to types.ts:
  - SubscribeOptions — Omit<Query, 'count'> + debounce + onError
  - Subscription — { unsubscribe(): void; readonly lastError: Error | null }

- Add to Ad4mModel.ts:
  - static subscribe<T>(perspective, options, callback): Subscription
  - Re-export SubscribeOptions, Subscription from ./types

- Add to query/QueryBuilder.ts:
  - .live(callback, deliveryOpts?): Subscription — fluent terminal alongside
    .get() / .subscribe(), uses accumulated queryParams + optional debounce/onError
has_child was a legacy mechanism for scoping instances to a parent
container node. It is superseded by explicit relation decorators
(@hasmany, @BelongsToOne etc.) which already encode the relationship
predicate in the graph. Every live perspective contains meaningless
source → ad4m://has_child → baseExpression links, most of them from
the magic default 'ad4m://self' string.

Removed:
- Ad4mModel constructor third param (source)
- Ad4mModel.#source private field
- MutationContext.source field
- has_child link write in saveInstance create path
- Link import from mutation.ts (now unused)
- Query.source field
- source filter in buildSurrealQuery (SurrealQueryBuilder.ts)
- ModelQueryBuilder.source() fluent method
- 'ad4m://has_child' from subscription watchedPredicates
- Source-scoped relevance check in subscription.ts isRelevant()
…ns() fluent method

Use include: IncludeMap for eager loading instead.
- SHACLShape: add parentShapes field, addParentShape(), emit sh:node
  in both toTurtle() and toLinks() serializers
- generateSHACL(): detect @Model-decorated parent class via className
  + WeakMap presence; child shape uses only own properties/relations
  and references parent shape via sh:node instead of duplicating

Runtime inheritance (findAll, save, hydration) already works via the
WeakMap prototype-chain walk in getPropertiesMetadata/getRelationsMetadata
(landed in Phase 2). Phase 4b completes the SHACL representation side.
The constant was read at module load time via fs.readFileSync, but
tst-tmp/agents/p/ad4m/agent.json is created by injectPublishingAgent.js
during test setup — after Mocha has already imported all suite files.
When cleanup.js runs between test-auth and test-integration, tst-tmp is
wiped, so the readFileSync threw ENOENT and the entire integration suite
failed to load.

Move the read into a before() hook so it executes after setup has run.
…ue proxy compatibility

JS hard-private fields (#) use WeakMap semantics that fail when 'this' is a
Vue reactive Proxy. Switching to TypeScript private (_) preserves compile-time
encapsulation while allowing Ad4mModel instances to be stored directly in Vue
reactive state without needing toRaw() wrappers.
The integration-tests-multi-user-simple CI job ran test-multi-user.sh
without any prepare-test step. On a fresh workdir, tst-tmp/languages/
is empty so the executor's language-language can't serve bundle-{hash}.js
for the agent language. installLanguage throws, #agentLanguage stays
null, but the HTTP server starts anyway (Rust layer is independent).

This caused byDID() and updatePublicPerspective() to fail with
'No Agent Language installed!' while the 6 other tests that don't call
getAgentLanguage() passed.

Fix: run 'pnpm run prepare-test' at the top of test-multi-user.sh before
any suites. This starts a publishing executor, publishes all system
languages, and writes bundle-{hash}.js files to tst-tmp/languages/ so
subsequent per-suite executors find them locally.

Also bump the multi-user job no_output_timeout 30m -> 60m to accommodate
the extra prepare-test overhead.
…ix circular JSON in createSubject; BelongsToMany retry fix

- PerspectiveClient.ts, AgentClient.ts, AIClient.ts, ExpressionClient.ts,
  LanguageClient.ts, NeighbourhoodClient.ts, RuntimeClient.ts, Ad4mClient.ts:
  silently ignore expected WebSocket 1006 close events in all subscription
  error handlers (isSocketCloseError guard)
- core/src/model/mutation.ts: filter Object.entries to only decorator-registered
  fields in createSubject, preventing circular JSON from private _perspective field
- core/src/model/decorators.ts: minor cleanup
- connect, ad4m-hooks: package.json / esbuild script updates
- pnpm-lock.yaml: updated lockfile
…nc test on CI; improve diagnostics on failure

- Use more generous retry count and sleep interval when running in CI
- Log actual link counts and attempts on failure for all three agents/phases
- Should reduce flakiness in slow or concurrent CI environments
…ingerprint for DRYness and circular JSON safety

- Move decorator-registered property extraction to a shared helper in decorators.ts
- Use instanceToSerializable in subscription.ts stableFingerprint
- Prevents circular structure errors and keeps code DRY
- Add resetBootstrapSeed.js to clear only trustedAgents and languageLanguageBundle
  before each prepare-test (not language addresses, which are stable config)
- Run resetBootstrapSeed.js at the start of prepare-test:default and :windows
- Fix bootstrapSeed.json committed state: remove stale publishing-agent DID and
  clear languageLanguageBundle (run cleanTestingData.js to restore baseline)
- Fix resetBootstrapSeed.js to use ESM import instead of require() (package is type:module)

Best practice: bootstrapSeed.json stays in git with stable language addresses and
a single baseline trustedAgent DID. Dynamic fields (extra DIDs, bundle) are cleared
before each run and must not be committed. Run cleanTestingData.js before committing.
The job was running test:ci without ever calling prepare-test, relying on
the self-hosted runner's persistent workdir state. This caused 'No agent
language' and ENOENT failures on fresh runs or when state was stale.

Add an explicit prepare-test step before test:ci with a 20m timeout.
Two concurrent CI jobs (integration-tests-js and integration-tests-multi-user-simple)
both run after build-and-test on the same coasys/marvin self-hosted runner.
Both call publishTestLangs.ts which hardcoded port 15700 for the setup-phase
executor, causing one job's executor to panic with 'Address already in use'.

Fix 1 — unique port ranges per job (config.yml):
  integration-tests-js:              AD4M_SETUP_GQL_PORT=15700/15701/15702
  integration-tests-multi-user-simple: AD4M_SETUP_GQL_PORT=15710/15711/15712
  Each job also gets a kill step that targets only its own port range.
  lsof flag fixed: -s TCP:LISTEN added so we only kill the server, not clients.

Fix 2 — waitForPortFree (publishTestLangs.ts):
  After kill -9, poll with lsof until the port shows no LISTEN process (up to
  10s) before starting the new executor. Closes the race window between the
  signal delivery and OS socket reclaim. Falls back gracefully with a warning.
update() was a thin alias for save() kept for backwards compatibility.
Removed in favour of save() which already auto-detects create vs update.
… errors

When the executor is killed in the outermost after-all hook, all active
GraphQL subscriptions receive a 1006 (abnormal closure) close event. The
per-subscription error handlers already suppress these for logging, but
there is a race on slower CI machines: if zen-observable has already
cleaned up a subscription's observer by the time graphql-ws fires the
close error, observer.error() throws rather than routing to our handler,
creating a new unhandled Promise rejection. Mocha catches this via its own
unhandledRejection listener and fails the after-all hook despite all 66
tests passing.

Fix: process.on('unhandledRejection') guard in setup.ts that swallows 1006
errors and re-throws everything else. Single location, test-only file,
does not touch production code.
- TC39_STAGE_3_DECORATORS.md: migration plan from legacy TS decorators to TC39 Stage 3
- SHACL_MUTATION_DECOUPLING.md: plan to remove SHACL from mutation hot path
- PREDICATE_COLLISION_FIX.md: analysis and fix for predicate collision causing data duplication
…tests

- Add snapshot-based dirty tracking (snapshot.ts) to prevent save() from
  re-writing unchanged relations (fixes channel pinning duplication)
- Fix properties projection to only delete schema-declared fields, preserving
  internal ORM machinery (_id, _perspective, addX/removeX/setX methods)
- Move properties:[] empty-array guard before hydration loop so error is thrown
- Fix include sub-query order preservation when sub-entry has order clause
- Wire dirty tracking into mutation.ts (isDirty checks in innerUpdate)
- Wire snapshot capture into operations.ts and fetchInstance.ts after hydration
- Reorganize model-query.test.ts into 16 clean sections (~1100 lines)
- Add 3 dirty-tracking tests to model-transactions.test.ts
… order test

surrealCompiler.ts:
- Extend buildGraphTraversalWhereClause to handle relation names in the where
  clause (previously silently skipped as 'no propMeta found').
- Forward relations (@hasmany / @hasone): generates ->link[WHERE predicate AND
  out.uri = value] > 0 graph-traversal filter.
- Reverse relations (@BelongsToOne / @BelongsToMany): generates <-link[WHERE
  predicate AND in.uri = value] > 0 filter.
- Array (IN), not, and not-IN variants all supported; comparison ops fall
  through to the existing JS post-filter.

model-query.test.ts:
- Add 'order by multiple fields — primary sort then secondary sort' integration
  test covering compound ORDER BY semantics (title ASC, viewCount ASC).
- Various additional coverage for where/order/limit/offset, IncludeMap
  sub-queries, nested include, edge cases, and properties projection.
- build-and-test -> build
- integration-tests-js -> integration-tests-core
- integration-tests-multi-user-simple -> integration-tests-multi-user
- remove integration-tests-email-verification (covered by test-auth in test:ci)
…y support

- Replace useModel (React + Vue) with useLive hook that supports:
  - parent scope: restrict results to children of a parent node via @hasmany
  - single-instance mode (id option) returning LiveInstanceResult
  - load-more / infinite-scroll pagination
  - preserveReferences mode for stable object identity
  - string model name override for generic class browsers

- Add Query.linkedFrom: { id, predicate } for parent-scoped SurrealDB queries
  - Compiles to count(<-link[WHERE predicate='...' AND in.uri='...']) > 0
  - Uses the same graph-traversal pattern as @BelongsToOne where clauses
  - Subscription layer watches linkedFrom.predicate so child additions/removals
    trigger live re-queries

- Update subscription.ts to watch linkedFrom predicate in addition to model predicates

- Update PREDICATE_COLLISION_FIX.md recommendation to Path 3 (hybrid:
  type-filtered mutation + collision warning, no forced migration)
…ead helpers package

- useLive: bail out early when parent scope is declared but parent.id is
  falsy (set during web-component property assignment race) — prevents
  escapeSurrealString(undefined) TypeError in SurrealDB count query
- decorators: instanceToSerializable now includes @HasMany/@hasone relation
  fields in the serialised fingerprint so subscription debounce detects
  relation-only changes (e.g. addTags) and re-broadcasts correctly
- model-subscriptions.test: add regression test for @hasmany re-fire
  (test 9); register TestComment/TestTag in before/beforeEach
- remove ad4m-hooks/helpers package entirely (src was empty; gutted in
  ADAM model migration) — deleted directory, removed from
  pnpm-workspace.yaml, package.json, publish.yml, publish_staging.yml
- Ad4mModel.create() now accepts { parent: { model, id, field? } } option
  that atomically links the new instance to its parent in the same call,
  removing the need for a manual perspective.add(new Link(...)) afterward.

- field is optional: when omitted, resolveParentPredicate() scans the
  parent's @hasmany relations and infers the predicate automatically.
  Throws a descriptive error if zero or multiple matches are found.

- Ad4mModel.delete() now automatically removes all incoming links
  ({ target: this.id }) before calling removeSubject(), so parent
  collection queries (linkedFrom) stay consistent without callers
  needing to pass parent info.

- ParentOptions type and resolveParentPredicate exported from core index.

- useLive ParentScope.field made optional in both React and Vue hooks;
  resolveParentPredicate used internally with a console.warn fallback.

- Tests added for resolveParentPredicate (explicit field, inferred field,
  bad field, no match, missing childCtor, ambiguous relations).
…ation tests

- Add Query.parent option (ParentQueryByPredicate | ParentQueryByModel) to
  findAll(), findOne(), subscribe(), and createSubscription()
- Add normalizeParentQuery() to resolve model-backed form to { id, predicate }
- Wire parent through SurrealQL compiler and subscription filter
- Change useLive() signature in React and Vue hooks: perspective is now arg 2
  (useLive(Model, perspective, options?)) rather than inside options object
- Remove redundant field prop from parent queries where predicate is unambiguous
- Add TestChannel model to integration test suite
- Add parent query integration tests (section 18) to model-query.test.ts
- Add parent-scoped subscription integration tests (section 10) to
  model-subscriptions.test.ts
- Add unit tests for normalizeParentQuery() and queryToSurrealQL with parent
- Rename useLive.ts -> useLiveQuery.ts in both ad4m-hooks/react and ad4m-hooks/vue
- Rename exported function useLive -> useLiveQuery in both hook files
- Update index.ts re-exports in both packages
- Update remaining comments and console.warn strings
- ParentScope is now { id, predicate } | { model, id, field? }
- resolveParentQuery() short-circuits on the raw form via 'predicate' in parent
- Fix dependency array TS error: parent?.field narrowed to parent && 'field' in parent
- Applied to both React and Vue useLiveQuery hooks
- options now accepts { parent?, batchId? }
- save() and addLinks() both receive batchId when provided
- addLinks() replaces perspective.add() so the parent link is also batched
- Ad4mModel.update(perspective, id, data, batchId?) fetches the existing
  instance, merges the partial data, then saves — replaces the common
  three-step new/assign/save pattern
- Ad4mModel.delete(perspective, id, batchId?) constructs an instance by
  id and calls instance.delete() — replaces the new/delete pattern
- Unit tests added in Ad4mModel.test.ts (mocked)
- Integration tests added in tests/js model-core suite
jhweir added 5 commits March 4, 2026 14:49
# Conflicts:
#	.github/workflows/publish_staging.yml
#	core/src/neighbourhood/NeighbourhoodClient.ts
#	executor/esbuild.ts
#	executor/package.json
#	pnpm-lock.yaml
#	rust-executor/package.json
#	tests/js/scripts/cleanup.js
#	tests/js/tests/integration.test.ts
#	tests/js/tests/language.ts
#	tests/js/tests/social-dna-flow.ts
#	tests/js/utils/utils.ts
#	turbo.json
The language-controller-refactoring moved languages into separate Deno
workers where agentProxy.did is frozen at load time. This broke managed
user publishing to the agent language because createPublic always saw
the main agent's DID.

Fix: when expression_create is called with a managed user AgentContext,
temporarily swap did, signingKeyId, and createSignedExpression on the
agentProxy before calling createPublic, then restore in finally.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant