Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
391ea28
feat: add SHACL core data structures
data-coasys Jan 30, 2026
7d56e4c
feat: add SHACL generation to ModelOptions decorator
data-coasys Jan 30, 2026
3f06eed
docs: add SHACL migration progress tracker
data-coasys Jan 30, 2026
5003f1a
feat: add SHACL storage methods to PerspectiveProxy
data-coasys Jan 30, 2026
f003bfe
feat: integrate SHACL into ensureSDNASubjectClass workflow
data-coasys Jan 30, 2026
2c8dfee
docs: update progress - core implementation complete
data-coasys Jan 30, 2026
dd2529a
feat(shacl): Add named property shapes for queryable SHACL
data-coasys Jan 31, 2026
1c6b894
feat(model): Populate 'name' field in SHACL property shapes
data-coasys Jan 31, 2026
f0d34dd
fix(perspectives): Use correct Literal API in SHACL methods
data-coasys Jan 31, 2026
fbc6646
test(shacl): Add test for named property shapes generation
data-coasys Jan 31, 2026
a5766dc
feat(shacl): Extract property name in fromLinks() for named shapes
data-coasys Jan 31, 2026
ef0e14c
chore: Remove Deno test file from TypeScript build
data-coasys Jan 31, 2026
f0f3056
docs: Add comprehensive SHACL architecture analysis
data-coasys Jan 31, 2026
a84f2f1
feat(shacl): Add Rust SHACL parser and integrate with add_sdna()
data-coasys Feb 2, 2026
9d2941b
feat(shacl): Wire TypeScript SHACL generation to Rust backend
data-coasys Feb 2, 2026
a5bb3b9
docs: Add SHACL implementation completion summary
data-coasys Feb 2, 2026
919bbc6
refactor(shacl): Remove Prolog generation, use SHACL exclusively
data-coasys Feb 2, 2026
8a8c1cc
fix(shacl): Fix Rust borrow error and unused variable warning
data-coasys Feb 2, 2026
5c64679
fix(shacl): Fix extract_namespace() logic
data-coasys Feb 2, 2026
82b62b5
refactor(shacl): Restore Prolog generation alongside SHACL
data-coasys Feb 2, 2026
2836b24
docs: Final SHACL architecture explanation
data-coasys Feb 2, 2026
2b56387
wip(shacl): Start Prolog removal, force test failures to find depende…
data-coasys Feb 2, 2026
7b1a72c
docs: Clear status and path forward for Prolog removal
data-coasys Feb 2, 2026
f1c3ff2
docs(shacl): Complete link structure overview with W3C conformance
data-coasys Feb 2, 2026
b99eb7b
docs(shacl): Unified separate links approach for Classes + Flows
data-coasys Feb 2, 2026
d5c928b
feat(shacl): Add action infrastructure for constructor/destructor/pro…
data-coasys Feb 3, 2026
c2c62b2
feat(shacl): Implement SHACL-first action execution with Prolog fallback
data-coasys Feb 3, 2026
40a197c
fix(shacl): Fix tuple destructuring and link normalization before sig…
data-coasys Feb 3, 2026
895d007
feat(shacl): Remove Prolog fallbacks - SDNA is now SHACL-only
data-coasys Feb 3, 2026
d6cadd0
fix(shacl): Remove duplicate ad4m://has_subject_class link from parse…
data-coasys Feb 3, 2026
6a8c810
feat(shacl): Add Prolog SDNA to SHACL links parser for backward compa…
data-coasys Feb 4, 2026
c3aed79
style: cargo fmt
data-coasys Feb 4, 2026
5e7dbfc
test: add comprehensive SHACLShape tests
data-coasys Feb 4, 2026
9985989
docs: add deprecation guidance for sdnaCode parameter
data-coasys Feb 4, 2026
9d67a3b
docs: recommend addShacl() with SHACLShape type over JSON strings
data-coasys Feb 4, 2026
a1fa6ef
feat(shacl): Add SHACLFlow type for SHACL-based state machines
data-coasys Feb 4, 2026
67df25f
fix: Address CodeRabbit review comments
data-coasys Feb 4, 2026
ce9fda9
style: cargo fmt
data-coasys Feb 4, 2026
248cae2
fix: Address additional CodeRabbit review comments
data-coasys Feb 4, 2026
3f7e7c1
fix: TypeScript build errors
data-coasys Feb 4, 2026
9ecd607
fix: Rust borrow checker error in collection name matching
data-coasys Feb 4, 2026
94c641f
fix: SHACLShape constructor auto-derives shape URI + remove unused sh…
data-coasys Feb 5, 2026
c68ea30
fix: Address CodeRabbit review comments - security and data handling
data-coasys Feb 5, 2026
8d80531
test: Add comprehensive toJSON/fromJSON tests for SHACLShape
data-coasys Feb 5, 2026
6aebe1f
refactor: DRY - import AD4MAction from SHACLShape instead of duplicating
data-coasys Feb 5, 2026
f9a56e1
feat: Remove Prolog code storage - SHACL is now source of truth
data-coasys Feb 5, 2026
cbf31e7
fix: Generate minimal Prolog facts from SHACL links for backward comp…
data-coasys Feb 5, 2026
886ac3a
Merge branch 'dev' into feat/shacl-sdna-migration
lucksus Feb 9, 2026
98a7b6b
fix: Generate comprehensive Prolog facts from SHACL links for backwar…
data-coasys Feb 11, 2026
32ab946
Merge remote-tracking branch 'origin/dev' into feat/shacl-sdna-migration
data-coasys Feb 11, 2026
e30c7e9
fix(sdk): Pass SHACL JSON to backend in addSdna calls
data-coasys Feb 11, 2026
ab3cbbf
fix: Disable Prolog mode, implement SHACL-only subject class lookup
data-coasys Feb 11, 2026
5e4d2b3
fix: resolve compilation errors in SHACL-only mode
data-coasys Feb 11, 2026
b2af23b
fix(tests): Add shaclJson argument to mock PerspectiveResolver
data-coasys Feb 11, 2026
a75d90c
feat: SHACL-based subject class lookup (Prolog-free)
data-coasys Feb 12, 2026
f843ab7
debug: Add logging to trace SHACL link storage and lookup
data-coasys Feb 12, 2026
9e4ef8d
Remove Prolog fallbacks from Ad4mModel system - make fully SHACL-native
data-coasys Feb 12, 2026
b12b0f5
Fix SurrealDB queries in getSubjectClassMetadataFromSDNA
data-coasys Feb 12, 2026
14ae3c0
Fix SHACL class query - use string::ends_with instead of CONTAINS
data-coasys Feb 12, 2026
8c1b1ba
Rewrite getSubjectClassMetadataFromSDNA to use link API
data-coasys Feb 12, 2026
edd2690
Restore ad4m://sdna link storage for backward compatibility
data-coasys Feb 12, 2026
81cdd03
test: add test for parse_prolog_sdna_to_shacl_links
data-coasys Feb 13, 2026
dbefc57
chore: add debug logging for SHACL link flow tracing
data-coasys Feb 13, 2026
e520841
chore: use warn level logging for SHACL debugging to ensure visibilit…
data-coasys Feb 13, 2026
1cb0841
chore: add UUID to add_sdna logging for correlation
data-coasys Feb 13, 2026
529e0f7
chore: add database-level logging for SHACL link storage and retrieval
data-coasys Feb 13, 2026
6a83126
Merge remote-tracking branch 'origin/dev' into feat/shacl-sdna-migration
data-coasys Feb 13, 2026
2199f51
fix: align find_subject_class_from_shacl_by_query with SHACL link pat…
data-coasys Feb 15, 2026
1e58e69
chore: retrigger CI
data-coasys Feb 15, 2026
e1c8437
Merge branch 'dev' into feat/shacl-sdna-migration
jhweir Feb 16, 2026
8757a26
getSubjectClassMetadataFromSDNA made public
jhweir Feb 16, 2026
d31d107
feat: Complete SHACL migration with Prolog-disabled support
jhweir Feb 17, 2026
d400594
Cargo fmt
jhweir Feb 17, 2026
4178ded
refactor: Apply CodeRabbit improvements for SHACL query optimization …
jhweir Feb 17, 2026
15f3a22
fix: collection name double-pluralization in queries and legacy SDNA
jhweir Feb 17, 2026
b865d25
feat(shacl): fix subject instance creation and collection filtering
jhweir Feb 18, 2026
9edc0fa
Merge branch 'dev' into feat/shacl-sdna-migration
jhweir Feb 18, 2026
82309f6
Lock file updates
jhweir Feb 18, 2026
f9adedf
Cargo fmt
jhweir Feb 18, 2026
ae8d0bc
fix(shacl_parser): unwrap Result in test_extract_namespace assertions
jhweir Feb 18, 2026
dcd44df
fix: prevent invalid URIs from empty literals and bare SHACL node kinds
jhweir Feb 18, 2026
e810916
refactor: use batching for SHACL/Flow link writes, single surreal que…
HexaField Feb 19, 2026
844d08b
Merge pull request #685 from coasys/fix/shacl-batching-and-imports
lucksus Feb 19, 2026
61ad475
Merge branch 'dev' into feat/shacl-sdna-migration
lucksus Feb 19, 2026
3ccfb38
refactor: make sdnaCode optional in addSdna API
data-bot-coasys Feb 19, 2026
f2898f7
fix: restore Prolog disabled warnings while returning empty matches
data-bot-coasys Feb 19, 2026
56d81cc
refactor: extract SHACL→Prolog compat code to separate module
data-bot-coasys Feb 19, 2026
dd0ee70
test: remove skipped Prolog-only tests
data-bot-coasys Feb 19, 2026
9e9b564
fix: make sdnaCode nullable in GraphQL schema
data-bot-coasys Feb 23, 2026
49f411c
Merge branch 'dev' into feat/shacl-sdna-migration
lucksus Feb 23, 2026
6e91e01
refactor: address PR review comments for SHACL SDNA migration
data-bot-coasys Feb 23, 2026
beb212f
fix: remove non-existent 'collection' property from SHACLPropertyShap…
data-bot-coasys Feb 23, 2026
5c48831
Convert ends_with to SurrealDB queries, migrate flow methods to SHACL…
data-bot-coasys Feb 23, 2026
708cc28
fix: migrate test addSdna calls to ensureSDNASubjectClass (Prolog→SHA…
data-bot-coasys Feb 23, 2026
bc405d1
test: add SHACL-based flow test for TODO workflow
data-bot-coasys Feb 23, 2026
80b75d8
refactor: remove subjectClassesFromSHACL GraphQL endpoint, use link q…
data-bot-coasys Feb 23, 2026
cc88ffe
fix: replace SQL LIKE with SurrealQL string::starts::with in getFlow()
data-bot-coasys Feb 23, 2026
5e0ae4d
fix: correct SurrealQL function name string::starts_with (not starts:…
data-bot-coasys Feb 23, 2026
e68adc7
Update core/src/perspectives/PerspectiveClient.ts
lucksus Feb 23, 2026
dbb1660
refactor: replace Prolog query-based SubjectClassOption with client-s…
data-bot-coasys Feb 23, 2026
2a96113
refactor: replace findClassByProperties with single SurrealDB query
data-bot-coasys Feb 23, 2026
4e480bd
fix: subjectClassesByTemplate falls back to findClassByProperties
data-bot-coasys Feb 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,5 @@ rust-executor/CUSTOM_DENO_SNAPSHOT.bin
rust-executor/test_data

.npmrc
docs-src/
.worktrees/
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

255 changes: 255 additions & 0 deletions SHACL_SDNA_ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,255 @@
# SHACL SDNA Architecture

**Replacing Prolog-based Subject DNA with W3C SHACL + AD4M Extensions**

## Overview

AD4M uses Subject DNA (SDNA) to define data schemas and their operational behavior. This document describes the migration from Prolog-based SDNA to a SHACL-based approach that:

1. **W3C Conformant** - Schema definitions use standard SHACL predicates
2. **Extensible** - AD4M-specific actions use a separate `ad4m://` namespace
3. **Queryable** - All metadata stored as links, queryable via SurrealQL or simple link queries
4. **Consistent** - Same pattern for Classes and Flows

---

## Architecture: Separate Links Approach

### Design Principles

1. **One predicate = one action type** - Clear semantics
2. **SHACL for schema, AD4M for behavior** - Clean separation
3. **Named URIs** - Property shapes have queryable URIs (not blank nodes)
4. **JSON in literals** - Actions stored as JSON arrays in `literal://string:` format

### Link Structure

For a class named `Recipe` with namespace `recipe://`:

| Link Type | Source | Predicate | Target |
| ------------- | ----------------------- | -------------------- | ------------------------ |
| Shape Type | `recipe://RecipeShape` | `rdf://type` | `sh://NodeShape` |
| Target Class | `recipe://RecipeShape` | `sh://targetClass` | `recipe://Recipe` |
| Has Property | `recipe://RecipeShape` | `sh://property` | `recipe://Recipe.name` |
| Property Type | `recipe://Recipe.name` | `rdf://type` | `sh://PropertyShape` |
| Property Path | `recipe://Recipe.name` | `sh://path` | `recipe://name` |
| Datatype | `recipe://Recipe.name` | `sh://datatype` | `xsd://string` |
| Constructor | `recipe://RecipeShape` | `ad4m://constructor` | `literal://string:[...]` |
| Destructor | `recipe://RecipeShape` | `ad4m://destructor` | `literal://string:[...]` |
| Setter | `recipe://Recipe.name` | `ad4m://setter` | `literal://string:[...]` |
| Adder | `recipe://Recipe.items` | `ad4m://adder` | `literal://string:[...]` |
| Remover | `recipe://Recipe.items` | `ad4m://remover` | `literal://string:[...]` |

---

## Example: Complete Recipe Class

### TypeScript Definition

```typescript
@ModelOptions({
name: "Recipe",
namespace: "recipe://",
})
class Recipe {
@SubjectProperty({ through: "recipe://name", writable: true })
name: string = "";

@SubjectProperty({ through: "recipe://rating", writable: true })
rating: number = 0;

@SubjectCollection({ through: "recipe://has_ingredient" })
ingredients: string[] = [];
}
```

### W3C SHACL Links (Schema)

```turtle
# Shape definition
recipe://RecipeShape rdf:type sh:NodeShape .
recipe://RecipeShape sh:targetClass recipe://Recipe .

# Property: name
recipe://Recipe.name rdf:type sh:PropertyShape .
recipe://Recipe.name sh:path recipe://name .
recipe://Recipe.name sh:datatype xsd:string .
recipe://Recipe.name sh:maxCount 1 .
recipe://RecipeShape sh:property recipe://Recipe.name .

# Property: rating
recipe://Recipe.rating rdf:type sh:PropertyShape .
recipe://Recipe.rating sh:path recipe://rating .
recipe://Recipe.rating sh:datatype xsd:integer .
recipe://Recipe.rating sh:maxCount 1 .
recipe://RecipeShape sh:property recipe://Recipe.rating .

# Collection: ingredients
recipe://Recipe.ingredients rdf:type ad4m:CollectionShape .
recipe://Recipe.ingredients sh:path recipe://has_ingredient .
recipe://Recipe.ingredients sh:nodeKind sh:IRI .
recipe://RecipeShape sh:property recipe://Recipe.ingredients .
```

### AD4M Action Links (Behavior)

```turtle
# Constructor - creates instance with default values
recipe://RecipeShape ad4m://constructor """literal://string:[
{"action": "addLink", "source": "this", "predicate": "recipe://name", "target": ""},
{"action": "addLink", "source": "this", "predicate": "recipe://rating", "target": "0"}
]""" .

# Destructor - removes instance links
recipe://RecipeShape ad4m://destructor """literal://string:[
{"action": "removeLink", "source": "this", "predicate": "recipe://name"},
{"action": "removeLink", "source": "this", "predicate": "recipe://rating"}
]""" .

# Property setters
recipe://Recipe.name ad4m://setter """literal://string:[
{"action": "setSingleTarget", "source": "this", "predicate": "recipe://name", "target": "value"}
]""" .

recipe://Recipe.rating ad4m://setter """literal://string:[
{"action": "setSingleTarget", "source": "this", "predicate": "recipe://rating", "target": "value"}
]""" .

# Collection operations
recipe://Recipe.ingredients ad4m://adder """literal://string:[
{"action": "addLink", "source": "this", "predicate": "recipe://has_ingredient", "target": "value"}
]""" .

recipe://Recipe.ingredients ad4m://remover """literal://string:[
{"action": "removeLink", "source": "this", "predicate": "recipe://has_ingredient", "target": "value"}
]""" .
```

---

## Action Types

### Shape-Level Actions

| Predicate | Purpose | Bound To |
| -------------------- | ----------------------------- | ----------------------------- |
| `ad4m://constructor` | Create instance with defaults | `{namespace}{ClassName}Shape` |
| `ad4m://destructor` | Remove instance and links | `{namespace}{ClassName}Shape` |

### Property-Level Actions

| Predicate | Purpose | Bound To |
| ---------------- | -------------------------- | ----------------------------------------- |
| `ad4m://setter` | Set single-valued property | `{namespace}{ClassName}.{propertyName}` |
| `ad4m://adder` | Add to collection | `{namespace}{ClassName}.{collectionName}` |
| `ad4m://remover` | Remove from collection | `{namespace}{ClassName}.{collectionName}` |

### Action JSON Format

```json
[
{
"action": "addLink|removeLink|setSingleTarget|collectionSetter",
"source": "this|uuid|literal",
"predicate": "namespace://predicate",
"target": "value|*|specific_value",
"local": true // optional
}
]
```

---

## Querying SHACL Links

### Get Constructor Actions

```rust
// Find shape with constructor
let links = self.get_links(&LinkQuery {
predicate: Some("ad4m://constructor".to_string()),
..Default::default()
}).await?;

// Find one matching class name
for link in links {
if link.data.source.ends_with(&format!("{}Shape", class_name)) {
// Parse JSON from literal://string:{json}
let actions = parse_literal_json(&link.data.target)?;
return Ok(actions);
}
}
```

### Get Property Setter

```rust
// Find property shape with setter
let prop_suffix = format!("{}.{}", class_name, property_name);
let links = self.get_links(&LinkQuery {
predicate: Some("ad4m://setter".to_string()),
..Default::default()
}).await?;

for link in links {
if link.data.source.ends_with(&prop_suffix) {
let actions = parse_literal_json(&link.data.target)?;
return Ok(actions);
}
}
```

---

## Implementation Status

### Phase 1: SHACL Infrastructure (Complete)

- [x] TypeScript: Generate SHACL JSON with actions in `generateSDNA()`
- [x] TypeScript: Serialize SHACL to links in `PerspectiveProxy.ensureSdnaLinks()`
- [x] Rust: Parse SHACL JSON in `shacl_parser.rs`
- [x] Rust: Store action links with separate predicates

### Phase 2: Replace Prolog Queries (In Progress)

- [x] `get_constructor_actions()` - Try SHACL first, fallback to Prolog
- [x] `get_destructor_actions()` - Try SHACL first, fallback to Prolog
- [x] `get_property_setter_actions()` - Try SHACL first, fallback to Prolog
- [x] `get_collection_adder_actions()` - Try SHACL first, fallback to Prolog
- [x] `get_collection_remover_actions()` - Try SHACL first, fallback to Prolog
- [x] `resolve_property_value()` - Try SHACL for resolve language
- [x] TypeScript `removeSubject()` - Try SHACL for destructor actions

### Phase 3: Remove Prolog Fallbacks ✅ (Completed in this PR)

> **Note:** Prolog engines (scryer-prolog) are kept available for complex queries and
> future advanced features. Only the _fallback pattern_ is removed - SHACL is the
> single source of truth for all SDNA actions.

- [x] Remove Prolog fallbacks for action retrieval (SHACL-first is now SHACL-only)
- [x] Migrate Flows to same SHACL link pattern (`SHACLFlow` class with `toLinks()`/`fromLinks()`)
- [x] Keep scryer-prolog dependency (for complex Prolog queries later)
- [x] Refactor TypeScript to use `SHACLShape.fromLinks()` / `toJSON()` throughout
- [x] Use batched link operations (`addLinks()`) and single SurrealDB queries

---

## Benefits

1. **W3C Standard** - Interoperable with SHACL ecosystem
2. **Cleaner Runtime** - SHACL as single source for SDNA actions (Prolog still available for complex queries)
3. **Queryable** - All metadata as links in SurrealDB
4. **Debuggable** - Inspect schema as regular links
5. **Extensible** - Add new action types without schema changes

---

## Files

| File | Purpose |
| -------------------------------------------------------- | ------------------------------------------------- |
| `core/src/model/decorators.ts` | `generateSHACL()` - Creates SHACL from decorators |
| `core/src/shacl/SHACLShape.ts` | SHACL shape class with `toLinks()` |
| `core/src/perspectives/PerspectiveProxy.ts` | `ensureSdnaLinks()` - Stores SHACL as links |
| `rust-executor/src/perspectives/shacl_parser.rs` | Parses SHACL JSON, generates links |
| `rust-executor/src/perspectives/perspective_instance.rs` | Action retrieval with SHACL-first |
2 changes: 2 additions & 0 deletions core/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,5 @@ export * from "./ai/AIClient"
export * from "./ai/Tasks"
export * from "./runtime/RuntimeResolver"
export * from './model/Ad4mModel'
export * from './shacl/SHACLShape'
export { SHACLFlow, FlowState, FlowTransition, LinkPattern, FlowableCondition } from './shacl/SHACLFlow'
18 changes: 15 additions & 3 deletions core/src/model/Ad4mModel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1671,8 +1671,12 @@ WHERE ${whereConditions.join(' AND ')}

// Only process if target has a value
if (target !== undefined && target !== null && target !== '') {
// Check if we need to resolve a non-literal language expression
if (propMeta.resolveLanguage != undefined && propMeta.resolveLanguage !== 'literal' && typeof target === 'string') {
// Check if we need to resolve a non-literal language expression.
// resolveLanguage must be defined and not 'literal' to trigger expression resolution.
// Also skip if the target itself is a literal:// URI — those are handled by the
// literal-parsing branch below (avoids calling getExpression on empty literals like
// "literal://string:" which would cause a deserialization error).
if (propMeta.resolveLanguage != undefined && propMeta.resolveLanguage !== 'literal' && typeof target === 'string' && !target.startsWith('literal://')) {
// For non-literal languages, resolve the expression via perspective.getExpression()
// Note: Literals are already parsed by SurrealDB's fn::parse_literal()
try {
Expand Down Expand Up @@ -2236,6 +2240,11 @@ WHERE ${whereConditions.join(' AND ')}
// Get resolve language from metadata (replaces Prolog query)
let resolveLanguage = metadata.resolveLanguage;

// Skip storing empty/null/undefined values to avoid invalid empty literals (e.g. literal://string:)
if (value === undefined || value === null || value === "") {
return;
}

if (resolveLanguage) {
value = await this.#perspective.createExpression(value, resolveLanguage);
}
Expand Down Expand Up @@ -2355,9 +2364,12 @@ WHERE ${whereConditions.join(' AND ')}
}
}

// Get the class name instead of passing the instance to avoid Prolog query generation
const className = await this.perspective.stringOrTemplateObjectToSubjectClassName(this);

// Create the subject with the initial values
await this.perspective.createSubject(
this,
className,
this.#baseExpression,
initialValues,
batchId
Expand Down
Loading