Skip to content

Commit 6b05e3b

Browse files
committed
docs: address limitaions
1 parent d6a649c commit 6b05e3b

File tree

4 files changed

+355
-12
lines changed

4 files changed

+355
-12
lines changed

docs/docs/Infrastructure/Web3-Adapter.md

Lines changed: 6 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -45,9 +45,9 @@ The adapter does not poll the database. The platform must detect changes (e.g. v
4545

4646
After data is stored or updated in an eVault, the [Awareness Protocol](/docs/W3DS%20Protocol/Awareness-Protocol) delivers webhooks to all other registered platforms. Those platforms use the same adapter's `fromGlobal` and ID mapping to apply the change locally. So the full loop is: Platform A → adapter → eVault → Awareness Protocol → Platform B's webhook → adapter → Platform B's DB.
4747

48-
### Design implications
48+
### Design Limitations
4949

50-
You can get better results by:
50+
Current implementation has the following known limitations, which we aim to fix with subsequent versions:
5151

5252
- **Ontology design**: Clear schema versioning, optional vs required fields, and conventions for references (e.g. eNames vs local IDs in payloads).
5353
- **Mapping expressiveness**: Richer `ownerEnamePath` (e.g. fallbacks), relation resolution, and handling of arrays and nested structures.
@@ -112,7 +112,7 @@ graph TB
112112

113113
### Mapping configuration (IMapping)
114114

115-
Mapping configs define how local fields map to the global ontology. For the full syntax (direct fields, relations, arrays, `__date`, `__calc`, owner path), see the **Web3 Adapter Mapping Rules** in the repository at `infrastructure/web3-adapter/MAPPING_RULES.md`.
115+
Mapping configs define how local fields map to the global ontology. For the full syntax (direct fields, relations, arrays, `__date`, `__calc`, owner path), see the [Web3 Adapter Mapping Rules](../../../infrastructure/web3-adapter/MAPPING_RULES.md).
116116

117117
Each mapping is a JSON file with:
118118

@@ -122,8 +122,6 @@ Each mapping is a JSON file with:
122122
- **localToUniversalMap**: Object mapping local field names to global field names or expressions (e.g. `"createdAt": "__date(createdAt)"`, relation syntax `"tableName(path),globalAlias"`).
123123
- **readOnly** (optional): If true, `handleChange` does not sync this table to the eVault.
124124

125-
For full syntax (direct, relation, array, `__date`, `__calc`, owner path), see the mapping rules in the repository at `infrastructure/web3-adapter/MAPPING_RULES.md`.
126-
127125
### Receiving data (inbound)
128126

129127
When a platform receives an awareness protocol packet at `POST /api/webhook`:
@@ -134,7 +132,7 @@ When a platform receives an awareness protocol packet at `POST /api/webhook`:
134132
4. Call `mappingDb.getLocalId(body.id)`; if found, update the existing local entity; otherwise create and then `mappingDb.storeMapping({ localId: newEntity.id, globalId: body.id })`.
135133
5. Return 200.
136134

137-
See the [Webhook Controller Guide](/docs/Post%20Platform%20Guide/webhook-controller) for a full implementation example.
135+
See the [Webhook Controller Guide](/docs/Post%20Platform%20Guide/webhook-controller) for a full implementation example and the [Awareness Protocol](/docs/W3DS%20Protocol/Awareness-Protocol) for the packet format and delivery mechanism.
138136

139137
## Sequence: Platform → Adapter → eVault
140138

@@ -165,10 +163,9 @@ sequenceDiagram
165163
end
166164
```
167165

168-
## Extension points
166+
## Limitations & Planned Extensions
169167

170-
- **Ontology versioning**: Today `schemaId` is a single UUID; you could version schemas and map multiple local tables to different versions.
168+
- **Ontology versioning**: Today `schemaId` is a single UUID, there are plans for adding a `schemaVersion` key to allow for schema versioning.
171169
- **Conflict resolution**: Add version or timestamp fields and resolve conflicts in the adapter or in a separate service.
172170
- **Idempotency**: Use idempotency keys in payloads or in the mapping DB to make webhook handling and outbound sync idempotent.
173-
- **ID mapping storage**: Replace SQLite with a shared store if multiple instances of the platform need the same mappings.
174171
- **Transactional outbox**: Have the platform write changes to an outbox table and have a worker call `handleChange` so that sync is tied to the same transaction as the local write.

docs/docs/W3DS Protocol/Awareness-Protocol.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ sidebar_position: 4
55
# Awareness Protocol
66

77
:::warning Prototype-level implementation
8-
The Awareness Protocol described here is **prototype-level**. The packet format, delivery behavior, and platform contract are subject to change. Do not rely on them for production without checking for updates.
8+
The Awareness Protocol described here is **prototype-level**. The packet format, delivery behavior, and platform contract are subject to change in upcoming updates
99
:::
1010

1111
The Awareness Protocol is the webhook deliverability mechanism in W3DS. When data in an eVault changes (create or update), the eVault notifies all other registered platforms so they can stay in sync. "Awareness" means platforms become aware of changes that happened elsewhere.
@@ -101,11 +101,11 @@ For a step-by-step implementation guide, see the [Webhook Controller Guide](/doc
101101

102102
## Limitations and Extension Points
103103

104-
The current protocol has intentional limitations that readers may improve in their own implementations:
104+
The current protocol has limitations which are going to be improved in subsequent versions:
105105

106106
- **No retries**: Failed webhooks are not retried. A more robust system could add retries with backoff or a dead-letter queue.
107107
- **No ordering guarantee**: Webhooks to different platforms are sent in parallel; there is no guarantee of order across platforms or across multiple mutations.
108108
- **No at-least-once guarantee**: Because delivery is fire-and-forget and there are no retries, a platform might never receive a given update. At-least-once delivery would require acknowledgments and retries (and possibly idempotency keys).
109-
- **Platform list from Registry**: The set of recipients is whatever the Registry returns for `GET /platforms`. How that list is maintained and updated is registry-specific.
109+
- **Platform list from Registry**: The set of recipients is whatever the Registry returns for `GET /platforms`. This is a prototype level shortcut and will be phased out.
110110

111111
Designing retries, ordering, or delivery guarantees would be natural extension points for a production-grade awareness mechanism.
Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
# eSigner API – Implementation Details (Workflow / Diagrams)
2+
3+
## Overview
4+
5+
eSigner API is an Express (Node.js) backend for document signature containers. Users upload files, invite signees, and sign via QR/deep link with an eID Wallet. Signatures are verified against the W3DS registry and stored with the file.
6+
7+
**Stack:** Express, TypeORM (Postgres), JWT auth, SSE for session status, Web3Adapter for sync with registry/file-manager.
8+
9+
---
10+
11+
## High-Level Architecture
12+
13+
```
14+
Client (eSigner Svelte app)
15+
|
16+
| JWT (Bearer)
17+
v
18+
Express API (authMiddleware → authGuard on protected routes)
19+
|
20+
+-- AuthController (public: offer, login, SSE session)
21+
+-- FileController (CRUD files, list, download, signatures)
22+
+-- InvitationController (invite, list, decline)
23+
+-- SignatureController (create session, SSE status, callback from wallet)
24+
+-- UserController (current user, search)
25+
+-- WebhookController (public: POST /api/webhook – registry sync)
26+
|
27+
v
28+
Services: FileService, InvitationService, SignatureService, UserService,
29+
NotificationService, PlatformEVaultService, MessageService, GroupService
30+
|
31+
v
32+
Database (Postgres): files, file_signees, signature_containers, users, groups, messages, user_evault_mappings
33+
```
34+
35+
---
36+
37+
## Auth Flow
38+
39+
- **Public:** `GET /api/auth/offer` → returns `w3ds://auth?redirect=...&session=<uuid>&platform=esigner`.
40+
- **Login:** Client opens wallet with that URI; wallet signs and POSTs to `POST /api/auth` with `{ ename, session, appVersion, signature }`. API verifies signature (signature-validator), finds/creates user, signs JWT, emits result on SSE for `session`; client polls `GET /api/auth/sessions/:id` (SSE) to get token.
41+
- **Protected routes:** After `authMiddleware` (optional: parses Bearer, sets `req.user`), `authGuard` requires `req.user`; 401 if missing.
42+
43+
---
44+
45+
## Main Entities (ER for Diagrams)
46+
47+
| Entity | Purpose |
48+
|--------|--------|
49+
| **File** | Document blob. `id`, `name`, `displayName`, `description`, `mimeType`, `size`, `md5Hash`, `data` (bytea), `ownerId`. Soft-delete: `name === "[[deleted]]"` hidden in eSigner. |
50+
| **FileSignee** | Invitation to sign a file. `fileId`, `userId`, `invitedBy`, `status` (pending \| signed \| declined), `invitedAt`, `signedAt`, `declinedAt`. Owner is auto-added as signee when inviting. |
51+
| **SignatureContainer** | One signature on a file. `fileId`, `userId`, `fileSigneeId`, `md5Hash`, `signature`, `publicKey`, `message`. Links to File and FileSignee. |
52+
53+
Relations: `File` 1─N `FileSignee`, `File` 1─N `SignatureContainer`, `FileSignee` 1─1 `SignatureContainer` (optional).
54+
55+
---
56+
57+
## File Workflows
58+
59+
### 1. Upload and list
60+
61+
- **POST /api/files** (multipart): `authGuard``FileController.uploadFile``FileService.createFile`. Validates name ≠ `[[deleted]]`, computes MD5, stores file. Optional `displayName`, `description`.
62+
- **GET /api/files**: `FileService.getDocumentsWithStatus(userId, listMode)`. `getUserFiles(userId)` = owned files (createdAt DESC) + files where user is in `file_signees`; merged list sorted by `createdAt` DESC. If `listMode === 'containers'`, only files with at least one signee. Response includes derived status: draft \| pending \| partially_signed \| fully_signed, plus signee/signature counts.
63+
- **GET /api/files/:id**, **PATCH**, **DELETE**, **GET :id/download**, **GET :id/signatures**: all gated by owner or presence in `file_signees`.
64+
65+
### 2. Invite signees
66+
67+
- **POST /api/files/:fileId/invite** body `{ userIds: string[] }`. `InvitationService.inviteSignees`: file must exist and caller is owner; file must have zero signatures (single-use); owner is always added as signee if not already; then each `userId` gets a `FileSignee` (status pending). Notifications sent (fire-and-forget). Returns created invitations.
68+
69+
### 3. Get invitations
70+
71+
- **GET /api/files/:fileId/invitations**: list invitations for that file (caller must be owner or signee).
72+
- **GET /api/invitations**: list invitations for current user (files they’re invited to).
73+
- **POST /api/invitations/:id/decline**: set that invitation to declined.
74+
75+
---
76+
77+
## Signing Session Flow (Core for Diagrams)
78+
79+
1. **Create session (user has pending invitation)**
80+
**POST /api/signatures/session** body `{ fileId }`.
81+
- `SignatureService.createSession(fileId, userId)`:
82+
- Ensures user has a pending `FileSignee` for that file.
83+
- Loads file, gets `md5Hash`.
84+
- `sessionId = userId_md5` (so signed payload binds user + file hash).
85+
- Builds message `{ message, md5Hash, sessionId }`, base64, then `qrData = w3ds://sign?session=...&data=...&redirect_uri=.../api/signatures/callback`.
86+
- In-memory session store: `sessions.set(sessionId, { sessionId, fileId, userId, md5Hash, qrData, expiresAt, status: "pending" })`. Expires in 15 minutes (status → "expired").
87+
- Response: `{ sessionId, qrData, expiresAt }`.
88+
89+
2. **Poll session status (SSE)**
90+
**GET /api/signatures/session/:id** (no auth).
91+
- Streams SSE. Sends initial status then polls every 1s. Events: `type: "signed", status: "completed"` \| `type: "expired"` \| `type: "security_violation"` \| `type: "error"`. Cleans up on client disconnect.
92+
93+
3. **Wallet callback (signature submission)**
94+
**POST /api/signatures/callback** (no auth) body `{ sessionId, signature, w3id, message }`.
95+
- `SignatureService.processSignedPayload`:
96+
- Load session by `sessionId`; must be "pending".
97+
- Validates payload structure; ename/signature verified via `verifySignature` (signature-validator + registry). Ensures signed message equals `userId_md5`, and `publicKey` matches user ename.
98+
- Creates `SignatureContainer` (fileId, userId, fileSigneeId, md5Hash, signature, publicKey, message).
99+
- Updates `FileSignee` to status "signed", sets `signedAt`.
100+
- Sets in-memory session status to "completed".
101+
- Returns 200 with success/error; SSE clients see "signed" and close.
102+
103+
---
104+
105+
## Status Derivation (List View)
106+
107+
For each file in "documents with status":
108+
109+
- No signees → **draft**
110+
- Signees exist, no one signed yet → **pending**
111+
- Some signed, not all → **partially_signed**
112+
- All signees signed → **fully_signed**
113+
114+
Computed from `File.signees` (counts by `status === 'signed'` / `'pending'` / `'declined'`).
115+
116+
---
117+
118+
## Webhook and Sync
119+
120+
- **POST /api/webhook**: Receives events from registry/file-manager (Web3Adapter). `schemaId` + `id` + `data`; adapter mapping picks entity (users, groups, messages, files, signature_containers). Actions: create/update/delete local rows and store globalId↔localId mapping. Used to keep eSigner in sync with external systems (e.g. file renames/deletes from file-manager; `[[deleted]]` files are hidden in eSigner).
121+
122+
---
123+
124+
## External Dependencies
125+
126+
- **Registry:** `PUBLIC_REGISTRY_URL` – used by signature-validator for ename/signature verification.
127+
- **Platform eVault:** On startup, `PlatformEVaultService` ensures a platform eVault exists for eSigner.
128+
- **Optional:** `ANCHR_URL` – webhook payloads are forwarded to `ANCHR_URL/esigner` for analytics/monitoring.
129+
130+
---
131+
132+
## Key Files for Diagrams
133+
134+
| Concern | File(s) |
135+
|--------|--------|
136+
| Routes | `src/index.ts` |
137+
| Auth | `src/controllers/AuthController.ts`, `src/middleware/auth.ts` |
138+
| Files | `src/controllers/FileController.ts`, `src/services/FileService.ts` |
139+
| Invitations | `src/controllers/InvitationController.ts`, `src/services/InvitationService.ts` |
140+
| Signing flow | `src/controllers/SignatureController.ts`, `src/services/SignatureService.ts` |
141+
| Entities | `src/database/entities/File.ts`, `FileSignee.ts`, `SignatureContainer.ts` |
142+
| Webhook/sync | `src/controllers/WebhookController.ts`, `src/web3adapter/watchers/subscriber.ts` |
143+
144+
---
145+
146+
## Mermaid Snippet Ideas
147+
148+
- **Sequence: Create reference → Invite → Sign**
149+
- Actor: Owner; Boundary: API; Participants: FileService, InvitationService, SignatureService; Wallet as external (callback).
150+
- **State: FileSignee**
151+
- pending → signed | declined.
152+
- **State: Signing session (in-memory)**
153+
- pending → completed | expired | security_violation.
154+
- **Flow: GET /api/files**
155+
- authGuard → getDocumentsWithStatus → getUserFiles (owned + invited) → sort by createdAt → filter containers vs all → map to status.
156+
157+
Use this doc to drive workflow diagrams (sequences, state machines, data flow) for docs or onboarding.

0 commit comments

Comments
 (0)