Skip to content

Commit 8fb33f6

Browse files
dp-web4claude
andcommitted
feat(web4-governance): add governance plugin with R6 workflow (Tiers 1-4)
Web4 governance plugin for Claude Code hooks — structured audit trails, trust tensors, entity witnessing, policy gating, and event streaming. Tiers: observational audit (T1), policy presets and rate limiting (T1.5), signing and persistent witnesses (T2), multi-target extraction (T3), event stream monitoring (T4). See plugins/web4-governance/README.md for full documentation. PR: #20448 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 85f2807 commit 8fb33f6

47 files changed

Lines changed: 13237 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,8 @@
11
.DS_Store
22

3+
*.ffs_db
4+
.gitnexus
5+
.claude/skills/gitnexus/
6+
CLAUDE.md
7+
AGENTS.md
8+
__pycache__/
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"name": "web4-governance",
3+
"version": "1.0.0",
4+
"description": "Lightweight AI governance with R6 workflow formalism and audit trails. Every tool call becomes a structured request with verifiable provenance.",
5+
"author": {
6+
"name": "Web4 Contributors",
7+
"email": "web4@metalinxx.io"
8+
},
9+
"repository": "https://github.com/dp-web4/web4",
10+
"keywords": ["governance", "audit", "trust", "r6", "provenance", "web4"],
11+
"license": "MIT"
12+
}
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Force LF line endings for scripts (critical for shebang on Unix/WSL)
2+
*.py text eol=lf
3+
*.sh text eol=lf
4+
5+
# Other text files can use native line endings
6+
*.md text
7+
*.json text
8+
*.txt text
Lines changed: 325 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,325 @@
1+
# Event Stream API
2+
3+
Real-time monitoring endpoint for the Web4 Governance plugin.
4+
5+
## Overview
6+
7+
The event stream provides a JSONL (JSON Lines) file that external clients can tail for real-time monitoring, alerting, and analytics.
8+
9+
**Stream Location**: `~/.web4/events.jsonl`
10+
11+
## Quick Start
12+
13+
### Tail the stream (real-time)
14+
```bash
15+
tail -f ~/.web4/events.jsonl | jq .
16+
```
17+
18+
### Filter by severity
19+
```bash
20+
tail -f ~/.web4/events.jsonl | jq -c 'select(.severity == "alert")'
21+
```
22+
23+
### Filter by event type
24+
```bash
25+
grep '"type":"policy_decision"' ~/.web4/events.jsonl | jq .
26+
```
27+
28+
### Python consumer
29+
```python
30+
import json
31+
32+
with open("~/.web4/events.jsonl", "r") as f:
33+
for line in f:
34+
event = json.loads(line)
35+
if event.get("severity") == "alert":
36+
print(f"ALERT: {event.get('reason')}")
37+
```
38+
39+
---
40+
41+
## Event Schema
42+
43+
Each line in the stream is a self-contained JSON object:
44+
45+
```json
46+
{
47+
"type": "policy_decision",
48+
"timestamp": "2026-02-05T10:30:00.123456+00:00",
49+
"severity": "alert",
50+
"session_id": "sess-abc123",
51+
"tool": "Bash",
52+
"target": "rm -rf /tmp/test",
53+
"category": "command",
54+
"decision": "deny",
55+
"reason": "Destructive command blocked by safety preset",
56+
"rule_id": "deny-destructive-commands"
57+
}
58+
```
59+
60+
### Required Fields
61+
62+
| Field | Type | Description |
63+
|-------|------|-------------|
64+
| `type` | string | Event type (see Event Types below) |
65+
| `timestamp` | string | ISO 8601 UTC timestamp |
66+
| `severity` | string | Severity level: `debug`, `info`, `warn`, `alert`, `error` |
67+
68+
### Optional Context Fields
69+
70+
| Field | Type | Description |
71+
|-------|------|-------------|
72+
| `session_id` | string | Session identifier |
73+
| `agent_id` | string | Agent/role identifier |
74+
75+
### Event-Specific Fields
76+
77+
| Field | Type | Description |
78+
|-------|------|-------------|
79+
| `tool` | string | Tool name (Bash, Read, Edit, etc.) |
80+
| `target` | string | Target path, URL, or command |
81+
| `category` | string | Tool category (file_read, file_write, command, network, etc.) |
82+
| `decision` | string | Policy decision: `allow`, `deny`, `warn` |
83+
| `reason` | string | Human-readable explanation |
84+
| `rule_id` | string | ID of matched policy rule |
85+
86+
### Metrics Fields
87+
88+
| Field | Type | Description |
89+
|-------|------|-------------|
90+
| `duration_ms` | integer | Operation duration in milliseconds |
91+
| `count` | integer | Generic count (rate limits, etc.) |
92+
93+
### Trust Fields
94+
95+
| Field | Type | Description |
96+
|-------|------|-------------|
97+
| `trust_before` | float | Trust value before update (0.0-1.0) |
98+
| `trust_after` | float | Trust value after update (0.0-1.0) |
99+
| `trust_delta` | float | Change in trust value |
100+
101+
### Error Fields
102+
103+
| Field | Type | Description |
104+
|-------|------|-------------|
105+
| `error` | string | Error message |
106+
| `error_type` | string | Error class/type |
107+
108+
### Extensible Metadata
109+
110+
| Field | Type | Description |
111+
|-------|------|-------------|
112+
| `metadata` | object | Additional key-value data |
113+
114+
---
115+
116+
## Event Types
117+
118+
### Session Lifecycle
119+
120+
| Type | Severity | Description |
121+
|------|----------|-------------|
122+
| `session_start` | info | New session started |
123+
| `session_end` | info | Session ended |
124+
125+
**Example:**
126+
```json
127+
{"type":"session_start","timestamp":"2026-02-05T10:00:00Z","severity":"info","session_id":"sess-abc123","metadata":{"project":"my-app","atp_budget":100}}
128+
```
129+
130+
### Tool Execution
131+
132+
| Type | Severity | Description |
133+
|------|----------|-------------|
134+
| `tool_call` | info | Tool invocation started |
135+
| `tool_result` | info | Tool completed |
136+
137+
**Example:**
138+
```json
139+
{"type":"tool_call","timestamp":"2026-02-05T10:01:00Z","severity":"info","session_id":"sess-abc123","tool":"Read","target":"/app/src/main.py","category":"file_read"}
140+
```
141+
142+
### Policy Decisions
143+
144+
| Type | Severity | Description |
145+
|------|----------|-------------|
146+
| `policy_decision` | varies | Policy evaluated (info=allow, warn=warn, alert=deny) |
147+
| `policy_violation` | alert | Policy rule violated |
148+
149+
**Example (deny):**
150+
```json
151+
{"type":"policy_decision","timestamp":"2026-02-05T10:02:00Z","severity":"alert","session_id":"sess-abc123","tool":"Bash","target":"rm -rf /","decision":"deny","reason":"Destructive command blocked by safety preset","rule_id":"deny-destructive-commands"}
152+
```
153+
154+
**Example (warn):**
155+
```json
156+
{"type":"policy_decision","timestamp":"2026-02-05T10:03:00Z","severity":"warn","session_id":"sess-abc123","tool":"Bash","target":"rm temp.txt","decision":"warn","reason":"File deletion flagged - use with caution","rule_id":"warn-file-delete"}
157+
```
158+
159+
### Rate Limiting
160+
161+
| Type | Severity | Description |
162+
|------|----------|-------------|
163+
| `rate_limit_check` | debug | Rate limit checked |
164+
| `rate_limit_exceeded` | alert | Rate limit exceeded |
165+
166+
**Example:**
167+
```json
168+
{"type":"rate_limit_exceeded","timestamp":"2026-02-05T10:04:00Z","severity":"alert","session_id":"sess-abc123","target":"ratelimit:bash:tool","count":6,"metadata":{"max_count":5}}
169+
```
170+
171+
### Trust Updates
172+
173+
| Type | Severity | Description |
174+
|------|----------|-------------|
175+
| `trust_update` | info | Agent trust level changed |
176+
177+
**Example:**
178+
```json
179+
{"type":"trust_update","timestamp":"2026-02-05T10:05:00Z","severity":"info","session_id":"sess-abc123","agent_id":"code-reviewer","trust_before":0.5,"trust_after":0.55,"trust_delta":0.05,"reason":"Successful code review"}
180+
```
181+
182+
### Agent Lifecycle
183+
184+
| Type | Severity | Description |
185+
|------|----------|-------------|
186+
| `agent_spawn` | info | Agent spawned |
187+
| `agent_complete` | info | Agent completed |
188+
189+
**Example:**
190+
```json
191+
{"type":"agent_spawn","timestamp":"2026-02-05T10:06:00Z","severity":"info","session_id":"sess-abc123","agent_id":"test-runner","metadata":{"capabilities":{"can_write":true,"can_execute":true}}}
192+
```
193+
194+
### Audit Events
195+
196+
| Type | Severity | Description |
197+
|------|----------|-------------|
198+
| `audit_record` | info | Standard audit record |
199+
| `audit_alert` | alert | High-priority audit event (credential access, etc.) |
200+
201+
**Example (credential access alert):**
202+
```json
203+
{"type":"audit_alert","timestamp":"2026-02-05T10:07:00Z","severity":"alert","session_id":"sess-abc123","tool":"Read","target":"/home/user/.aws/credentials","category":"credential_access","reason":"Credential file access detected"}
204+
```
205+
206+
### System Events
207+
208+
| Type | Severity | Description |
209+
|------|----------|-------------|
210+
| `system_info` | info | System information |
211+
| `system_error` | error | System error |
212+
213+
**Example:**
214+
```json
215+
{"type":"system_error","timestamp":"2026-02-05T10:08:00Z","severity":"error","error":"Database connection failed","error_type":"sqlite3.OperationalError"}
216+
```
217+
218+
---
219+
220+
## Severity Levels
221+
222+
| Level | When Used | Action |
223+
|-------|-----------|--------|
224+
| `debug` | Verbose debugging | Usually filtered |
225+
| `info` | Normal operations | Log/monitor |
226+
| `warn` | Potential issues | Review |
227+
| `alert` | Security events, policy violations | Immediate attention |
228+
| `error` | System errors | Investigate |
229+
230+
---
231+
232+
## File Rotation
233+
234+
The stream file automatically rotates at 100MB:
235+
- Current: `~/.web4/events.jsonl`
236+
- Rotated: `~/.web4/events.jsonl.1`
237+
238+
Only one backup is kept. For long-term retention, configure an external log collector.
239+
240+
---
241+
242+
## Integration Examples
243+
244+
### Forward to syslog
245+
```bash
246+
tail -f ~/.web4/events.jsonl | while read line; do
247+
logger -t web4-governance "$line"
248+
done
249+
```
250+
251+
### Send alerts to Slack
252+
```python
253+
import json
254+
import requests
255+
256+
WEBHOOK_URL = "https://hooks.slack.com/services/..."
257+
258+
with open("~/.web4/events.jsonl", "r") as f:
259+
f.seek(0, 2) # Seek to end
260+
while True:
261+
line = f.readline()
262+
if line:
263+
event = json.loads(line)
264+
if event.get("severity") == "alert":
265+
requests.post(WEBHOOK_URL, json={
266+
"text": f":warning: {event.get('type')}: {event.get('reason')}"
267+
})
268+
```
269+
270+
### Prometheus metrics (conceptual)
271+
```python
272+
from prometheus_client import Counter
273+
274+
policy_decisions = Counter('web4_policy_decisions', 'Policy decisions', ['decision'])
275+
276+
# In your event consumer:
277+
if event["type"] == "policy_decision":
278+
policy_decisions.labels(decision=event["decision"]).inc()
279+
```
280+
281+
### Structured logging (JSON to stdout)
282+
```bash
283+
tail -f ~/.web4/events.jsonl | jq -c '{
284+
time: .timestamp,
285+
level: .severity,
286+
msg: .reason // .type,
287+
tool: .tool,
288+
session: .session_id
289+
}'
290+
```
291+
292+
---
293+
294+
## Configuration
295+
296+
The event stream can be configured when initializing:
297+
298+
```python
299+
from governance.event_stream import EventStream, Severity
300+
301+
# Custom location and minimum severity
302+
stream = EventStream(
303+
storage_path="~/.my-app/governance",
304+
filename="audit-events.jsonl",
305+
min_severity=Severity.WARN # Only emit WARN and above
306+
)
307+
```
308+
309+
---
310+
311+
## Best Practices
312+
313+
1. **Use `tail -f`** for real-time monitoring rather than polling
314+
2. **Filter by severity** to reduce noise (`alert` for critical events)
315+
3. **Use `jq`** for ad-hoc queries and formatting
316+
4. **Configure external log rotation** for long-term retention
317+
5. **Register callbacks** for in-process alerting (low latency)
318+
319+
---
320+
321+
## Version History
322+
323+
| Version | Changes |
324+
|---------|---------|
325+
| 0.4.0 | Initial event stream API |

0 commit comments

Comments
 (0)