-
Notifications
You must be signed in to change notification settings - Fork 596
Description
🧭 Type of Feature
- New feature or capability
🧭 Epic
Title: Tool Call Anomaly Detection Plugin
Goal: Detect unusual tool-calling patterns per user/agent by learning behavioral baselines.
Why now: ContextForge has strong permission-based controls (RBAC, PDP) and content-based security (PII filter, secrets detection), but no way to flag when a user's tool-calling behavior changes — e.g. sudden access to new tools, burst patterns, or off-hours activity. This would complement existing security and feed data into the planned Security Posture Dashboard (#2309).
🧑🏻💻 User Story 1
As a: platform administrator
I want: automatic detection when a user/agent's tool-calling pattern deviates from their baseline
So that: I'm alerted to potentially compromised credentials or prompt injection — even when RBAC permits the access.
✅ Acceptance Criteria
Scenario: Known tool call scores low risk
Given user "alice" regularly calls "db_query"
When "alice" calls "db_query" with typical arguments
Then the plugin allows the call with risk_score < 0.5
Scenario: Novel tool triggers elevated risk
Given user "alice" has never called "delete_records"
When "alice" calls "delete_records"
Then risk_score is elevated and a warning is logged
Scenario: Burst pattern detected
Given burst_threshold is 20 calls per 60 seconds
When a user makes 25 calls in 30 seconds
Then burst score exceeds threshold📐 Design Sketch
flowchart TD
A[Tool Call] --> B{Learning phase?}
B -->|Yes| C[Record baseline, allow]
B -->|No| D[Score: novelty + burst + frequency + off-hours]
D --> E{Above threshold?}
E -->|Block mode| F[Block + violation]
E -->|Warn mode| G[Allow + warn log]
E -->|Below| H[Allow + metadata]
- Hooks:
tool_pre_invoke,tool_post_invoke - Dependencies: None — pure Python, in-memory baselines
- Priority: Runs after RBAC/PDP checks
🔗 MCP Standards Check
- Change adheres to current MCP specifications
- No breaking changes to existing MCP-compliant integrations
🔄 Alternatives Considered
None at this stage.
📓 Additional Context
None.