Skip to content

[FEATURE]: Tool call anomaly detection plugin #3845

@anujshrivastava15

Description

@anujshrivastava15

🧭 Type of Feature

  • New feature or capability

🧭 Epic

Title: Tool Call Anomaly Detection Plugin
Goal: Detect unusual tool-calling patterns per user/agent by learning behavioral baselines.
Why now: ContextForge has strong permission-based controls (RBAC, PDP) and content-based security (PII filter, secrets detection), but no way to flag when a user's tool-calling behavior changes — e.g. sudden access to new tools, burst patterns, or off-hours activity. This would complement existing security and feed data into the planned Security Posture Dashboard (#2309).


🧑🏻‍💻 User Story 1

As a: platform administrator
I want: automatic detection when a user/agent's tool-calling pattern deviates from their baseline
So that: I'm alerted to potentially compromised credentials or prompt injection — even when RBAC permits the access.

✅ Acceptance Criteria

Scenario: Known tool call scores low risk
  Given user "alice" regularly calls "db_query"
  When "alice" calls "db_query" with typical arguments
  Then the plugin allows the call with risk_score < 0.5

Scenario: Novel tool triggers elevated risk
  Given user "alice" has never called "delete_records"
  When "alice" calls "delete_records"
  Then risk_score is elevated and a warning is logged

Scenario: Burst pattern detected
  Given burst_threshold is 20 calls per 60 seconds
  When a user makes 25 calls in 30 seconds
  Then burst score exceeds threshold

📐 Design Sketch

flowchart TD
    A[Tool Call] --> B{Learning phase?}
    B -->|Yes| C[Record baseline, allow]
    B -->|No| D[Score: novelty + burst + frequency + off-hours]
    D --> E{Above threshold?}
    E -->|Block mode| F[Block + violation]
    E -->|Warn mode| G[Allow + warn log]
    E -->|Below| H[Allow + metadata]
Loading
  • Hooks: tool_pre_invoke, tool_post_invoke
  • Dependencies: None — pure Python, in-memory baselines
  • Priority: Runs after RBAC/PDP checks

🔗 MCP Standards Check

  • Change adheres to current MCP specifications
  • No breaking changes to existing MCP-compliant integrations

🔄 Alternatives Considered

None at this stage.


📓 Additional Context

None.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions