You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
aws-aditya21
changed the title
feat(evaluator): Add kmsKeyArn support for custom evaluator
feat(evaluator): add kmsKeyArn support for custom evaluator
Apr 27, 2026
The kmsKeyArn field added here is persisted into agentcore.json but is never actually applied to the deployed evaluator, so the feature has no effect at runtime. Specifically:
CDK L3 construct does not read it.AgentCoreEvaluator in aws/agentcore-l3-cdk-constructs (src/cdk/constructs/components/primitives/evaluator/AgentCoreEvaluator.ts) never references evaluator.kmsKeyArn, and the EvaluatorSchema in that repo does not include a kmsKeyArn field. When deploy synthesizes CFN, the value is silently dropped. There is also no companion PR in agentcore-l3-cdk-constructs to add it.
CloudFormation resource does not support it.AWS::BedrockAgentCore::Evaluator only accepts Description, EvaluatorConfig, EvaluatorName, Level, and Tags (confirmed via CFN docs and aws-cdk-lib's generated CfnEvaluator type — which exposes kmsKeyArn only on the Gateway resource, not Evaluator). Even if the construct were updated to forward it, CFN would reject the template.
Service API does not support it either.CreateEvaluatorRequest and GetEvaluatorResponse in @aws-sdk/client-bedrock-agentcore-control (latest) do not contain a kmsKeyArn field. That's why the new getEvaluator code has to cast with (response as unknown as Record<string, unknown>).kmsKeyArn — there is no typed field to read, and in practice the service will never return one. The new toEvaluatorSpec test passes only because the test manually constructs a GetEvaluatorResult with kmsKeyArn set; round-tripping via the actual SDK will always yield undefined.
Net effect: a user who runs agentcore evaluator add --kms-key-arn … (or enters a KMS ARN in the TUI wizard) will see the ARN written to agentcore.json, but the deployed evaluator will not be encrypted with that key. That's worse than not having the option, because it silently implies encryption is configured.
Options to resolve:
Hold this PR until the backend supports it. Ship kmsKeyArn on the evaluator once the service API, the CFN resource, and the L3 construct all accept it. At that point, update this PR together with a matching change in agentcore-l3-cdk-constructs so the field is actually forwarded to CfnEvaluator/CfnResource props.
Drop the evaluator kmsKeyArn entirely for now. If the intent was to mirror what already exists for the identity token vault (identityKmsKeyArn), note that those are separate code paths and don't feed evaluator encryption.
If there is a documented non-CFN path (e.g., a post-deploy SDK call that applies KMS to an evaluator), wire it up in the deploy pipeline — right now nothing reads evaluator.kmsKeyArn from the spec at deploy time (verified with grep -rn 'evaluator.*kmsKeyArn' src/ — only hit is the wizard's own memo).
Happy to re-review once the plumbing is end-to-end.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add optional
kmsKeyArnfield to the evaluator primitive to support customer-managed KMS encryption.Changes (9 files):
src/schema/schemas/agentcore-project.ts— addedkmsKeyArntoEvaluatorSchemasrc/cli/primitives/EvaluatorPrimitive.ts— added--kms-key-arnCLI flag, updated options, action handler, andcreateEvaluatorsrc/cli/tui/screens/evaluator/types.ts— addedkms-key-arnstep andkmsKeyArntoAddEvaluatorConfigsrc/cli/tui/screens/evaluator/useAddEvaluatorWizard.ts— added KMS state, callback, wired into all wizard flowssrc/cli/tui/screens/evaluator/AddEvaluatorScreen.tsx— added KMS key ARN text input and confirm fieldsrc/cli/tui/hooks/useCreateEvaluator.ts— passeskmsKeyArnthrough to primitivesrc/cli/commands/import/import-evaluator.ts—toEvaluatorSpecforwardskmsKeyArnfrom API responsesrc/cli/aws/agentcore-control.ts— addedkmsKeyArntoGetEvaluatorResultsrc/cli/commands/import/__tests__/import-evaluator.test.ts— added tests for kmsKeyArn forwardingType of Change
Testing
How have you tested the change?
Testing
npm run test:unitandnpm run test:integnpm run typechecknpm run lintsrc/assets/, I rannpm run test:update-snapshotsand committed the updated snapshotsChecklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.