Skip to content

Feature: AWS bedrock generator#1462

Merged
jmartin-tech merged 13 commits intoNVIDIA:mainfrom
fabriziorocco:feature/bedrock_generator
Nov 26, 2025
Merged

Feature: AWS bedrock generator#1462
jmartin-tech merged 13 commits intoNVIDIA:mainfrom
fabriziorocco:feature/bedrock_generator

Conversation

@fabriziorocco
Copy link
Contributor

@fabriziorocco fabriziorocco commented Nov 6, 2025

Adding support for the Bedrock Generator. Adding this because customer are requesting to test LLM on AWS (focus on Amazon Nova models).
Bedrock use boto3 SDK and Converse API to make requests. Credentials are managed using AWS Credentials or new Bedrock API Key (recently introduced).

Verification

List the steps needed to make sure this thing works

  • python -m garak --model_type bedrock --model_name nova-pro

If you are opening a PR for a new plugin that targets a specific piece of hardware or requires a complex or hard-to-find testing environment, we recommend that you send us as much detail as possible.

Complex Software Examples:

No specific hardware requirements

Signed-off-by: Fabrizio Rocco <34063386+fabriziorocco@users.noreply.github.com>
Signed-off-by: Fabrizio Rocco <34063386+fabriziorocco@users.noreply.github.com>
Signed-off-by: Fabrizio Rocco <34063386+fabriziorocco@users.noreply.github.com>
@github-actions
Copy link
Contributor

github-actions bot commented Nov 6, 2025

DCO Assistant Lite bot All contributors have signed the DCO ✍️ ✅

@fabriziorocco
Copy link
Contributor Author

I have read the DCO Document and I hereby sign the DCO

@fabriziorocco
Copy link
Contributor Author

recheck

github-actions bot added a commit that referenced this pull request Nov 6, 2025
@leondz leondz added the generators Interfaces with LLMs label Nov 7, 2025
@fabriziorocco
Copy link
Contributor Author

Claude 3 sonnet is not supported so test is failing. I'm open to change the model in pytest

Signed-off-by: Fabrizio Rocco <34063386+fabriziorocco@users.noreply.github.com>
Signed-off-by: Fabrizio Rocco <34063386+fabriziorocco@users.noreply.github.com>
Signed-off-by: Fabrizio Rocco <34063386+fabriziorocco@users.noreply.github.com>
Signed-off-by: Fabrizio Rocco <34063386+fabriziorocco@users.noreply.github.com>
Signed-off-by: Fabrizio Rocco <34063386+fabriziorocco@users.noreply.github.com>
@fabriziorocco
Copy link
Contributor Author

Hi @leondz the test errors should be fixed now. I didn't include the .rst file

Copy link
Collaborator

@jmartin-tech jmartin-tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, this is a great addition.

Comments are based on static code review. Full integration testing is still in queue.

@fabriziorocco
Copy link
Contributor Author

added Value Error logic. It was deleted from previous commit

@leondz leondz self-requested a review November 20, 2025 15:40
Copy link
Collaborator

@jmartin-tech jmartin-tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor additional thoughts on UX and supported message roles. I suspect these may result in less confusion if this generator is utilized with complex configurations.

fabriziorocco and others added 3 commits November 23, 2025 19:23
Co-authored-by: Jeffrey Martin <jmartin@Op3n4M3.dev>
Signed-off-by: Fabrizio Rocco <34063386+fabriziorocco@users.noreply.github.com>
Co-authored-by: Jeffrey Martin <jmartin@Op3n4M3.dev>
Signed-off-by: Fabrizio Rocco <34063386+fabriziorocco@users.noreply.github.com>
@fabriziorocco
Copy link
Contributor Author

I tried to use the base class's _conversation_to_list method which formatted message content as a plain string, but AWS Bedrock's Converse API requires content to be a list of content blocks. The result was the test failing

Copy link
Collaborator

@jmartin-tech jmartin-tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks complete, thank you!

Comment on lines +169 to +172
turn_list = [
{"role": turn.role, "content": [{"text": turn.content.text}]}
for turn in conversation.turns
]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the misdirection, I see this is a slightly different format than the what is used in the base class.

@jmartin-tech jmartin-tech changed the title Feature/bedrock generator Feature: AWS bedrock generator Nov 26, 2025
@jmartin-tech jmartin-tech merged commit 813cbfd into NVIDIA:main Nov 26, 2025
15 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Nov 26, 2025
@leondz
Copy link
Collaborator

leondz commented Nov 27, 2025

Thank you Fabrizio!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

generators Interfaces with LLMs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants