-
Notifications
You must be signed in to change notification settings - Fork 257
Add Fine tuned model switch #82
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Summary of Changes
Hello @riggaroo, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request significantly enhances the application's AI capabilities by introducing a flexible system for dynamic selection between different generative AI models for image creation. It leverages remote configuration to allow switching between a general Imagen model and a specific fine-tuned model, providing adaptability without requiring application updates. Furthermore, it refines the default prompts used for AI interactions, ensuring more precise and controlled content generation.
Highlights
- AI Model Selection: Implemented a mechanism to dynamically switch between using the 'Imagen' model and a specified fine-tuned model for image generation, controlled via remote configuration.
- Remote Configuration Integration: Introduced new remote configuration keys (
use_imagenandfine_tuned_model_name) to enable runtime control over which AI model is utilized for image generation. - Enhanced Prompt Management: Updated and added detailed default prompts for image description and generation within the remote config defaults, incorporating strict guidelines for content generation and exclusion rules.
- Test Infrastructure Update: Extended the test remote config data source to properly support the newly introduced model selection parameters.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces a feature switch to select between the Imagen model and a fine-tuned model for image generation. The changes include adding new remote config flags, updating the data source to handle the model switching logic, and adding new prompts.
| } else { | ||
| val finedTunedModel = createFineTunedModel() | ||
| val response = finedTunedModel.generateContent(imageGenerationPrompt) | ||
| return response.candidates[0].content.parts[0].asImageOrNull()!! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This line is unsafe and can lead to runtime crashes for several reasons:
response.candidatescould be empty, causing anIndexOutOfBoundsExceptionwhen accessing[0].content.partscould also be empty, leading to anotherIndexOutOfBoundsException.asImageOrNull()can returnnull, and the non-null assertion operator!!will then throw aNullPointerException.
You should use safe calls (?.) and handle potential empty lists or null values gracefully, for example by throwing a more specific exception that can be handled by the caller.
return response.candidates.firstOrNull()?.content?.parts?.firstOrNull()?.asImageOrNull()
?: throw IllegalStateException("Could not extract image from fine-tuned model response")| override fun getFineTunedModelName(): String { | ||
| TODO("Not yet implemented") | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is not implemented and calls TODO(), which will throw a NotImplementedError and crash any test that covers the useImagen() == false path. You should provide a test implementation, for example, by returning a dummy model name.
| override fun getFineTunedModelName(): String { | |
| TODO("Not yet implemented") | |
| } | |
| override fun getFineTunedModelName(): String { | |
| return "test-fine-tuned-model" | |
| } |
| override fun useImagen(): Boolean { | ||
| return remoteConfig.getBoolean("use_imagen") | ||
| } | ||
| override fun getFineTunedModelName() : String { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's a space before the colon in the function signature, which goes against the standard Kotlin formatting guidelines. The signature should be getFineTunedModelName(): String. This issue is also present in the interface definition on line 40.
| override fun getFineTunedModelName() : String { | |
| override fun getFineTunedModelName(): String { |
core/network/src/main/java/com/android/developers/androidify/vertexai/FirebaseAiDataSource.kt
Outdated
Show resolved
Hide resolved
| <value>This 3D rendered, cartoonish Android mascot rendered in a photorealistic style, the pose is relaxed and straightforward, facing directly forward with his shoulders at ease, as if posing for a photo. The cartoonish exaggeration is subtle, lending a playful touch to the otherwise realistic rendering of the figure. The figure is centered against a muted, neutral warm cream coloured background (#F8F2E4) gives the figurine a unique and collectible appeal. | ||
| The bot should take on the body shape of the newest Google Android Robot (Shape: It has the distinctive rounded body. The main body is a slightly barrel-shaped form with a smooth, continuous surface connecting to the head without a distinct neck, a semi-circular head with Two short, straight antennae protrude vertically from the top of the dome, positioned towards the sides, and simple, cylindrical arms and legs.) but the characteristics of the description should be used. It should NOT use the model shape or color from pre 2024.</value> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This prompt uses the gendered pronoun "his" (...with his shoulders at ease...). This contradicts the guidelines in the prompt_image_description key (line 38) which states: "The description should always include gender neutral language when referring to the subject, such as 'it' or 'the figure'." To maintain consistency across your prompts, you should use gender-neutral terms here as well.
<value>This 3D rendered, cartoonish Android mascot rendered in a photorealistic style, the pose is relaxed and straightforward, facing directly forward with its shoulders at ease, as if posing for a photo. The cartoonish exaggeration is subtle, lending a playful touch to the otherwise realistic rendering of the figure. The figure is centered against a muted, neutral warm cream coloured background (#F8F2E4) gives the figurine a unique and collectible appeal.
The bot should take on the body shape of the newest Google Android Robot (Shape: It has the distinctive rounded body. The main body is a slightly barrel-shaped form with a smooth, continuous surface connecting to the head without a distinct neck, a semi-circular head with Two short, straight antennae protrude vertically from the top of the dome, positioned towards the sides, and simple, cylindrical arms and legs.) but the characteristics of the description should be used. It should NOT use the model shape or color from pre 2024.</value>
No description provided.