Skip to content

[Misc] add Gemma 4 config parser#579

Open
ankrovv wants to merge 1 commit intoome-projects:mainfrom
ankrovv:gemma4-parser
Open

[Misc] add Gemma 4 config parser#579
ankrovv wants to merge 1 commit intoome-projects:mainfrom
ankrovv:gemma4-parser

Conversation

@ankrovv
Copy link
Copy Markdown

@ankrovv ankrovv commented Apr 17, 2026

Motivation

Adds a config parser for Google Gemma 4 multimodal models to pkg/hfutil/modelconfig.

Modifications

  • Add Gemma4Config struct implementing the HuggingFaceModel interface, with nested Gemma4TextConfig, Gemma4VisionConfig, and Gemma4AudioConfig
  • Model Gemma 4's MoE structure via pointer fields (NumExperts, TopKExperts, MoeIntermediateSize) so dense variants (null in JSON) round-trip as nil and MoE variants (26B-A4B) retain their 128-expert / top-k-8 shape.
  • Model the optional audio encoder as AudioConfig *Gemma4AudioConfig so null on text+vision-only variants (26B, 31B) deserialises to nil and populated encoders (E2B, E4B) round-trip correctly.
  • Promote the top-level dtype field (used by transformers 5.x) to TorchDtype during LoadGemma4Config so EstimateModelSizeBytes and downstream runtime sizing stay accurate
  • GetContextLength() returns text_config.max_position_embeddings directly: 131072 on E2B/E4B, 262144 on 26B-A4B/31B.
  • GetParameterCount() first reads from co-located safetensors files via FindAndParseSafetensors, then falls back to a shape-based lookup table for the four published variants, then to estimateMoEParamCount for MoE and estimateTextParams + estimateVisionParams for dense unknowns.
  • GetQuantizationType() inspects the optional quantization_config.quant_method and returns "fp8" for FP8 checkpoints, "int4" for any other non-nil quant config, "" otherwise so the runtime matcher can reject quantized checkpoints from unquantized runtimes.
  • HasVision() returns true for every variant (all four ship the vision tower); IsEmbedding() returns false.
  • Register "gemma4" in the model loader registry via init() so LoadModelConfig dispatches correctly.
  • Add gemma4_test.go
  • Add testdata/gemma4_{e2b,e4b,26b_a4b,31b}.json
  • Update pkg/hfutil/modelconfig/README.md supported-families list and config/models/SUPPORTED_MODELS.md Google Gemma table.

Checklist

  • Format your code with gofumpt and golines (make format).
  • Add unit tests as outlined in the Modifications section.
  • Update documentation / examples if applicable.

@github-actions github-actions Bot added documentation Documentation changes models Model configuration changes tests Test changes config Configuration changes labels Apr 17, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for the Google Gemma 4 model family, implementing configuration loading, parameter count estimation for dense and MoE variants, and unit tests. Feedback suggests sanitizing JSON input to handle non-standard values like NaN or Infinity and replacing fmt.Printf with log.Printf for library-level warnings to avoid polluting stdout.

Comment on lines +97 to +98
var config Gemma4Config
if err := json.Unmarshal(data, &config); err != nil {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The configuration data should be sanitized before unmarshaling to handle non-standard JSON values such as Infinity, -Infinity, or NaN. These values are common in Hugging Face configurations exported from Python but are not valid in standard JSON, causing json.Unmarshal to fail. Using SanitizeJSONBytes (defined in interface.go) ensures the parser is robust against these cases, maintaining consistency with the generic loader path.

	data = SanitizeJSONBytes(data)

	var config Gemma4Config
	if err := json.Unmarshal(data, &config); err != nil {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing applies to all model parsing logic. Will address in the later PR.

Comment thread pkg/hfutil/modelconfig/gemma4.go Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

config Configuration changes documentation Documentation changes models Model configuration changes tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants