Skip to content

Normalize negative GroupNorm axes against the input rank - #923

Open
Linxiushen wants to merge 1 commit into
google-deepmind:mainfrom
Linxiushen:codex/fix-groupnorm-negative-axes
Open

Linxiushen wants to merge 1 commit into
google-deepmind:mainfrom
Linxiushen:codex/fix-groupnorm-negative-axes

Conversation

@Linxiushen

Copy link
Copy Markdown

GroupNorm(axis=(-3, -2, -1)) currently gives different outputs and input gradients from the equivalent positive-axis configuration on a rank-four input. The reduction axes are mapped after a group dimension is inserted, so negative indices can select different dimensions and mix groups. This affects both NHWC and NCHW layouts, including a single integer axis such as axis=-2.

Canonicalize negative axes against the original input rank before mapping them into the grouped shape. Positive axes and slice handling retain their existing mapping. Add a 24-case regression matrix covering both layouts, integer and tuple axes, affine parameters enabled/disabled, eager/JIT execution, and weighted input gradients.

Validation on Windows CPU with Python 3.12.14, JAX 0.8.0 and NumPy 2.5.4, against current main 4ec60bb58723d5db5aa11c12b3fc6374b5b01726:

  • The exact new regression tests with unchanged base production code: 24 failed, 28 deselected.
  • python -m pytest haiku/_src/group_norm_test.py haiku/_src/layer_norm_test.py haiku/_src/batch_norm_test.py -q: 112 passed, 1 skipped after the fix.
  • git diff --check passes.
  • Strict Pylint 4.1.3 does not pass on either base or candidate: the repository configuration contains unsupported options and existing source/style findings. The candidate adds one import-order warning for the new jax import, which follows the surrounding alphabetical import convention. No lint suppression was added.

This was tested through Haiku's native JAX CPU unit paths. GPU/TPU execution, full-model training, distributed execution, performance and the complete repository suite were not tested.

AI assistance was used to investigate, implement, test and independently review this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant