Repository navigation
Normalize negative GroupNorm axes against the input rank - #923
Open
Linxiushen wants to merge 1 commit into
Open
Linxiushen wants to merge 1 commit into
Linxiushen wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GroupNorm(axis=(-3, -2, -1))currently gives different outputs and input gradients from the equivalent positive-axis configuration on a rank-four input. The reduction axes are mapped after a group dimension is inserted, so negative indices can select different dimensions and mix groups. This affects both NHWC and NCHW layouts, including a single integer axis such asaxis=-2.Canonicalize negative axes against the original input rank before mapping them into the grouped shape. Positive axes and slice handling retain their existing mapping. Add a 24-case regression matrix covering both layouts, integer and tuple axes, affine parameters enabled/disabled, eager/JIT execution, and weighted input gradients.
Validation on Windows CPU with Python 3.12.14, JAX 0.8.0 and NumPy 2.5.4, against current main
4ec60bb58723d5db5aa11c12b3fc6374b5b01726:python -m pytest haiku/_src/group_norm_test.py haiku/_src/layer_norm_test.py haiku/_src/batch_norm_test.py -q: 112 passed, 1 skipped after the fix.git diff --checkpasses.jaximport, which follows the surrounding alphabetical import convention. No lint suppression was added.This was tested through Haiku's native JAX CPU unit paths. GPU/TPU execution, full-model training, distributed execution, performance and the complete repository suite were not tested.
AI assistance was used to investigate, implement, test and independently review this change.