added muon optimizer component#449
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds support for a new Muon optimizer component to the Modalities component/registry system, including configuration wiring and unit tests to ensure the optimizer factory constructs the optimizer using the expected parameter groups.
Changes:
- Added
OptimizerFactory.get_muon(...)that builds a Muon optimizer fromget_optimizer_groups(...). - Introduced
MuonOptimizerConfigand registered the"optimizer" / "muon"component variant. - Added unit tests verifying optimizer factory methods construct Adam/AdamW/Muon using computed optimizer groups.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
tests/test_optimizer_factory.py |
Adds unit tests for Adam/AdamW/Muon factory construction and checkpointed optimizer loading behavior. |
src/modalities/registry/components.py |
Registers the new "muon" optimizer component variant and wires it to MuonOptimizerConfig. |
src/modalities/optimizers/optimizer_factory.py |
Adds get_muon factory method and imports Muon from torch.optim. |
src/modalities/config/config.py |
Adds MuonOptimizerConfig to represent Muon optimizer configuration in the component system. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
le1nux
left a comment
There was a problem hiding this comment.
Two tiny comments. I'll leave the final decision to you. Apart from that all looks good to me and can be merged :)
| ) | ||
|
|
||
|
|
||
| def test_get_fsdp1_checkpointed_optimizer_loads_optimizer_state(): |
There was a problem hiding this comment.
Should we upgrade this to FSDP2 since FSDP1 is anyways deprecated?
| ) | ||
|
|
||
|
|
||
| def test_get_muon_builds_optimizer_from_groups(monkeypatch): |
There was a problem hiding this comment.
not sure if these optimizer_builds_optimizer_from_groups tests are important. To me they don't really test anything apart from making sure that the factory calls the right opimizer functions.
What does this PR do?
This PR adds muon optimizer component
General Changes
Checklist before submitting final PR
python tests/tests.py)CHANGELOG_DEV.md)