-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable Checkpointing #420
Enable Checkpointing #420
Conversation
README.md
Outdated
... | ||
``` | ||
|
||
Note that we recommend restoring the model using the same `opt_level`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Also note that we recommend calling the load_state_dict
methods after amp.initialize
."
docs/source/amp.rst
Outdated
# Continue training | ||
... | ||
|
||
Note that we recommend restoring the model using the same opt_level. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"... opt_level
. Also note that we recommend calling the load_state_dict
methods after amp.initialize
."
PR enables checkpointing for
apex
by:amp.state_dict()
,amp.load_state_dict()
methodmodel.state_dict()
to return FP32 parametersTODO: