allow for non-distributed envs (Windows) #531
Conversation
|
Looks good to me, thanks! Note that Windows is technically not officially supported... |
I am aware. Luckily I was able to build Apex on Windows 10 including all extensions (64bit, CUDA 10.1) and so the only thing stopping me from using it was this small issue. If I recall correctly, DDP will still not work on Windows (?), but at least AMP should work well out-of-the-box. PS: credit where credit is due: I did nothing more than push the changes that @ptrblck made to the current head. His fork is here. |
| from .amp import init as amp_init | ||
|
|
||
| optimizers_was_list = False | ||
| if isinstance(optimizers, torch.optim.Optimizer) or isinstance(optimizers, LARC): | ||
| if isinstance(optimizers, torch.optim.Optimizer) or ('LARC' in sys.modules and isinstance(optimizers, LARC)): |
There was a problem hiding this comment.
This line does not work in linux, because 'LARC' is always not in sys.modules. Please revert this commit.
>>> import sys
>>> import apex
>>> 'LARC' in sys.modules
False
>>> 'apex.parallel.LARC' in sys.modules
True
There was a problem hiding this comment.
Hm, perhaps @ptrblck can comment on this. I am not sure what that change is supposed to signify. But from the looks of it, you're right that it should be 'apex.parallel.LARC' in sys.modules.
closes #429