Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wrong gradient type when dividing by integer #727

Closed
CarloLucibello opened this issue Jul 8, 2020 · 4 comments · Fixed by JuliaDiff/ChainRules.jl#233
Closed

wrong gradient type when dividing by integer #727

CarloLucibello opened this issue Jul 8, 2020 · 4 comments · Fixed by JuliaDiff/ChainRules.jl#233

Comments

@CarloLucibello
Copy link
Member

When performing division of a Float32 by an Int I get a Float64 gradient

julia> using Zygote

julia> gradient(x -> x/2, 1f0)
(0.5,)

@oxinabox @MikeInnes @sethaxen @DhairyaLGandhi any clue of when was this introduced?

This is causing FluxML/Flux.jl#1269 and possibly also FluxML/Flux.jl#1255

@CarloLucibello
Copy link
Member Author

Is this the rule used? Should we replace it with

@scalar_rule x / y (one(x)/y, -((x / y) / y))

or something similar?

@sethaxen
Copy link
Contributor

sethaxen commented Jul 8, 2020

Yeah, that's the culprit:

julia> z, back = rrule(/, 1f0, 2)
(0.5f0, ChainRules.var"#575#/_pullback#149"{Float32,Int64}(1.0f0, 2))

julia> back(1)
(Zero(), 0.5, -0.25f0)

Your proposed change looks right to me. inv is just too problematic for promoting everything to Float64 or higher, and we should probably just remove it from all @scalar_rrules.

@mcabbott
Copy link
Member

mcabbott commented Jul 8, 2020

Also fixed in JuliaDiff/DiffRules.jl#46, but without a test it seems.

@DhairyaLGandhi
Copy link
Member

+1 on removing dependency on inv

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants