Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix scale dtype and refactor q_dot_dq #4229

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

wenscarl
Copy link
Collaborator

@wenscarl wenscarl commented Sep 25, 2024

  1. Address type mismatches or inconsistencies in scale factors for FP8 operations.
  2. Refactor the q-dot-dq (direct quantization pattern) into smaller, more granular components to enable greater flexibility.
    @kaixih @levskaya
    Support fp8 direct quantization praxis#69 depends on this PR.

wenscarl

This comment was marked as off-topic.

@levskaya
Copy link
Collaborator

I'm not able to do a deep review of the fp8 quant details, but everything looks fine on the surface - you just need to correct one trailing space issue that's blocking our formatting presubmits.

I can't tell if this is ready to go in, but if it is let me know and I can merge it in!

@wenscarl
Copy link
Collaborator Author

I'm not able to do a deep review of the fp8 quant details, but everything looks fine on the surface - you just need to correct one trailing space issue that's blocking our formatting presubmits.

I can't tell if this is ready to go in, but if it is let me know and I can merge it in!

Thanks for reviewing it. The formatting is fixed. It's ready to go in from our e2e test.

@levskaya
Copy link
Collaborator

@wenscarl - I'm seeing actual test failures here? is it just a tolerance issue or something more serious?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants