Usage of NVIDIA tensor core

[andrii0lomakin]

Hi guys.
NVIDIA has a tensor core that can speed up matrix multiplication (sub-matrix).

Are there any ways how I can leverage its usage in TorndavoVM?

[jjfumero]

Hi @Laa , at the moment, TornadoVM does not generate PTX instructions for tensor core operations. However, it is in our future plans to add those as well as Intel AMX instructions.

[andrii0lomakin]

Hi @jjfumero thank you for quick response.
Do you have any issue that I can follow . I am quite interested in this feature.

[andrii0lomakin]

Good day.
I will implement GEMM operation soon, so I wanted to ask if there are any updates on this topic.
And follow-up question. Do you have plans to publish documentation about vectors API soon?

[jjfumero]

We are still analysing performance with vector types. What we have seen is that the combination between the TornadoVM JIT compiler and the driver compiler (E.g, for OpenCL). the generated code is auto-vectorized in some cases, but we are still running benchmarks.

We can prepare a quick overview of the Vector API in TornadoVM in the upcoming week1, but in a nutshell the all vector types are mapped to the target vector language types:

• For example, if you use the TornadoVM type VectorFloat4, and you run with the OpenCL backend, the TornadoVM JIT compiler generates a vector of type float4, with the corresponding loads/stores and vector operations. If you run on Intel CPUs, this code is vectorized and will make use of SSE/AVX/AVX2 instructions for instance.

[andrii0lomakin]

@jjfumero, thank you for your response. I always enjoy conversations with you. Hopefully, I will have the opportunity to contribute.