T5X is a modular, composable, research-friendly framework for high-performance, configurable, self-service training, evaluation, and inference of sequence models (starting with language) at many scales.
It is essentially a new and improved implementation of the T5 codebase (based on Mesh TensorFlow) in JAX and Flax. To learn more, see the T5X Paper.
Here are some quick tutorials to help you get started with common use-cases on T5X:
If you are new to T5X, we recommend starting with our introductory Colab series, which introduces core concepts of both T5X and SeqIO. More colabs will be added to this series regularly!
This tutorial outlines the steps to fine-tune an existing pre-trained model with T5X on common downstream Tasks/Mixtures available on SeqIO. This is one of the simplest and most common use cases of T5X. If you're new to T5X, this tutorial is the recommended starting point.
This tutorial outlines the steps to evaluate a model with T5X on downstream Tasks/Mixtures defined in SeqIO.
This tutorial outlines the steps to run inference on a model with T5X.
This tutorial outlines the steps to pretrain a model with T5X on Tasks/Mixtures defined in SeqIO.
This tutorial provides a quick introduction to Gin, a lightweight configuration framework for Python that is used to configure training, eval and inference jobs on T5X.
This tutorial provides background on what model and data partitioning are and how it can be configured in T5X.
This tutorial provides an overview of how metrics can be used and customized to evaluate T5X models.