We provide links for you to download the pre-trained weights of some Uni-Perceiver models.
| Model | Link | Params | Hidden size | Intermediate size | Num. of heads | Enc layers |
|---|---|---|---|---|---|---|
| Uni Perceiverbase | Downloadtorch Downloadds | 124M | 768 | 3072 | 12 | 12 |
| Uni Perceiver MoEbase | Downloadds | 167M | 768 | 3072 | 12 | 12 |
| Uni Perceiverlarge | Downloadds | 354M | 1024 | 4096 | 16 | 24 |
| Uni Perceiver MoElarge | Downloadds | 505M | 1024 | 4096 | 16 | 24 |
-
Models torch are pretrained in our current codebase, while models ds are pretrained in our previous codebase with DeepSpeed engine.
-
Paramsin the table is the parameters required during model deployment for image-text tasks. Note that it may be different on other tasks. -
More pre-trained weights will be released.