BETO is Spanish BERT

In this repo we open-source our SpanishBERT model.

Download

Tensorflow

Uncased BETO (vocab, config, weights)

Cased BETO (vocab, config, weights) - Coming soon!

PyTorch

Uncased BETO (vocab, config, weights)

Cased BETO (vocab, config, weights) - Coming soon!

Example of use

For further details you can visit the amazing 🤗 Transformers repo.

Evaluation

This is WIP: For now you can visit beto-benchmarking to see the progress.

Training details

Corpora

We used the Spanish Unnanotated Corpora to train. For further details please visit the respective repo: SUC

Vocabulary

Acknowledgments

FAQ

References

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (paper, code)

🤗 Transformers: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch (paper, code)