DeepSpeedExamples major restructure (microsoft#245)

AndrewZhe · Mar 31, 2023 · 127c7a1 · 127c7a1
1 parent ba7c0e5
commit 127c7a1
Show file tree

Hide file tree

Showing 1,685 changed files with 15 additions and 63,752 deletions.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
diff --git a/README.md b/README.md
@@ -1,15 +1,17 @@
-
 # DeepSpeed Examples
-This repository contains various example models that use [DeepSpeed](https://github.com/microsoft/DeepSpeed) for training and inference.
+This repository contains various examples including training, inference, compression, benchmarks, and applications that use [DeepSpeed](https://github.com/microsoft/DeepSpeed).
+
+## 1. Training
+There are several training and finetuning examples so please see the individual folders for specific instructions.
 
-# Inference Examples
+## 2. Inference
 The DeepSpeed Huggingface inference [README](./inference/huggingface/README.md) explains how to get started with running DeepSpeed Huggingface inference examples.
 
-# Training Examples
-There are several trianing examples in this repository. Please see the individual folders. 
+## 3. Compression
+Model compression examples.
 
-## Note on Megatron examples
-Please use the latest [Megatron-DeepSpeed fork](https://github.com/microsoft/Megatron-DeepSpeed) instead of the deprecated/old megatron forks in the megatron folder.
+## 4. Benchmarks
+TODO: Move the DeepSpeed benchmarks folder here.
 
 # Contributing
 
@@ -23,4 +25,4 @@ provided by the bot. You will only need to do this once across all repos using o
 
 This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
 For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
-contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
+contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
diff --git a/benchmarks/README.md b/benchmarks/README.md
@@ -0,0 +1 @@
+The new home for DeepSpeed benchmarks. TODO: Move DS benchmarks to this repo. 
diff --git a/model_compression/bert/README.md → compression/bert/README.md b/model_compression/bert/README.md → compression/bert/README.md
diff --git a/...n/bert/bash_script/XTC/layer_reduction.sh → ...n/bert/bash_script/XTC/layer_reduction.sh b/...n/bert/bash_script/XTC/layer_reduction.sh → ...n/bert/bash_script/XTC/layer_reduction.sh
diff --git a/...t/bash_script/XTC/layer_reduction_1bit.sh → ...t/bash_script/XTC/layer_reduction_1bit.sh b/...t/bash_script/XTC/layer_reduction_1bit.sh → ...t/bash_script/XTC/layer_reduction_1bit.sh
diff --git a/...ession/bert/bash_script/XTC/quant_1bit.sh → ...ession/bert/bash_script/XTC/quant_1bit.sh b/...ession/bert/bash_script/XTC/quant_1bit.sh → ...ession/bert/bash_script/XTC/quant_1bit.sh
diff --git a/.../bert/bash_script/ZeroQuant/zero_quant.sh → .../bert/bash_script/ZeroQuant/zero_quant.sh b/.../bert/bash_script/ZeroQuant/zero_quant.sh → .../bert/bash_script/ZeroQuant/zero_quant.sh
diff --git a/...t/bash_script/ZeroQuant/zero_quant_lkd.sh → ...t/bash_script/ZeroQuant/zero_quant_lkd.sh b/...t/bash_script/ZeroQuant/zero_quant_lkd.sh → ...t/bash_script/ZeroQuant/zero_quant_lkd.sh
diff --git a/...ssion/bert/bash_script/layer_reduction.sh → ...ssion/bert/bash_script/layer_reduction.sh b/...ssion/bert/bash_script/layer_reduction.sh → ...ssion/bert/bash_script/layer_reduction.sh
diff --git a/...pression/bert/bash_script/pruning_head.sh → compression/bert/bash_script/pruning_head.sh b/...pression/bert/bash_script/pruning_head.sh → compression/bert/bash_script/pruning_head.sh
diff --git a/...mpression/bert/bash_script/pruning_row.sh → compression/bert/bash_script/pruning_row.sh b/...mpression/bert/bash_script/pruning_row.sh → compression/bert/bash_script/pruning_row.sh
diff --git a/...ession/bert/bash_script/pruning_sparse.sh → ...ession/bert/bash_script/pruning_sparse.sh b/...ession/bert/bash_script/pruning_sparse.sh → ...ession/bert/bash_script/pruning_sparse.sh
diff --git a/...sion/bert/bash_script/quant_activation.sh → ...sion/bert/bash_script/quant_activation.sh b/...sion/bert/bash_script/quant_activation.sh → ...sion/bert/bash_script/quant_activation.sh
diff --git a/...pression/bert/bash_script/quant_weight.sh → compression/bert/bash_script/quant_weight.sh b/...pression/bert/bash_script/quant_weight.sh → compression/bert/bash_script/quant_weight.sh
diff --git a/...nfig/XTC/ds_config_W1A8_Qgroup1_fp32.json → ...nfig/XTC/ds_config_W1A8_Qgroup1_fp32.json b/...nfig/XTC/ds_config_W1A8_Qgroup1_fp32.json → ...nfig/XTC/ds_config_W1A8_Qgroup1_fp32.json
diff --git a/.../ds_config_layer_reduction_W1Q8_fp32.json → .../ds_config_layer_reduction_W1Q8_fp32.json b/.../ds_config_layer_reduction_W1Q8_fp32.json → .../ds_config_layer_reduction_W1Q8_fp32.json
diff --git a/...g/XTC/ds_config_layer_reduction_fp16.json → ...g/XTC/ds_config_layer_reduction_fp16.json b/...g/XTC/ds_config_layer_reduction_fp16.json → ...g/XTC/ds_config_layer_reduction_fp16.json
diff --git a/...nt/ds_config_W48A8_Qgroup48_lkd_fp32.json → ...nt/ds_config_W48A8_Qgroup48_lkd_fp32.json b/...nt/ds_config_W48A8_Qgroup48_lkd_fp32.json → ...nt/ds_config_W48A8_Qgroup48_lkd_fp32.json
diff --git a/...roQuant/ds_config_W8A8_Qgroup48_fp32.json → ...roQuant/ds_config_W8A8_Qgroup48_fp32.json b/...roQuant/ds_config_W8A8_Qgroup48_fp32.json → ...roQuant/ds_config_W8A8_Qgroup48_fp32.json
diff --git a/model_compression/bert/config/ds_config.json → compression/bert/config/ds_config.json b/model_compression/bert/config/ds_config.json → compression/bert/config/ds_config.json
diff --git a/...ssion/bert/config/ds_config_TEMPLATE.json → ...ssion/bert/config/ds_config_TEMPLATE.json b/...ssion/bert/config/ds_config_TEMPLATE.json → ...ssion/bert/config/ds_config_TEMPLATE.json
diff --git a/.../config/ds_config_W1A8_Qgroup64_fp16.json → .../config/ds_config_W1A8_Qgroup64_fp16.json b/.../config/ds_config_W1A8_Qgroup64_fp16.json → .../config/ds_config_W1A8_Qgroup64_fp16.json
diff --git a/.../config/ds_config_W1A8_Qgroup64_fp32.json → .../config/ds_config_W1A8_Qgroup64_fp32.json b/.../config/ds_config_W1A8_Qgroup64_fp32.json → .../config/ds_config_W1A8_Qgroup64_fp32.json
diff --git a/...nfig/ds_config_W1or2A8_Qgroup64_fp16.json → ...nfig/ds_config_W1or2A8_Qgroup64_fp16.json b/...nfig/ds_config_W1or2A8_Qgroup64_fp16.json → ...nfig/ds_config_W1or2A8_Qgroup64_fp16.json
diff --git a/.../huggingface_transformer/modeling_bert.py → .../huggingface_transformer/modeling_bert.py b/.../huggingface_transformer/modeling_bert.py → .../huggingface_transformer/modeling_bert.py
diff --git a/model_compression/bert/requirements.txt → compression/bert/requirements.txt b/model_compression/bert/requirements.txt → compression/bert/requirements.txt
diff --git a/model_compression/bert/run_glue_lkd.py → compression/bert/run_glue_lkd.py b/model_compression/bert/run_glue_lkd.py → compression/bert/run_glue_lkd.py
diff --git a/...l_compression/bert/run_glue_no_trainer.py → compression/bert/run_glue_no_trainer.py b/...l_compression/bert/run_glue_no_trainer.py → compression/bert/run_glue_no_trainer.py
diff --git a/model_compression/bert/util.py → compression/bert/util.py b/model_compression/bert/util.py → compression/bert/util.py
diff --git a/model_compression/cifar/README.md → compression/cifar/README.md b/model_compression/cifar/README.md → compression/cifar/README.md
diff --git a/...l_compression/cifar/config/ds_config.json → compression/cifar/config/ds_config.json b/...l_compression/cifar/config/ds_config.json → compression/cifar/config/ds_config.json
diff --git a/...cifar/config/ds_config_channel_prune.json → ...cifar/config/ds_config_channel_prune.json b/...cifar/config/ds_config_channel_prune.json → ...cifar/config/ds_config_channel_prune.json
diff --git a/model_compression/cifar/resnet.py → compression/cifar/resnet.py b/model_compression/cifar/resnet.py → compression/cifar/resnet.py
diff --git a/model_compression/cifar/run_compress.sh → compression/cifar/run_compress.sh b/model_compression/cifar/run_compress.sh → compression/cifar/run_compress.sh
diff --git a/model_compression/cifar/train.py → compression/cifar/train.py b/model_compression/cifar/train.py → compression/cifar/train.py
diff --git a/model_compression/cifar/utils.py → compression/cifar/utils.py b/model_compression/cifar/utils.py → compression/cifar/utils.py
diff --git a/model_compression/gpt2/README.md → compression/gpt2/README.md b/model_compression/gpt2/README.md → compression/gpt2/README.md
diff --git a/...ession/gpt2/bash_script/run_zero_quant.sh → ...ession/gpt2/bash_script/run_zero_quant.sh b/...ession/gpt2/bash_script/run_zero_quant.sh → ...ession/gpt2/bash_script/run_zero_quant.sh
diff --git a/model_compression/gpt2/config/ds_config.json → compression/gpt2/config/ds_config.json b/model_compression/gpt2/config/ds_config.json → compression/gpt2/config/ds_config.json
diff --git a/...nfig/ds_config_W4or8A8_Qgroup64_fp16.json → ...nfig/ds_config_W4or8A8_Qgroup64_fp16.json b/...nfig/ds_config_W4or8A8_Qgroup64_fp16.json → ...nfig/ds_config_W4or8A8_Qgroup64_fp16.json
diff --git a/...nfig/ds_config_W4or8A8_Qgroup64_fp32.json → ...nfig/ds_config_W4or8A8_Qgroup64_fp32.json b/...nfig/ds_config_W4or8A8_Qgroup64_fp32.json → ...nfig/ds_config_W4or8A8_Qgroup64_fp32.json
diff --git a/.../config/ds_config_W8A8_Qgroup64_fp16.json → .../config/ds_config_W8A8_Qgroup64_fp16.json b/.../config/ds_config_W8A8_Qgroup64_fp16.json → .../config/ds_config_W8A8_Qgroup64_fp16.json
diff --git a/.../config/ds_config_W8A8_Qgroup64_fp32.json → .../config/ds_config_W8A8_Qgroup64_fp32.json b/.../config/ds_config_W8A8_Qgroup64_fp32.json → .../config/ds_config_W8A8_Qgroup64_fp32.json
diff --git a/model_compression/gpt2/requirements.txt → compression/gpt2/requirements.txt b/model_compression/gpt2/requirements.txt → compression/gpt2/requirements.txt
diff --git a/model_compression/gpt2/run_clm_no_trainer.py → compression/gpt2/run_clm_no_trainer.py b/model_compression/gpt2/run_clm_no_trainer.py → compression/gpt2/run_clm_no_trainer.py
diff --git a/megatron/Megatron-LM-v1.1.5-3D_parallelism/LICENSE b/megatron/Megatron-LM-v1.1.5-3D_parallelism/LICENSE
diff --git a/megatron/Megatron-LM-v1.1.5-3D_parallelism/MANIFEST.in b/megatron/Megatron-LM-v1.1.5-3D_parallelism/MANIFEST.in