Commits

Branch selector

 main

User selector

thomasw21

Datepicker

All time

Commit History

Commits on Jul 7, 2022

• Fix causal attention mask (bigscience-workshop#306)

  thomasw21

  committedJul 7, 2022
  38607ae

• Add bias a weight we need to sync as well (bigscience-workshop#307)

  thomasw21

  committedJul 7, 2022
  0d0d84c

Commits on Jul 6, 2022

• Add support for weighted train (bigscience-workshop#299)

  thomasw21

  committedJul 6, 2022
  43ab0e0

Commits on Jul 5, 2022

• MTF train script (bigscience-workshop#295)

  

  4 peoplecommittedJul 5, 2022
  3d5d151

Commits on Jul 2, 2022

• MTF dataset and packing (bigscience-workshop#293)

  

  4 peoplecommittedJul 2, 2022
  c5b88fb

Commits on Jun 30, 2022

• Merge MLM too fast 2 (bigscience-workshop#294)

  thomasw21

  committedJun 30, 2022
  131bd43

Commits on Jun 27, 2022

• Fixed MLM dataset arguments(bigscience-workshop#290)

  thomasw21

  committedJun 27, 2022
  55f8cf8

Commits on May 3, 2022

• Fix mixed fused layer norm to mimick nn.LayerNorm for torch>1.11 (bigscience-workshop#281)

  thomasw21

  committedMay 3, 2022
  908dc9c

Commits on Feb 26, 2022

• Try to read the data path arguments directly from a file (bigscience-workshop#254)

  thomasw21

  committedFeb 26, 2022
  cf2d23a

Commits on Feb 25, 2022

• Use padded vocab size in preprocessing scripts (bigscience-workshop#253)

  thomasw21

  committedFeb 25, 2022
  99c8fe0

• Fix preprocess_data_many_cores to use dtype

  thomasw21

  committedFeb 25, 2022
  b289eb8

• Fix preprocess_data_many_cores to use dtype

  thomasw21

  committedFeb 25, 2022
  2c43a0f

Commits on Jan 6, 2022

• Fix alibi (bigscience-workshop#222)

  thomasw21

  committedJan 6, 2022
  7ab5c05

Commits on Nov 27, 2021

• Revert "Checking we use fused kernels to compute scaled masked softmax on prefix lm (bigscience-workshop#209)"

  thomasw21

  committedNov 27, 2021
  c9afebc

Commits on Nov 26, 2021

• Checking we use fused kernels to compute scaled masked softmax on prefix lm (bigscience-workshop#209)

  thomasw21

  committedNov 26, 2021
  b227590

• Reweighting strat for prefix lm (bigscience-workshop#190)

  thomasw21

  committedNov 26, 2021
  b3cf175

Commits on Nov 10, 2021

• [PrefixLM] Figuring out why prefix lm is doing poorly on short context (bigscience-workshop#169)

  thomasw21

  committedNov 10, 2021
  6d146b5

Commits on Nov 4, 2021

• Alternative fix to TP > 1 (bigscience-workshop#178)

  thomasw21

  committedNov 4, 2021
  2d9744f

Commits on Oct 29, 2021

• Fix prefix lm offsets (bigscience-workshop#167)

  thomasw21

  committedOct 29, 2021
  5e1f210

Commits on Oct 25, 2021

• Bump minimum version for torch (bigscience-workshop#156)

  thomasw21

  committedOct 25, 2021
  5b20b75

Commits on Oct 22, 2021

• [Logging] Improve logging mechanism (bigscience-workshop#154)

  thomasw21

  committedOct 22, 2021
  a82462b

• Fix glu activation (bigscience-workshop#148)

  

  thomasw21

  and

  stas00

  committedOct 22, 2021
  bdc6ad6

Commits on Oct 10, 2021

• Fix deepspeed prefix-lm (bigscience-workshop#107)

  thomasw21

  committedOct 10, 2021
  da31db6

Commits on Sep 16, 2021

• Fix model tests (bigscience-workshop#103)

  thomasw21

  committedSep 16, 2021
  48dcee4

• Prefix lm (bigscience-workshop#52)

  

  9 peoplecommittedSep 16, 2021
  68b46f2

Commits on Aug 26, 2021

• Check cardon directory is not None (bigscience-workshop#74)

  thomasw21

  committedAug 26, 2021
  e96df7d

Commits on Aug 18, 2021

• Add openwebtext1000.jsonl to .gitignore

  thomasw21

  committedAug 18, 2021
  9341269

• - Recompute bin/idx using microsoft/Megatron-DeepSpeed (Not changes)

  thomasw21

  committedAug 18, 2021
  5eeae0b

Commits on Aug 6, 2021

• Fix pretrain_gpt_single_node example script to have only one occurence of lr-decay-style

  thomasw21

  committedAug 6, 2021
  7228b89

Commits on Aug 4, 2021

• Faster preprocessing (bigscience-workshop#18)

  

  thomasw21

  and

  Thomas

  committedAug 4, 2021
  7b99881

Commits on Jul 28, 2021

• Implement rotary embeddings (bigscience-workshop#7)

  

  thomasw21

  and

  Thomas

  committedJul 28, 2021
  dc4e0cb

• Fix inconsistent merge (bigscience-workshop#30)

  

  thomasw21

  and

  Thomas

  committedJul 28, 2021
  0125aaa

Commits on Jul 26, 2021

• Seperate merge script from process_data script (bigscience-workshop#19)

  

  thomasw21

  and

  Thomas

  committedJul 26, 2021
  e8fcbae