Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minor Fine-Tuning Scheduler Tutorial Updates for Lightning 1.9.x #225

Merged
merged 4 commits into from
Mar 14, 2023

Conversation

speediedan
Copy link
Contributor

What does this PR do?

Three minor updates to the Fine-Tuning Scheduler tutorial to better support the release of Lightning 1.9.x (supported by Fine-Tuning Scheduler 0.4.x)

  • updated the list of supported strategies (Multi-phase Scheduled Fine-Tuning with FSDP is now supported! Noted deprecation of Fairscale-based strategies like ddp_sharded)
  • Added references to StrategyAdapters. If users want to extend Fine-Tuning Scheduler (FTS) to use a custom, currently unsupported strategy or override current FTS behavior in the context of a given training strategy, subclassing StrategyAdapter is now a way to do so. See FSDPStrategyAdapter for an example implementation.
  • replaced a typehint reference to a datasets class (Batch) that had been replaced in 2.8.0 necessitating the package ceiling in the FTS (0.3.x) (Lightning 1.8.x) version of the tutorial

I recognize #224 might need to get merged before this PR can be. When #224 is passing, I'll update this PR to verify it complies with the new ruff validation/transformation. ruff looks amazing btw!

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?

…upport the release of Lightning 1.9.x (supported by Fine-Tuning Scheduler 0.4.x)
@mergify mergify bot requested a review from Borda January 25, 2023 20:13
@codecov
Copy link

codecov bot commented Jan 25, 2023

Codecov Report

Merging #225 (ae1616f) into main (cc59322) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files
@@         Coverage Diff         @@
##           main   #225   +/-   ##
===================================
  Coverage    73%    73%           
===================================
  Files         2      2           
  Lines       382    382           
===================================
  Hits        280    280           
  Misses      102    102           

@speediedan speediedan marked this pull request as ready for review January 25, 2023 20:22
@Borda Borda enabled auto-merge (squash) March 14, 2023 01:10
@Borda Borda disabled auto-merge March 14, 2023 01:18
@Borda Borda merged commit fbed9e5 into Lightning-AI:main Mar 14, 2023
@Borda Borda added the enhancement New feature or request label Mar 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants