Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Request]: Support for DORA and other types of LORAs #3192

Open
1 task done
santiagosfx opened this issue Jun 25, 2024 · 5 comments · May be fixed by #3084
Open
1 task done

[Feature Request]: Support for DORA and other types of LORAs #3192

santiagosfx opened this issue Jun 25, 2024 · 5 comments · May be fixed by #3084
Labels
enhancement New feature or request

Comments

@santiagosfx
Copy link

santiagosfx commented Jun 25, 2024 •
edited
Loading

Is there an existing issue for this?

  • I have searched the existing issues and checked the recent builds/commits

What would your feature do?

Compatibility with DORAs and other types of LORAs to generate images. Fooocus simply cannot detect a Dora. You can load it, but it will not be reflected in the image generation. It will not influence it.

Proposed workflow

Work as usual

Additional information

Dora is basically a potential successor to Lora, or a variant. I'll leave more information about it in the commentary, and some Dora models from Civitai as examples.
https://www.reddit.com/r/StableDiffusion/comments/1asxk3u/dora_weightdecomposed_lowrank_adaptation/ https://sebastianraschka.com/blog/2024/lora-dora.html

Sample civitai lora model: https://civitai.com/models/483185/takagi-san-karakai-jouzu-no-takagi-san
@santiagosfx santiagosfx added enhancement New feature or request triage This needs an (initial) review labels Jun 25, 2024
@mashb1t
Copy link
Collaborator

mashb1t commented Jun 25, 2024

Please provide more details about what your expectation is with this feature and what does / does not work. The description can e.g. be enriched with references to papers or models.

@santiagosfx
Copy link
Author

According to the tests I have done, Foocus simply does not detect the Doras. I tried with a concept, and a drawing style, and it failed to be detected. It generated images, but it was as if the dora was not there. Even if I had loaded it and used the trigger word.
As far as I know, Dora is basically a potential successor to Lora, or a variant. I'll leave more information about it in the commentary, and some Dora models from Civitai as examples.
https://www.reddit.com/r/StableDiffusion/comments/1asxk3u/dora_weightdecomposed_lowrank_adaptation/ https://sebastianraschka.com/blog/2024/lora-dora.html

Sample civitai lora model: https://civitai.com/models/483185/takagi-san-karakai-jouzu-no-takagi-san

@mashb1t
Copy link
Collaborator

mashb1t commented Jun 25, 2024 •
edited
Loading

Thanks for elaborating. DoRA support is also included in #3084 (more specific: here), but sadly i need some extra time and/or support here. Any help is welcome!

@mashb1t mashb1t linked a pull request Jun 26, 2024 that will close this issue
feat: update ldm_patched #3084
Draft
@mashb1t mashb1t removed the triage This needs an (initial) review label Jun 26, 2024
@raochinmay6

This comment was marked as off-topic.

Sign in to view
@mashb1t

This comment was marked as off-topic.

Sign in to view
@mohamed-benali mohamed-benali mentioned this issue Sep 4, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: update ldm_patched lllyasviel/Fooocus
3 participants
@mashb1t @santiagosfx @raochinmay6