Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change of models in the middle of the image generation #1901

Closed
morganavr opened this issue Oct 7, 2022 · 3 comments
Closed

Change of models in the middle of the image generation #1901

morganavr opened this issue Oct 7, 2022 · 3 comments
Labels
enhancement New feature or request

Comments

@morganavr
Copy link

morganavr commented Oct 7, 2022

Waifu Diffusion and NovelAI (private) models know many danbooru tags, such as "leg_up", "knees_together_feet_apart", "kneeling" not recognized by standard SD model. Prompt editing already supports feature of changing part of prompts on the run with syntax [woman:man:0.2]. Would not it be possible to change models (ckpt files) as well?

I think the prompt syntax can be like this:
[waifu.ckpt:"1girl, leg up, masterpiece, bikini";default_sd.ckpt:"photo of young woman wearing bikini, kodak portra 400";0.6]

It will work this way - first "waifu.ckpt" will be used to generate 60% of steps, then model will be changed to "default_sd.ckpt" and remaining steps will be processed with it. I hope that this change of models in runtime will result in correct pose (not possible to achieve with default SD) and then style will be changed from anime to realistic in the remaining 40% steps.

@bmaltais
Copy link

bmaltais commented Oct 7, 2022

Could you not achieve this manually by switching the model and doing the 2nd part in img2img?

@utility-ovo
Copy link

i exchange Waifu Diffusion to NovelAI model,and try to use img2img .But the cmd only told me“Draw mask' is not in list”.How to
solve this problem?

@mezotaken mezotaken added the enhancement New feature or request label Jan 12, 2023
@catboxanon
Copy link
Collaborator

Would be resolved by #12371, #12377

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants