add lcm realtime canvas painting

[mashb1t]

Fixes #990

This feature introduces a new tab with a canvas to act as alternative input for uov_input_image with vary strong.
It's recommended to use this with arg --highvram to further reduce generation time.

[diaolulu1]

This feature will cause the shortcut for inpaint to become invalid after updating gradio, and images in the gallery cannot be clicked for fullscreen.

[mashb1t]

@diaolulu1 thank you for pointing that out, will try to check it with gradio v3.45.0. Please keep me posted if you already tried this version.

[thiner]

I tried your enhancement, that's excellent! Thanks for your contribution.
I also found something improvable:

1. When enable realtime painting, it's better set the image numbers to 1 automatically. That's more reasonable for realtime scenario.
2. After adding several objects, seems the previousely painted objects get ignored.
3. The default background color is white, when drawing on the cavas, the background color seems always reconized as user input. E.g., if I input text prompt "moutains", the white part will be rendered as snow. Maybe should not pass the background as image prompt.

[mashb1t]

@thiner this feature might already be superseded by SDXL Turbo.

1. this is absolutely reasonable, adjusted in last commit
2. the generation only happens after the button is interactive again as changing the canvas triggers a javascript click of the button. I couldn't find a better solution but this still seems to be a bit hacky. Feel free to suggest a better way without code duplication + while still being able to maintain the button code.
3. this is in the nature of image generation with img2img ^^

[mashb1t]

@diaolulu1 there is a hardcoded svelte class which changed after updating gradio.
Please test again if it works for you now as well.

[diaolulu1]

@diaolulu1 there is a hardcoded svelte class which changed after updating gradio. Please test again if it works for you now as well.

I've tried the latest modification, the shortcut issue with the inpaint canvas has been fixed, but the images in the gallery preview window still can't be fullscreened after clicking.

[diaolulu1]

At the start of each drawing session, Fooocus loads the model into the VRAM, which certainly wastes time. Is it possible to keep the model constantly in VRAM when in the realtime canvas mode?

[diaolulu1]

It seems unnecessary to upgrade gradio to 3.50.2 as the live canvas still operates normally and all bugs are non-existent.

[thiner]

@diaolulu1 there is a hardcoded svelte class which changed after updating gradio. Please test again if it works for you now as well.

I have encountered a problem on the server side with this Gradio upgrade. The issue is the same as #728 (comment)
I fixed the problem by downgrade Gradio version from 3.50.2 to 3.45.0.

[mashb1t]

@diaolulu1 yes, this is indeed possible. As mentioned in the PR description it is advised to use the argument --highvram to keep the model in VRAM and reduce generaton time. Please give it a shot.

Upgrading Gradio is necessary (at least to 3.45.0) as only updated versions include support for updating image shape.
See gradio-app/gradio#5019 and commit message mashb1t@bed48be

@thiner ok, then let's downgrade to 3.45.0

[diaolulu1]

I've found that the resolution of the real-time canvas doesn't change when I modify the resolution settings.

[mashb1t]

@diaolulu1 tested it on mobile and desktop viewports in both Chrome and Firefox and all combinations are working fine.
Can you please make a screencast and attach it here to figure out what the problems are?
Are there any issues in the browser console when switching aspect ratios?

[diaolulu1]

@diaolulu1 tested it on mobile and desktop viewports in both Chrome and Firefox and all combinations are working fine. Can you please make a screencast and attach it here to figure out what the problems are? Are there any issues in the browser console when switching aspect ratios?

It was my mistake, my gradio was still at version 3.41.2. My issue is now resolved, thank you very much!

[usliders]

простите, уважаемые, вы где такие версии нашли > 3.41.0 ?

Aug 24, 2023
@github-actions github-actions
v3.41.0
gradio-app/gradio@ba4c6d9
Compare
v3.41.0

[mashb1t]

@usliders see https://pypi.org/project/gradio/#history