zoom_enhance makes no changes to image in 8.3.1

[ratsark]

In the latest commit (8.3.1), zoom_enhance will run img2img jobs (visible in the server logs) but doesn't affect the output images. This issue is not present in 8.3.0 (or the immediately-following fix, 7bae7f8), verified via git reset <sha> && git stash.

I'm using show_original to get a side-by-side comparison, and use_workaround seems to have no effect.

[ratsark]

Looking at line 345 of zoom_enhance.py:

# workaround for txt2img, not sure if compatible with controlnet

I do have controlnet installed, but the issue persists even if I don't use it. I'll try disabling.

[ratsark]

Disabling controlnet has no effect.

[ratsark]

Turning debug on, it gets all the way to this line:

(DEBUG) [../../shortcodes/stable_diffusion/zoom_enhance.py] Adding zoom_enhance result for image_idx 5

So it doesn't seem to be raising an exception.

[ratsark]

Two other things I should probably mention:

• I'm using txt2img
• I love the extension, thank you for all the hard work! zoom_enhance is particularly awesome; I was doing this manually for so long and now you've automated me out of a job :)

[p3iurhgmc98h]

If it helps, this is an issue even if you don't have ControlNet installed.

[ThereforeGames]

Hi @ratsark,

Thank you for reaching out! I haven't been able to reproduce this issue yet. After turning debug on, could you check your WebUI folder for a series of zoom_enhance images? Look for a file there called zoom_enhance_5f.png - this should be the enhanced face image.

A few other diagnostic questions:

• Does the shortcode work for you in img2img?
• What version of Python are you on?
• What operating system are you on?

Hope we can get to the bottom of it!

[ThereforeGames]

Hi,

I just released a big update--Unprompted v9.0.0--with a lot of fixes and improvements. It may or may not resolve the issue you've been having with [zoom_enhance]. Feel free to give it a try and let me know!

[R3dPhil]

it persists for me

[ThereforeGames]

Hi @R3dPhil (as well as anyone else who is still having trouble with zoom_enhance),

Please provide any error messages in your console as well as answers to these diagnostic questions to help me identify the cause. Thank you!

[vinch00]

@ThereforeGames there are no error messages in console for zoom enhance. It just doesn't do anything in A1111 latest build.

[Fenwich]

I can't seem to get zoom_enhance working either. I'm a bit new to SD so might just be my fault, but to answer your diagnostic questions:

• It doesn't work in img2img
• Python 3.10.6-amd64
• Windows 10, Geforce 3080

I'm attaching the output, no errors, but it doesn't seem to be finishing (or maybe it's supposed to look like this?). Running debug I see all the pictures generated in the webui folder and the zoomed in faces do show up in the img2img folder. show_original doesn't seem to be doing anything either.

[ThereforeGames]

Hi folks,

I have pushed a new version of [zoom_enhance] that includes a battery of tests I'd like you to try.

First, here's the image we're going for:

• Checkpoint: deliberate_v2.safetensors
• Sampler: Euler a @ 20 steps
• Resolution: 512x512
• CFG scale: 7
• Seed: 1

I want you to run the following prompt with the new test flag and let me know which, if any, produce the correct result:

[after][zoom_enhance replacement="walter white face" debug test=0][/after]an amazing full body photo of walter white

[after][zoom_enhance replacement="walter white face" debug test=1][/after]an amazing full body photo of walter white

[after][zoom_enhance replacement="walter white face" debug test=2][/after]an amazing full body photo of walter white

[after][zoom_enhance replacement="walter white face" debug test=3][/after]an amazing full body photo of walter white

[after][zoom_enhance replacement="walter white face" debug test=4][/after]an amazing full body photo of walter white

[after][zoom_enhance replacement="walter white face" debug test=5][/after]an amazing full body photo of walter white

Check your console for errors after each generation. If you get an error, you'll need to restart the UI before trying the next one.

For this test, please also make sure you meet the following conditions:

• You're using the latest version of the A1111 WebUI and not a third-party repo.
• You do not have other active extensions that could interfere with image output. (better to temporarily disable all other extensions)
• Batch size and batch count of 1.

Finally, please look in your WebUI folder for the new debug image zoom_enhance_final_result.png and let me know if it looks correct.

Thank you for your help.

[Fenwich]

Ran the tests, it never successfully placed the updated image in the txt2img directory, but it did create a successful final result image with the debug in 4/6 tests. It's just not putting them back where the original came from.

• Test 0 final result looks correct, but txt2img output looks like original (wasn't replaced)
• Test 1 error before final result image generated.
  (ERROR) [shortcodes\stable_diffusion\zoom_enhance.py] Could not append zoom_enhance result: 'NoneType' object has no attribute 'save'
• Test 2 error before final result image generated.
  (ERROR) [shortcodes\stable_diffusion\zoom_enhance.py] Could not append zoom_enhance result: 'NoneType' object has no attribute 'copy'
• Test 3 final result looks correct, but txt2img output looks like original (wasn't replaced)
• Test 4 final result looks correct, but txt2img output looks like original (wasn't replaced)
• Test 5 final result looks correct, but txt2img output looks like original (wasn't replaced)

[ThereforeGames]

Thank you, @Fenwich.

That at least narrows the problem down to the postprocess() routine - it's not receiving the final image for some reason.

I just pushed a small update with a couple extra sanity checks. Could you let me know if it makes a difference? You can disable the test flag or just set it to 0.

I appreciate your patience with this. It's a tough thing to debug as I still haven't been able to reproduce the issue on any of my devices.

[Fenwich]

I appreciate your patience working with us on this and fast responses!

Unfortunately, it looks like that didn't fix anything. I still see the original image in my txt2img-images folder (for reference, mine is at "D:\Stable Diffusion\stable-diffusion-webui\outputs\txt2img-images\2023-04-29\00000-1.png").

I see the updated face in my img2img-images folder, "D:\Stable Diffusion\stable-diffusion-webui\outputs\img2img-images\2023-04-29\00000-1.png".

And, I still see the zoom_enhance_final_result created successfully in the stable-diffusion-webui folder.

[ThereforeGames]

Hmm. Alright, I'm attaching another file you can try - replace the existing extensions/unprompted/scripts/unprompted.py with this, then restart the WebUI.

If it still doesn't work, look for an error in your console to the effect of, after_processed var is... or error during test.

Also, I have a couple more diagnostic questions:

• What browser are you using? I've tested this in Brave and Firefox.
• With ControlNet, are you able to see detectmaps in your output window? They're added to the output window in a similar way as zoom_enhance.

Thanks!

unprompted.zip

[Fenwich]

I'm using Chrome.

I have installed ControlNet but I haven't really done any playing with it, so I'm not sure how much help I can be with what works or doesn't work on that front.

Still didn't work, but I did get "after_processed var is: [<PIL.Image.Image image mode=RGB size=512x512 at 0x22512E67850>]"

[boythee]

i am using firefox. control net is disabled. testing in txt2img. sometimes it writes the files into webui, sometimes it doesn't. the most perplexing thing is that i tried the line
[after][zoom_enhance replacement="walter white face" debug test=0][/after]an amazing full body photo of walter white
switching it to buffy summer, then abraham lincoln, then marilyn monroe. now, it is mixing the images together, in the webui folder. it doesn't appear to be having an effect in the stable diffusion interface itself.

File "E:\SD2023_05\stable-diffusion-webui\modules\scripts.py", line 436, in postprocess
script.postprocess(p, processed, *script_args)
File "E:\SD2023_05\stable-diffusion-webui\extensions\unprompted\scripts\unprompted.py", line 531, in postprocess
val = Unprompted.shortcode_objects[i].after(p,processed)
File "E:\SD2023_05\stable-diffusion-webui\extensions\unprompted/shortcodes\basic\after.py", line 36, in after
self.Unprompted.process_string(content,"after")
File "E:\SD2023_05\stable-diffusion-webui\extensions\unprompted\lib_unprompted\shared.py", line 96, in process_string
string = self.shortcode_parser.parse(self.sanitize_pre(string,self.Config.syntax.sanitize_before),context)
File "E:\SD2023_05\stable-diffusion-webui\extensions\unprompted\lib_unprompted\shortcodes.py", line 245, in parse
return stack.pop().render(context)
File "E:\SD2023_05\stable-diffusion-webui\extensions\unprompted\lib_unprompted\shortcodes.py", line 59, in render
return ''.join(child.render(context) for child in self.children)
File "E:\SD2023_05\stable-diffusion-webui\extensions\unprompted\lib_unprompted\shortcodes.py", line 59, in
return ''.join(child.render(context) for child in self.children)
File "E:\SD2023_05\stable-diffusion-webui\extensions\unprompted\lib_unprompted\shortcodes.py", line 124, in render
raise ShortcodeRenderingError(msg) from ex
lib_unprompted.shortcodes.ShortcodeRenderingError: An exception was raised while rendering the 'zoom_enhance' shortcode in line 1.

by the way, an earlier version works fine that i have in another folder with an earlier version of stable diffusion. that one has the original control net. the outputs go to the temp folder, but they look good.

this is an awesome extension. i love the modular nature of it. i'm really hoping to get great hands and faces with this, unfortunately i am using the batch on control net, so i need to use the current one.

thanks for all your work on this.

EDIT: still working on it. it definitely is remembering previous terms. i will try that file replacement you mentioned earlier

EDIT: okay, i switched to the new file you mentioned 2 posts above.
[after][zoom_enhance replacement="buffy summers face"][/after]
RAW, digital style, girl, 20yo [marilyn monroe : buffy summers : 0.7], ((high detailed skin, skin details,)), sunny, bright, vivid, wearing a sexy revealing lace dress on a sci fi futuristic city style of logan's run 8k uhd, high quality, film grain, Fujifilm XT3 "
it is now printing 4 files in the output of the sd interface, but there is no switching of the face happening. no file appears in the webui folder, or the temp folder.
here's the output:
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
before_process_batch : RAW, digital style, girl, 20yo [marilyn monroe : buffy summers : 0.7], ((high detailed skin, skin details,)), sunny, bright, vivid, wearing a sexy revealing lace dress on a sci fi futuristic city style of logan's run 8k uhd, high quality, film grain, Fujifilm XT3 "
['RAW, digital style, girl, 20yo [marilyn monroe : buffy summers : 0.7], ((high detailed skin, skin details,)), sunny, bright, vivid, wearing a sexy revealing lace dress on a sci fi futuristic city style of logan's run 8k uhd, high quality, film grain, Fujifilm XT3 "']
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00, 3.57it/s]
Euler arogress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:02<00:00, 6.31it/s]
100%|████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:01<00:00, 2.56it/s]
Total progress: 25it [00:11, 2.15it/s]
Euler arogress: 25it [00:11, 1.45it/s]
100%|████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 9.30it/s]
Total progress: 100%|████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 6.28it/s]
after_processed var is: [<PIL.Image.Image image mode=RGB size=512x512 at 0x1D983E6B8E0>, <PIL.Image.Image image mode=RGB size=512x512 at 0x1D9838973A0>]

[boythee]

GOOD NEWS, EVERYONE!
the file you provided is working fine so far, without control net running. the result is printed to the temp directory and the sd interface, not to the image output folder. but it works! so all we need is for the output to save in the output directory. faces are being saved to the imgtoimg folder, which maybe? might explain the "memory" issue i noticed earlier. i will test with control net next.
here's my output:
before_process_batch : an amazing full body photo of walter white
['an amazing full body photo of walter white']
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:02<00:00, 8.15it/s]
Euler arogress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:02<00:00, 8.98it/s]
100%|████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 9.09it/s]
Total progress: 26it [00:04, 6.29it/s]
Euler arogress: 26it [00:04, 6.32it/s]
100%|████████████████████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 9.06it/s]
Total progress: 100%|████████████████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 6.62it/s]
after_processed var is: [<PIL.Image.Image image mode=RGB size=512x512 at 0x22E839107C0>, <PIL.Image.Image image mode=RGB size=512x512 at 0x22E83913610>, <PIL.Image.Image image mode=RGB size=512x512 at 0x22E839AEF80>, <PIL.Image.Image image mode=RGB size=512x512 at 0x22F34F0F5B0>]
etc.

[boythee]

It works singly with control net, saving the correct image to the temp folder. it does not work with control net batch. in neither circumstance does it save the correct image to the image output folder.

[Dawgmastah]

Newest version actually breaks more things (with newest auto version as well)
Extension (with normal name) now crashes if used with controlnet or with adetailer, even if both are disabled
If I uninstall them both, still dont get the correct modified image anywhere, it runs and generates face in the out folder, but never does the stitching (neither in tmp folder)

No errors in console

[Dawgmastah]

Last time extension was able to provide stitched image (altough now its doing the masking wrong, faces dont seem to align no clue why)
Was
https://github.com/ThereforeGames/unprompted/tree/42f4aadb17592d08170df270502bea1ccdaa1523

[boythee]

thanks Dawgmastah for that info, i will give that a shot. the brand new pull i got today is working great with everything, but i haven't tried the stitching yet. it is working with controlnet, the block code logic is solid.

[Dawgmastah]

thanks Dawgmastah for that info, i will give that a shot. the brand new pull i got today is working great with everything, but i haven't tried the stitching yet. it is working with controlnet, the block code logic is solid.

Oh wow, wish it worked with me :(

[Poliveris]

Yup can't get this extension to work anymore; trying with versions 7.8 - 7.9.1 as the new upscaler method is horrible

[ratsark]

I just tried a full reinstall with only the unprompted extension, and still the issue persists on my machine (Ubuntu under WSL).

@ThereforeGames is there any other diagnostic information that would be helpful? I'll try to find some time to debug it in more depth.

[ratsark]

Digging into the code I noticed the test flag, which I somehow missed in the discussion above.

Tests 3, 4, and 5 all work for me, and show the edited result in the window.