feat: add inpaint mask generation functionality #2032

mashb1t · 2024-01-24T01:43:44Z

Implements #1987
Uses rembg, see https://github.com/danielgatis/rembg

EDIT: Shopify just announced this exact same feature in https://www.shopify.com/magic, see also https://www.theverge.com/24056980/shopify-generative-ai-image-editing-search

This feature allows users to upload an inpaint image and automatically generate the mask for it based on rembg models, so they don't need to switch tools for this.

Process:

enable mask in advanced inpainting settings
upload inpaint image
select mask generation model (rembg)
click generate mask from image button

Possible extension (in general, not necessarily regarding this feature): add mask handling for image mask gradio canvas, so users can add an additive mask to the mask image OR disable mask canvas for this image.

uses rembg, see https://github.com/danielgatis/rembg

diaolulu1 · 2024-01-24T13:29:25Z

Where should I place these mask generation models?

mashb1t · 2024-01-24T15:04:20Z

@diaolulu1 the models should automatically be downloaded when selecting the model and using it the first time (see screenshot below, default mask generation model is isnet-general-use.

rayronvictor · 2024-01-24T16:59:25Z

I'm getting an error when using sam as mask generation model

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "/content/Fooocus/webui.py", line 205, in generate_mask
    return generate_mask_from_image(image, mask_model)
  File "/content/Fooocus/extras/inpaint_mask.py", line 11, in generate_mask_from_image
    return remove(
  File "/usr/local/lib/python3.10/dist-packages/rembg/bg.py", line 257, in remove
    masks = session.predict(img, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/rembg/sessions/sam.py", line 169, in predict
    validate(instance=prompt, schema=schema)
  File "/usr/local/lib/python3.10/dist-packages/jsonschema/validators.py", line 1306, in validate
    raise error
jsonschema.exceptions.ValidationError: '{}' is not of type 'array'

Failed validating 'type' in schema:
    {'items': {'properties': {'data': {'items': {'type': 'number'},
                                       'type': 'array'},
                              'label': {'type': 'integer'},
                              'type': {'type': 'string'}},
               'type': 'object'},
     'type': 'array'}

On instance:
    '{}'

diaolulu1 · 2024-01-24T17:15:33Z

I'm getting an error when using sam as mask generation model

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 488, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1431, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1103, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/gradio/utils.py", line 707, in wrapper
    response = f(*args, **kwargs)
  File "/content/Fooocus/webui.py", line 205, in generate_mask
    return generate_mask_from_image(image, mask_model)
  File "/content/Fooocus/extras/inpaint_mask.py", line 11, in generate_mask_from_image
    return remove(
  File "/usr/local/lib/python3.10/dist-packages/rembg/bg.py", line 257, in remove
    masks = session.predict(img, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/rembg/sessions/sam.py", line 169, in predict
    validate(instance=prompt, schema=schema)
  File "/usr/local/lib/python3.10/dist-packages/jsonschema/validators.py", line 1306, in validate
    raise error
jsonschema.exceptions.ValidationError: '{}' is not of type 'array'

Failed validating 'type' in schema:
    {'items': {'properties': {'data': {'items': {'type': 'number'},
                                       'type': 'array'},
                              'label': {'type': 'integer'},
                              'type': {'type': 'string'}},
               'type': 'object'},
     'type': 'array'}

On instance:
    '{}'

Me too😭

mashb1t · 2024-01-24T18:35:25Z

Yes, segment map models will return segments (duh) and not one map, so sadly i'll have to remove them later today.

rayronvictor · 2024-01-24T23:42:14Z

@mashb1t

Gravacao.de.Tela.2024-01-24.as.20.39.32.mp4

rayronvictor · 2024-01-25T00:28:06Z

PR mashb1t#5

…ration Add options to u2net_cloth_seg, add u2net_human_seg again

rayronvictor · 2024-01-25T18:53:39Z

@mashb1t, how about SAM with prompt?

Gravacao.de.Tela.2024-01-25.as.15.49.54.mp4

mashb1t · 2024-01-25T19:15:58Z

@rayronvictor sure, looks really good! I'd propose to put the advanced settings in developer debug mode inpainting, but don't see why we shouldn't use this! Feel free to provide your code in an additional PR to the fork branch (like last time), so we can merge it here.

…ation' into feature/add-inpaint-mask-generation # Conflicts: # webui.py

rayronvictor · 2024-01-25T19:52:05Z

PR created mashb1t#6

automatically unload model when not needed anymore

…-inpaint-mask-generation

this is overridden anyways

JulienZLQ · 2024-01-26T03:15:35Z

When I start, it prompts
CMD Failed requirements: install -r "requirements_versions.txt"

and when I use the inpaint mask generation, just error

Traceback (most recent call last):
File "D:\Fooocus\python_embeded\lib\site-packages\gradio\routes.py", line 488, in run_predict
output = await app.get_blocks().process_api(
File "D:\Fooocus\python_embeded\lib\site-packages\gradio\blocks.py", line 1431, in process_api
result = await self.call_function(
File "D:\Fooocus\python_embeded\lib\site-packages\gradio\blocks.py", line 1103, in call_function
prediction = await anyio.to_thread.run_sync(
File "D:\Fooocus\python_embeded\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "D:\Fooocus\python_embeded\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "D:\Fooocus\python_embeded\lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
File "D:\Fooocus\python_embeded\lib\site-packages\gradio\utils.py", line 707, in wrapper
response = f(*args, **kwargs)
File "D:\Fooocus\FooocusNov\webui.py", line 245, in generate_mask
from extras.inpaint_mask import generate_mask_from_image
File "D:\Fooocus\FooocusNov\extras\inpaint_mask.py", line 4, in
from rembg import remove, new_session
ModuleNotFoundError: No module named 'rembg'

mashb1t · 2024-01-26T03:20:07Z

As you can see rembg is in the requirements_versions.txt, see https://github.com/lllyasviel/Fooocus/pull/2032/files#diff-2284b86f286dc7e0ea4bd09a0ec20c78fbb17d6724d7f0053e78428d0715bbb1R19

Please make sure to manually install the dependencies if you're not using the automated way. Is there a specific error in the install?

JulienZLQ · 2024-01-26T03:26:19Z

As you can see rembg is in the requirements_versions.txt, see https://github.com/lllyasviel/Fooocus/pull/2032/files#diff-2284b86f286dc7e0ea4bd09a0ec20c78fbb17d6724d7f0053e78428d0715bbb1R19

Please make sure to manually install the dependencies if you're not using the automated way. Is there a specific error in the install?

I just followed the instructions in this link to manually modify and add them. rembg is sure in the requirements_versions.txt

JulienZLQ · 2024-01-26T03:40:50Z

[System ARGV] ['FooocusNov\entry_with_update.py']
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Fooocus version: 2.1.862
Installing requirements
Couldn't install requirements.
Command: "D:\Fooocus\python_embeded\python.exe" -m pip install -r "requirements_versions.txt" --prefer-binary
Error code: 1
stdout: Requirement already satisfied: torchsde==0.2.5 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 1)) (0.2.5)
Requirement already satisfied: einops==0.4.1 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 2)) (0.4.1)
Requirement already satisfied: transformers==4.30.2 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 3)) (4.30.2)
Requirement already satisfied: safetensors==0.3.1 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 4)) (0.3.1)
Requirement already satisfied: accelerate==0.21.0 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 5)) (0.21.0)
Requirement already satisfied: pyyaml==6.0 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 6)) (6.0)
Requirement already satisfied: Pillow==9.2.0 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 7)) (9.2.0)
Requirement already satisfied: scipy==1.9.3 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 8)) (1.9.3)
Requirement already satisfied: tqdm==4.64.1 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 9)) (4.64.1)
Requirement already satisfied: psutil==5.9.5 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 10)) (5.9.5)
Requirement already satisfied: pytorch_lightning==1.9.4 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 11)) (1.9.4)
Requirement already satisfied: omegaconf==2.2.3 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 12)) (2.2.3)
Requirement already satisfied: gradio==3.41.2 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 13)) (3.41.2)
Requirement already satisfied: pygit2==1.12.2 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 14)) (1.12.2)
Requirement already satisfied: opencv-contrib-python==4.8.0.74 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 15)) (4.8.0.74)
Requirement already satisfied: httpx==0.24.1 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 16)) (0.24.1)
Requirement already satisfied: onnxruntime==1.16.3 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 17)) (1.16.3)
Requirement already satisfied: timm==0.9.2 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 18)) (0.9.2)
Collecting rembg==2.0.53 (from -r requirements_versions.txt (line 19))
Obtaining dependency information for rembg==2.0.53 from https://files.pythonhosted.org/packages/55/6e/5a336d1308105fbe2a9738e7b99e79549628e80595d62142c4334e319b67/rembg-2.0.53-py3-none-any.whl.metadata
Using cached rembg-2.0.53-py3-none-any.whl.metadata (14 kB)
Collecting groundingdino-py==0.4.0 (from -r requirements_versions.txt (line 20))
Using cached groundingdino-py-0.4.0.tar.gz (82 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'error'

stderr: error: subprocess-exited-with-error

python setup.py egg_info did not run successfully.
exit code: 1

thiner · 2024-01-26T07:18:46Z

Why does the seg models are saved to a different folder?

Downloading data from 'https://github.com/danielgatis/rembg/releases/download/v0.0.0/u2netp.onnx' to file '/root/.u2net/u2netp.onnx'.

This is really inconvenient for Fooocus running in Docker or remote server. Is it possible move the models to path_inpaint?

mashb1t · 2024-01-26T09:55:03Z

@thiner it seems like we can set and override the env var U2NET_HOME in https://github.com/danielgatis/rembg/blob/49d1686f65b71b72df1c3b420551406d13074426/rembg/sessions/base.py#L78, will test this in the evening (in ~6-8h).
@rayronvictor Feel free to already adjust and provide your feedback!

mashb1t · 2024-03-03T20:52:27Z

@ChineseNorris i've double checked, gradio does either allow

uploading a mask and inpainting a single color (white brush with opacity 1) using sketch and gradio.ImageMask

OR

a full canvas editor with multi colors with a base canvas and no file upload (technically possible using color-sketch with gradio.ImagePaint, see https://www.gradio.app/3.50.2/docs/image, but heavy bugs / display errors in 3.x, causing the WS connection to break and image not to be displayed after mask generation / upload).

To keep the mask upload functionality is IMHO mandatory and there currently is nobody working on the Gradio 3.x => 4.x upgrade, so this feature won't be available anytime soon.
You're welcome to try it yourself, maybe you have a solution!

ChineseNorris · 2024-03-03T22:13:34Z

@mashb1t I see, I agree with you that this functionality is mandatory and we should move the "correction" of the mask in another thread, I'll see what I can do but it's going to take some time since I'm not familiar with Gradio at all lol

Amit30swgoh · 2024-03-08T08:33:01Z

could you please add add inpaint mask generationto other branches where the waiting time to start are faster compare to Main branch. ty so much. also is it hard to implement it if id like the add inpaint mask generation to have it on my personal branch?

ChineseNorris · 2024-03-15T13:50:18Z

@ChineseNorris i've double checked, gradio does either allow
1. uploading a mask and inpainting a single color (white brush with opacity 1) using sketch and gradio.ImageMask
OR
2. a full canvas editor with multi colors with a base canvas and no file upload (technically possible using color-sketch with gradio.ImagePaint, see https://www.gradio.app/3.50.2/docs/image, but heavy bugs / display errors in 3.x, causing the WS connection to break and image not to be displayed after mask generation / upload).
To keep the mask upload functionality is IMHO mandatory and there currently is nobody working on the Gradio 3.x => 4.x upgrade, so this feature won't be available anytime soon. You're welcome to try it yourself, maybe you have a solution!

I've found a solution for that but I can't push the commit to the repository, maybe you can help me out? @mashb1t

mashb1t · 2024-03-15T15:43:12Z

@ChineseNorris sure, you can make a fork of my fork, commit and make a PR to this branch on my fork: mashb1t:feature/add-inpaint-mask-generation

ChineseNorris · 2024-03-17T23:40:50Z

@ChineseNorris sure, you can make a fork of my fork, commit and make a PR to this branch on my fork: mashb1t:feature/add-inpaint-mask-generation

I'm trying to PR but it keeps saying that I don't have access to this repository
Can't create a new pull request: Push failed: remote: Permission to mashb1t/Fooocus.git denied to ChineseNorris. unable to access 'https://github.com/mashb1t/Fooocus/': The requested URL returned error: 403

mashb1t · 2024-03-18T06:26:42Z

@ChineseNorris you can't directly push to my repository, please make a fork of it. Alternatively you can also upload the changed files as a zip

ChineseNorris · 2024-03-20T17:12:25Z

@ChineseNorris you can't directly push to my repository, please make a fork of it. Alternatively you can also upload the changed files as a zip

I've made a PR on your fork, please check it when you have time.

I also noticed another "issue" with the automatic mask generation, where the elements inside the mask are changed in the borders, mostly parts where the mask end. Is it possible to tweak this setting in order to alter more or less the part that is unmasked?

mashb1t · 2024-03-20T17:27:33Z

@ChineseNorris this can be modified, check out the dilate or erode slider in developer debug mode right above the checkboxes for enabling the mask.

ChineseNorris · 2024-03-26T23:22:11Z

I've updated to the latest version but it seems that I'm having issues with the last commit, what am I missing? I got assertion error

`

Traceback (most recent call last):
File "/Fooocus/modules/async_worker.py", line 921, in worker
handler(task)
File "/anaconda3/envs/fooocus/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/anaconda3/envs/fooocus/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Fooocus/modules/async_worker.py", line 365, in handler
inpaint_head_model_path, inpaint_patch_model_path = modules.config.downloading_inpaint_models(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Fooocus/modules/config.py", line 580, in downloading_inpaint_models
assert v in modules.flags.inpaint_engine_versions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
`

@mashb1t

mashb1t · 2024-03-26T23:45:35Z

@ChineseNorris somehow you're not using an inpaint engine in the array of modules.flags.inpaint_engine_versions, please change accordingly or reset the input to the available values.

elnoyve · 2024-04-12T03:58:22Z

C:\Fooocus\Fooocus_win64_2-1-831>.\python_embeded\python.exe -s Fooocus\entry_with_update.py --preset realistic --theme dark
Already up-to-date
Update succeeded.
[System ARGV] ['Fooocus\entry_with_update.py', '--preset', 'realistic', '--theme', 'dark']
Loaded preset: C:\Fooocus\Fooocus_win64_2-1-831\Fooocus\presets\realistic.json
Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]
Fooocus version: 2.2.0
Error checking version for groundingdino-py: No package metadata was found for groundingdino-py
Installing requirements
Couldn't install requirements.
Command: "C:\Fooocus\Fooocus_win64_2-1-831\python_embeded\python.exe" -m pip install -r "requirements_versions.txt" --prefer-binary
Error code: 1
stdout: Requirement already satisfied: torchsde==0.2.5 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 1)) (0.2.5)
Requirement already satisfied: einops==0.4.1 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 2)) (0.4.1)
Requirement already satisfied: transformers==4.30.2 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 3)) (4.30.2)
Requirement already satisfied: safetensors==0.3.1 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 4)) (0.3.1)
Requirement already satisfied: accelerate==0.21.0 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 5)) (0.21.0)
Requirement already satisfied: pyyaml==6.0 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 6)) (6.0)
Requirement already satisfied: Pillow==9.2.0 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 7)) (9.2.0)
Requirement already satisfied: scipy==1.9.3 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 8)) (1.9.3)
Requirement already satisfied: tqdm==4.64.1 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 9)) (4.64.1)
Requirement already satisfied: psutil==5.9.5 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 10)) (5.9.5)
Requirement already satisfied: pytorch_lightning==1.9.4 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 11)) (1.9.4)
Requirement already satisfied: omegaconf==2.2.3 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 12)) (2.2.3)
Requirement already satisfied: gradio==3.41.2 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 13)) (3.41.2)
Requirement already satisfied: pygit2==1.12.2 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 14)) (1.12.2)
Requirement already satisfied: opencv-contrib-python==4.8.0.74 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 15)) (4.8.0.74)
Requirement already satisfied: httpx==0.24.1 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 16)) (0.24.1)
Requirement already satisfied: onnxruntime==1.16.3 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 17)) (1.16.3)
Requirement already satisfied: timm==0.9.2 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 18)) (0.9.2)
Requirement already satisfied: rembg==2.0.53 in c:\fooocus\fooocus_win64_2-1-831\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 19)) (2.0.53)
Collecting groundingdino-py==0.4.0 (from -r requirements_versions.txt (line 20))
Using cached groundingdino-py-0.4.0.tar.gz (82 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'error'

stderr: error: subprocess-exited-with-error

python setup.py egg_info did not run successfully.
exit code: 1

[6 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "C:\Users\RYAN\AppData\Local\Temp\pip-install-69exdztx\groundingdino-py_06a57fcdf63d4f4db884246e025736f4\setup.py", line 41, in
readme = readme_file.read()
UnicodeDecodeError: 'cp949' codec can't decode byte 0xf0 in position 2876: illegal multibyte sequence
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

Encountered error while generating package metadata.

See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

CMD Failed requirements: install -r "requirements_versions.txt"

@mashb1t No matter what I do to fix this issue, I am getting the above messages, and inpain rembg does not work on my fooocus. Could youd help me fix this issue please? I have installed your latest file from this Git and paste those files into my fooocus directory.

elnoyve · 2024-04-12T04:35:24Z

[System ARGV] ['FooocusNov\entry_with_update.py'] Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] Fooocus version: 2.1.862 Installing requirements Couldn't install requirements. Command: "D:\Fooocus\python_embeded\python.exe" -m pip install -r "requirements_versions.txt" --prefer-binary Error code: 1 stdout: Requirement already satisfied: torchsde==0.2.5 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 1)) (0.2.5) Requirement already satisfied: einops==0.4.1 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 2)) (0.4.1) Requirement already satisfied: transformers==4.30.2 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 3)) (4.30.2) Requirement already satisfied: safetensors==0.3.1 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 4)) (0.3.1) Requirement already satisfied: accelerate==0.21.0 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 5)) (0.21.0) Requirement already satisfied: pyyaml==6.0 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 6)) (6.0) Requirement already satisfied: Pillow==9.2.0 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 7)) (9.2.0) Requirement already satisfied: scipy==1.9.3 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 8)) (1.9.3) Requirement already satisfied: tqdm==4.64.1 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 9)) (4.64.1) Requirement already satisfied: psutil==5.9.5 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 10)) (5.9.5) Requirement already satisfied: pytorch_lightning==1.9.4 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 11)) (1.9.4) Requirement already satisfied: omegaconf==2.2.3 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 12)) (2.2.3) Requirement already satisfied: gradio==3.41.2 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 13)) (3.41.2) Requirement already satisfied: pygit2==1.12.2 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 14)) (1.12.2) Requirement already satisfied: opencv-contrib-python==4.8.0.74 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 15)) (4.8.0.74) Requirement already satisfied: httpx==0.24.1 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 16)) (0.24.1) Requirement already satisfied: onnxruntime==1.16.3 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 17)) (1.16.3) Requirement already satisfied: timm==0.9.2 in d:\fooocus\python_embeded\lib\site-packages (from -r requirements_versions.txt (line 18)) (0.9.2) Collecting rembg==2.0.53 (from -r requirements_versions.txt (line 19)) Obtaining dependency information for rembg==2.0.53 from https://files.pythonhosted.org/packages/55/6e/5a336d1308105fbe2a9738e7b99e79549628e80595d62142c4334e319b67/rembg-2.0.53-py3-none-any.whl.metadata Using cached rembg-2.0.53-py3-none-any.whl.metadata (14 kB) Collecting groundingdino-py==0.4.0 (from -r requirements_versions.txt (line 20)) Using cached groundingdino-py-0.4.0.tar.gz (82 kB) Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'error'

stderr: error: subprocess-exited-with-error

python setup.py egg_info did not run successfully. exit code: 1

User
I am having exactly the same issue. Did you find the way to fix this?

thiner · 2024-04-12T05:17:14Z

C:\Users\RYAN\AppData\Local\Temp\pip-install-69exdztx\groundingdino-py_06a57fcdf63d4f4db884246e025736f4\setup.py

you need to check line 41 of file C:\Users\RYAN\AppData\Local\Temp\pip-install-69exdztx\groundingdino-py_06a57fcdf63d4f4db884246e025736f4\setup.py, see what kind of special character is there which is not compatible with CP949 character set.
I tested this PR previously, it's working like a chime. A possible quick workaround may be changing your local language to English or use Unicode(utf-8 for instance) at runtime.

elnoyve · 2024-04-12T06:19:22Z

C:\Users\RYAN\AppData\Local\Temp\pip-install-69exdztx\groundingdino-py_06a57fcdf63d4f4db884246e025736f4\setup.py

you need to check line 41 of file C:\Users\RYAN\AppData\Local\Temp\pip-install-69exdztx\groundingdino-py_06a57fcdf63d4f4db884246e025736f4\setup.py, see what kind of special character is there which is not compatible with CP949 character set. I tested this PR previously, it's working like a chime. A possible quick workaround may be changing your local language to English or use Unicode(utf-8 for instance) at runtime.

Thank you for your advice. I actually tried to find 'setup.py' in the temp folder on my Windows 10 computer. However, since it is a temporary folder, I couldn't find that file. I tried converting every related file to UTF-8, but I failed to fix the issue. Is there any way I can find setup.py in any other places in the Fooocus folder?

thiner · 2024-04-12T07:21:38Z

groundingdino

from the path, it seems like the temporary file generated by pip while installing the dependency GroundingDINO. I found the setup.py file here: https://github.com/IDEA-Research/GroundingDINO/blob/main/setup.py.

Just checked the file changes in the PR, there is a new requirement added groundingdino-py==0.4.0. Maybe you can try to install this dependency manually.

elnoyve · 2024-04-12T10:37:19Z

groundingdino

from the path, it seems like the temporary file generated by pip while installing the dependency GroundingDINO. I found the setup.py file here: https://github.com/IDEA-Research/GroundingDINO/blob/main/setup.py.

Just checked the file changes in the PR, there is a new requirement added groundingdino-py==0.4.0. Maybe you can try to install this dependency manually.

Thank you very much. After I changed the computer's region setting to the U.S. and English, it works pretty well. Again, I really appreciate your help.

ChineseNorris · 2024-04-16T10:02:39Z

@mashb1t is it possible to merge this PR first and we keep up with the modifications? Otherwise this function is going to take forever

aolko · 2024-04-24T16:58:30Z

@mashb1t is it possible to merge this PR first and we keep up with the modifications? Otherwise this function is going to take forever

true, when it's going to be merged?

mashb1t · 2024-04-24T18:50:42Z

@ChineseNorris @aolko i originally planned to include this feature in 2.3.0, but decided against it so we could have a package update only release, which is also why the milestone for said updates has been created: https://github.com/lllyasviel/Fooocus/milestone/4

I currently have little time to push development of Fooocus further but will try my best in the time i have.
=> There is no ETA, but roughly in the next 2-3 weeks.

mashb1t · 2024-07-16T18:02:07Z

part of v2.5.0-rc1

feat: add inpaint mask generation functionality

db5512f

uses rembg, see https://github.com/danielgatis/rembg

mashb1t requested a review from lllyasviel as a code owner January 24, 2024 01:43

mashb1t added 2 commits January 24, 2024 19:38

fix: remove segmentation models from rembg model selection

d7d3464

fix: add correct flag validation for default_inpaint_mask_model config

dea4b6c

Add options to u2net_cloth_seg

f0fb978

mashb1t and others added 2 commits January 25, 2024 10:34

Merge pull request #5 from rayronvictor/feature/add-inpaint-mask-gene…

d1c55db

…ration Add options to u2net_cloth_seg, add u2net_human_seg again

refactor: rename cloth_category to inpaint_mask_cloth_category

d2b697f

rayronvictor added 2 commits January 25, 2024 16:45

Add SAM support

228a0aa

Merge remote-tracking branch 'upstream/feature/add-inpaint-mask-gener…

860ac91

…ation' into feature/add-inpaint-mask-generation # Conflicts: # webui.py

mashb1t added 5 commits January 26, 2024 02:05

feat: add model patching

90be73a

automatically unload model when not needed anymore

fix: remove unnecessary fp32 / fp16 handling

d515d0f

feat: optimize gradio element visibility changes and data provisioning

338004c

Merge branch 'feature/add-inpaint-mask-generation-2' into feature/add…

486c5b4

…-inpaint-mask-generation

chore: always use cpu as default device

62fe86f

this is overridden anyways

mashb1t added this to the 2.3.0 milestone Mar 3, 2024

mashb1t modified the milestones: 2.3.0 (draft), 2.4.0 (package updates) Mar 15, 2024

This was referenced Mar 31, 2024

[Feature Request]: Inpainting should allow the user to select the colour of the overlay #2660

Closed

[Feature Request]: Segmentation feature for inpaint feature #2720

Closed

mashb1t mentioned this pull request Jun 2, 2024

is it possible to add magic selection tool in inpaint [Feature Request]: #3085

Open

1 task

mashb1t modified the milestones: package updates, v2.5.0 Jul 16, 2024

mashb1t closed this Jul 16, 2024

feat: add inpaint mask generation functionality #2032

feat: add inpaint mask generation functionality #2032

Conversation

mashb1t commented Jan 24, 2024 • edited Loading

diaolulu1 commented Jan 24, 2024

mashb1t commented Jan 24, 2024

rayronvictor commented Jan 24, 2024

diaolulu1 commented Jan 24, 2024

mashb1t commented Jan 24, 2024

rayronvictor commented Jan 24, 2024

rayronvictor commented Jan 25, 2024

rayronvictor commented Jan 25, 2024

mashb1t commented Jan 25, 2024 • edited Loading

rayronvictor commented Jan 25, 2024

JulienZLQ commented Jan 26, 2024 • edited Loading

mashb1t commented Jan 26, 2024

JulienZLQ commented Jan 26, 2024

JulienZLQ commented Jan 26, 2024

thiner commented Jan 26, 2024 • edited Loading

mashb1t commented Jan 26, 2024

mashb1t commented Mar 3, 2024

ChineseNorris commented Mar 3, 2024 • edited Loading

Amit30swgoh commented Mar 8, 2024

ChineseNorris commented Mar 15, 2024

mashb1t commented Mar 15, 2024 • edited Loading

ChineseNorris commented Mar 17, 2024

mashb1t commented Mar 18, 2024

ChineseNorris commented Mar 20, 2024

mashb1t commented Mar 20, 2024

ChineseNorris commented Mar 26, 2024 • edited Loading

mashb1t commented Mar 26, 2024

elnoyve commented Apr 12, 2024 • edited Loading

CMD Failed requirements: install -r "requirements_versions.txt"

elnoyve commented Apr 12, 2024

thiner commented Apr 12, 2024

elnoyve commented Apr 12, 2024

thiner commented Apr 12, 2024 • edited Loading

elnoyve commented Apr 12, 2024

ChineseNorris commented Apr 16, 2024

aolko commented Apr 24, 2024

mashb1t commented Apr 24, 2024 • edited Loading

mashb1t commented Jul 16, 2024

mashb1t commented Jan 24, 2024 •

edited

Loading

mashb1t commented Jan 25, 2024 •

edited

Loading

JulienZLQ commented Jan 26, 2024 •

edited

Loading

thiner commented Jan 26, 2024 •

edited

Loading

ChineseNorris commented Mar 3, 2024 •

edited

Loading

mashb1t commented Mar 15, 2024 •

edited

Loading

ChineseNorris commented Mar 26, 2024 •

edited

Loading

elnoyve commented Apr 12, 2024 •

edited

Loading

thiner commented Apr 12, 2024 •

edited

Loading

mashb1t commented Apr 24, 2024 •

edited

Loading