Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

can't run success on inference.py, The error:IndexError: too many indices for array: array is 0-dimensional, but 3 were indexed #28

Closed
nicken opened this issue Jun 27, 2022 · 5 comments

Comments

@nicken
Copy link

nicken commented Jun 27, 2022

i try run the demo of inference.py when i get code
but i get this error info:

Traceback (most recent call last):
  File "inference.py", line 41, in <module>
    input = to_input(aligned_rgb_img)
  File "inference.py", line 24, in to_input
    brg_img = ((np_img[:,:,::-1] / 255.) - 0.5) / 0.5
IndexError: too many indices for array: array is 0-dimensional, but 3 were indexed

i find get error because the face is none on the face detection and don't known why i can't get face, i only run the code in new env and don't have any change on code

i think maybe the env have different, this is my env:

Package                 Version
----------------------- -----------
absl-py                 1.1.0
aiohttp                 3.8.1
aiosignal               1.2.0
async-timeout           4.0.2
asynctest               0.13.0
attrs                   21.4.0
bcolz                   1.2.1
cachetools              5.2.0
certifi                 2022.6.15
charset-normalizer      2.0.12
cycler                  0.11.0
fonttools               4.33.3
frozenlist              1.3.0
fsspec                  2022.5.0
future                  0.18.2
google-auth             2.8.0
google-auth-oauthlib    0.4.6
graphviz                0.8.4
grpcio                  1.47.0
idna                    3.3
imageio                 2.19.3
importlib-metadata      4.12.0
joblib                  1.1.0
kiwisolver              1.4.3
Markdown                3.3.7
matplotlib              3.5.2
menpo                   0.11.0
multidict               6.0.2
mxnet                   1.9.1
networkx                2.6.3
numpy                   1.21.6
oauthlib                3.2.0
opencv-python           4.6.0.66
packaging               21.3
pandas                  1.3.5
Pillow                  9.1.1
pip                     22.1.2
prettytable             3.3.0
protobuf                3.19.4
pyasn1                  0.4.8
pyasn1-modules          0.2.8
pyDeprecate             0.3.1
pyparsing               3.0.9
python-dateutil         2.8.2
pytorch-lightning       1.4.4
pytz                    2022.1
PyWavelets              1.3.0
PyYAML                  6.0
requests                2.28.0
requests-oauthlib       1.3.1
rsa                     4.8
scikit-image            0.19.3
scikit-learn            1.0.2
scipy                   1.7.3
setuptools              62.6.0
six                     1.16.0
tensorboard             2.9.1
tensorboard-data-server 0.6.1
tensorboard-plugin-wit  1.8.1
threadpoolctl           3.1.0
tifffile                2021.11.2
torch                   1.8.1+cu111
torchaudio              0.8.1
torchmetrics            0.6.0
torchvision             0.9.1+cu111
tqdm                    4.64.0
typing_extensions       4.2.0
urllib3                 1.26.9
wcwidth                 0.2.5
Werkzeug                2.1.2
wheel                   0.37.1
yarl                    1.7.2
zipp                    3.8.0
@mk-minchul
Copy link
Owner

Hi, it happens when the MTCNN face detector cannot find face properly.
MTCNN detects and aligns the image that is same as the training data. So this step is necessary to get the most performance.
One thing you may do is to tweak the parameter of MTCNN in line

self.thresholds = [0.6,0.7,0.9]

or use a more powerful face detector like https://github.com/serengil/retinaface (this one you have to check whether the alignment is same as MTCNN).
Another thing you could do if you need to force inference is just bypass the alignment step and resize the input to 112x112x3 and feed it to the model directly. Since AdaFace model is a convnet, it does not throw error even if the face is not aligned. Just make sure that the color channel is BGR.

@zhangguoliang-666
Copy link

Can I change the face detector code myself using retinaface instead of MTCNN if so where can I change it

@nicken
Copy link
Author

nicken commented Jul 18, 2022

Hi, it happens when the MTCNN face detector cannot find face properly. MTCNN detects and aligns the image that is same as the training data. So this step is necessary to get the most performance. One thing you may do is to tweak the parameter of MTCNN in line

self.thresholds = [0.6,0.7,0.9]

or use a more powerful face detector like https://github.com/serengil/retinaface (this one you have to check whether the alignment is same as MTCNN).
Another thing you could do if you need to force inference is just bypass the alignment step and resize the input to 112x112x3 and feed it to the model directly. Since AdaFace model is a convnet, it does not throw error even if the face is not aligned. Just make sure that the color channel is BGR.

OK thanks

@f-izzat
Copy link

f-izzat commented Apr 19, 2023

Following on this, how do we bypass the detection phase? I understand that theres a align function in the class

    def align(self, img):
        _, landmarks = self.detect_faces(img, self.min_face_size, self.thresholds, self.nms_thresholds, self.factor)
        facial5points = [[landmarks[0][j], landmarks[0][j + 5]] for j in range(5)]
        warped_face = warp_and_crop_face(np.array(img), facial5points, self.refrence, crop_size=self.crop_size)
        return Image.fromarray(warped_face)

However, it still goes through face detection, my case is that I already have cropped face images and so therefore the bounding box would be the whole input image

Edit:
I think a better question would be, what does the bounding_box array represent in MTCNN.detect_faces ? what is each dimension of the array referring to? i.e (x,y) pair coordinates?

@mk-minchul

@ANDRESHZ
Copy link

ANDRESHZ commented May 8, 2023

Hi @nicken, I used something that comment @mk-minchul, and I solved by the block of de "try: Except" into align.py file and call in there and evaluation model of retinaface or YOLO pretrained for example in prediction mode.

So into align.py at get_aligned_face function just change exception code to:

def get_aligned_face(image_path, rgb_pil_image=None):
    if rgb_pil_image is None:
        img = Image.open(image_path).convert('RGB')
    else:
        assert isinstance(rgb_pil_image, Image.Image), 'Face alignment module requires PIL image or path to the image'
        img = rgb_pil_image
    # find face
    try:
        bboxes, faces = mtcnn_model.align_multi(img, limit=1)
        face = faces[0]
    except Exception as e: #CHANGE CODE HERE!!!!
        width, height = img.size
        #Function to call YOLO or RetinaFace and obtain x, y , w and h [bbox in percentage % format ]
        results = model(img)  # this example use YOLO  created and instanced before using https://docs.ultralytics.com/tasks/detect/#val
        x, y, w, h= results2xywh(results) #send the biggest face identify at the image from YOLO format
        left = width*(x)
        top = height*(y)
        right = width*(w)
        bottom = height*(h)
        img2=img.crop((left, top, right, bottom))
        face=img2.resize((112,112))
        #print('Face detection Failed due to error.')
        #print(e)
        #face = None
    return face

You do not forget to reload the modules and import.

I hope to you help you @nicken. I have been trying to training with custom data an I found #75.
Did you found some solution? or even you got a training well passed script? THAT WOULD BE GREAT!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants