the miou of test set

[renmmmmmm]

how can i get the value of miou from cityscape test set?when i change the code ,and run test.py ,i meet the bug?how i slove it ?
data = dict(
samples_per_gpu=2,
workers_per_gpu=2,
train=dict(
type=dataset_type,
data_root=data_root,
img_dir='leftImg8bit/train',
ann_dir='gtFine/train',
pipeline=train_pipeline),
val=dict(
type=dataset_type,
data_root=data_root,
img_dir='leftImg8bit/val',
ann_dir='gtFine/val',
pipeline=test_pipeline),
test=dict(
type=dataset_type,
data_root=data_root,
img_dir='leftImg8bit/test',
ann_dir='gtFine/test',
pipeline=test_pipeline))

---

          |  u   |  e   |  r   |  o   |  s   |  d   |  g   |  r   |  s   |  p   |  r   |  b   |  w   |  f   |  g   |  b   |  t   |  p   |  p   |  t   |  t   |  v   |  t   |  s   |  p   |  r   |  c   |  t   |  b   |  c   |  t   |  t   |  m   |  b   | Prior |

---

unlabeled | 0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.38   0.05   0.00   0.00   0.20   0.00   0.01   0.00   0.00   0.00   0.01   0.00   0.00   0.01   0.17   0.01   0.04   0.01   0.00   0.08   0.00   0.00   0.00   0.00   0.00   0.00   0.00  0.9290 

ego vehicle | 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.96 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.0448
rectification | 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.65 0.01 0.00 0.00 0.18 0.01 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.07 0.00 0.03 0.00 0.00 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.0111
out of roi | 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.39 0.03 0.00 0.00 0.23 0.01 0.01 0.00 0.00 0.00 0.01 0.00 0.00 0.00 0.17 0.01 0.08 0.00 0.00 0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.0151

---

classes IoU nIoU

road : nan nan
sidewalk : nan nan
building : nan nan
wall : nan nan
fence : nan nan
pole : nan nan
traffic light : nan nan
traffic sign : nan nan
vegetation : nan nan
terrain : nan nan
sky : nan nan
person : nan nan
rider : nan nan
car : nan nan
truck : nan nan
bus : nan nan
train : nan nan
motorcycle : nan nan
bicycle : nan nan

Score Average : nan nan

categories IoU nIoU

flat : nan nan
construction : nan nan
object : nan nan
nature : nan nan
sky : nan nan
human : nan nan
vehicle : nan nan

Score Average : nan nan

/home/rss/tmp/mmsegmentation-master/mmseg/core/evaluation/mean_iou.py:66: RuntimeWarning: invalid value encountered in double_scalars
all_acc = total_area_intersect.sum() / total_area_label.sum()
/home/rss/tmp/mmsegmentation-master/mmseg/core/evaluation/mean_iou.py:67: RuntimeWarning: invalid value encountered in true_divide
acc = total_area_intersect / total_area_label
/home/rss/tmp/mmsegmentation-master/mmseg/core/evaluation/mean_iou.py:68: RuntimeWarning: invalid value encountered in true_divide
iou = total_area_intersect / total_area_union
/home/rss/tmp/mmsegmentation-master/mmseg/datasets/custom.py:280: RuntimeWarning: Mean of empty slice
iou_str = '{:.2f}'.format(np.nanmean(iou) * 100)
/home/rss/tmp/mmsegmentation-master/mmseg/datasets/custom.py:281: RuntimeWarning: Mean of empty slice
acc_str = '{:.2f}'.format(np.nanmean(acc) * 100)
per class results:
Class IoU Acc
road nan nan
sidewalk nan nan
building nan nan
wall nan nan
fence nan nan
pole nan nan
traffic light nan nan
traffic sign nan nan
vegetation nan nan
terrain nan nan
sky nan nan
person nan nan
rider nan nan
car nan nan
truck nan nan
bus nan nan
train nan nan
motorcycle nan nan
bicycle nan nan
Summary:
Scope mIoU mAcc aAcc
global nan nan nan

/home/rss/tmp/mmsegmentation-master/mmseg/datasets/custom.py:287: RuntimeWarning: Mean of empty slice
eval_results['mIoU'] = np.nanmean(iou)
/home/rss/tmp/mmsegmentation-master/mmseg/datasets/custom.py:288: RuntimeWarning: Mean of empty slice
eval_results['mAcc'] = np.nanmean(acc)

[xvjiarui]

Hi @renmmmmmm
You need to submit it to the evaluation server. Please refer to the section 5 here for details.

[RoyZhDec]

Hi @renmmmmmm You need to submit it to the evaluation server. Please refer to the section 5 here for details.

The link is invalid, can you provide a new link?