add test tutorial

docs/getting_started.md

@@ -125,23 +125,34 @@ Assume that you have already downloaded the checkpoints to the directory `checkp
         --eval mAP
     ```
 
-4. Test PSPNet with 8 GPUs, and evaluate the standard mIoU and cityscapes metric.
+4. Test PSPNet with 4 GPUs, and evaluate the standard mIoU and cityscapes metric.
 
     ```shell
     ./tools/dist_test.sh configs/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes.py \
         checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth \
-        8 --out results.pkl --eval mIoU cityscapes
+        4 --out results.pkl --eval mIoU cityscapes
     ```
 
-5. Test PSPNet on cityscapes test split with 8 GPUs, and generate the png files to be submit to the official evaluation server.
+5. Test PSPNet on cityscapes test split with 4 GPUs, and generate the png files to be submit to the official evaluation server.
+
+    First, add following to config file `configs/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes.py`,
+
+    ```python
+    data = dict(
+        test=dict(
+            img_dir='leftImg8bit/test',
+            ann_dir='gtFine/test'))
+    ```
+   Then run test.
 
     ```shell
     ./tools/dist_test.sh configs/pspnet/pspnet_r50-d8_512x1024_40k_cityscapes.py \
         checkpoints/pspnet_r50-d8_512x1024_40k_cityscapes_20200605_003338-2966598c.pth \
-        8 --format-only --options "imgfile_prefix=./pspnet_test_results"
+        4 --format-only --options "imgfile_prefix=./pspnet_test_results"
     ```
 
 You will get png files under `./pspnet_test_results` directory.
+You may run `zip -r results.zip pspnet_test_results/` and submit the zip file to [evaluation server](https://www.cityscapes-dataset.com/submit/).
 
 
 ### Image demo
@@ -205,8 +216,8 @@ By default we evaluate the model on the validation set after some iterations, yo
 evaluation = dict(interval=4000)  # This evaluate the model per 4000 iterations.
 ```
 
-**\*Important\***: The default learning rate in config files is for 8 GPUs and 1 img/gpu (batch size = 8x1 = 8).
-Equivalently, you may also use 4 GPUs and 2 imgs/gpu since all models using cross-GPU SyncBN.
+**\*Important\***: The default learning rate in config files is for 4 GPUs and 2 img/gpu (batch size = 4x2 = 8).
+Equivalently, you may also use 8 GPUs and 1 imgs/gpu since all models using cross-GPU SyncBN.
 
 ### Train with a single GPU
 

mmseg/datasets/cityscapes.py

@@ -61,6 +61,7 @@ def results2img(self, results, imgfile_prefix, to_label_id):
             list[str: str]: result txt files which contains corresponding
             semantic segmentation images.
         """
+        mmcv.mkdir_or_exist(imgfile_prefix)
         result_files = []
         prog_bar = mmcv.ProgressBar(len(self))
         for idx in range(len(self)):
@@ -135,9 +136,9 @@ def evaluate(self,
                 the prefix of filename, e.g., "a/b/prefix".
                 If results are evaluated with cityscapes protocol, it would be
                 the prefix of output png files. The output files would be
-                png images under folder "a/b/prefix/xxx/", where "xxx" is the
-                video name of cityscapes. If not specified, a temp file will
-                be created.
+                png images under folder "a/b/prefix/xxx.png", where "xxx" is
+                the image name of cityscapes. If not specified, a temp file
+                will be created for evaluation.
                 Default: None.
 
         Returns:

tools/dist_test.sh

@@ -4,7 +4,6 @@ CONFIG=$1
 CHECKPOINT=$2
 GPUS=$3
 PORT=${PORT:-29500}
-$CONFIG\/$GPUS/
 PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
 python -m torch.distributed.launch --nproc_per_node=$GPUS --master_port=$PORT \
     $(dirname "$0")/test.py $CONFIG $CHECKPOINT --launcher pytorch ${@:4}