Merge branch-0.18 into branch-0.19

.github/labeler.yml

@@ -5,26 +5,33 @@
 python:
   - 'python/**'
   - 'notebooks/**'
+
+benchmarks:
   - 'benchmarks/**'
 
 doc:
   - 'docs/**'
   - '**/*.md'
+  - 'datasets/**'
+  - 'notebooks/**'
+  - '**/*.txt'
+  - '**/*.rst'
+  - '**/*.ipynb'
+  - '**/*.pdf'
+  - '**/*.png'
 
 datasets:
   - 'datasets/**'
 
 cuGraph:
   - 'cpp/**'
-
+  
 CMake:
   - '**/CMakeLists.txt'
   - '**/cmake/**'
-
-Ops:
-  - '.github/**'
-  - 'ci/**'
+
+gpuCI:
+   - 'ci/**'
+
+conda:
   - 'conda/**'
-  - '**/Dockerfile'
-  - '**/.dockerignore'
-  - 'docker/**'

CHANGELOG.md

@@ -6,15 +6,9 @@
 
 ## Bug Fixes
 
+# 0.18.0
 
-# cuGraph 0.18.0 (Date TBD)
-
-## New Features
-
-## Improvements
-
-## Bug Fixes
-- PR #1321 Fix benchmark script trap setup to come after the PATH variable update
+Please see https://github.com/rapidsai/cugraph/releases/tag/branch-0.18-latest for the latest changes to this development branch.
 
 # cuGraph 0.17.0 (10 Dec 2020)
 ## New Features

README.md

@@ -8,7 +8,7 @@ The [RAPIDS](https://rapids.ai) cuGraph library is a collection of GPU accelerat
 
 **NOTE:** For the latest stable [README.md](https://github.com/rapidsai/cugraph/blob/main/README.md) ensure you are on the latest branch.
 
-
+As an example, the following Python snippet loads graph data and computes PageRank:
 
 ```python
 import cugraph
@@ -30,6 +30,17 @@ for i in range(len(df_page)):
 		" PageRank is " + str(df_page['pagerank'].iloc[i]))
 ```
 
+## Getting cuGraph
+There are 3 ways to get cuGraph :
+1. [Quick start with Docker Repo](#quick)
+2. [Conda Installation](#conda)
+3. [Build from Source](#source)
+<br/><br/>
+
+---
+# Currently Supported Features
+As of Release 0.18 - including 0.18 nightly
+
 
 ## Supported Algorithms
 
@@ -40,6 +51,7 @@ for i in range(len(df_page)):
 |              | Betweenness Centrality                 | Single-GPU   |                     |
 |              | Edge Betweenness Centrality            | Single-GPU   |                     |
 | Community    |                                        |              |                     |
+|              | EgoNet                                 | Single-GPU   |                     |
 |              | Leiden                                 | Single-GPU   |                     |
 |              | Louvain                                | Multi-GPU    |                     |
 |              | Ensemble Clustering for Graphs         | Single-GPU   |                     |
@@ -67,7 +79,7 @@ for i in range(len(df_page)):
 |              | Weighted Jaccard Similarity            | Single-GPU   |                     |
 |              | Overlap Similarity                     | Single-GPU   |                     |
 | Traversal    |                                        |              |                     |
-|              | Breadth First Search (BFS)             | Multi-GPU    |                     |
+|              | Breadth First Search (BFS)             | Multi-GPU    | with cutoff support |
 |              | Single Source Shortest Path (SSSP)     | Multi-GPU    |                     |
 | Structure    |                                        |              |                     |
 |              | Renumbering                            | Single-GPU   | multiple columns, any data type  |
@@ -76,7 +88,6 @@ for i in range(len(df_page)):
 |              | Hungarian Algorithm                    | Single-GPU   |                     |
 |              | Minimum Spanning Tree                  | Single-GPU   |                     |
 |              | Maximum Spanning Tree                  | Single-GPU   |                     |
-
 |  |  |
 
 </br></br>
@@ -85,13 +96,13 @@ for i in range(len(df_page)):
 | --------------- | --------------------------------------------------- |
 | Graph           | An undirected Graph                                 |
 | DiGraph         | A Directed Graph                                    |
-| _Multigraph_      | _coming in 0.18_                                      |
-| _MultiDigraph_    | _coming in 0.18_                                      |
+| Multigraph      | A Graph with multiple edges between a vertex pair   |
+| MultiDigraph    | A Directed Graph with multiple edges between a vertex pair   |
 |  |  |
 
 </br></br>
 ## Supported Data Types
-cuGraph supports the creation of a graph several data types:
+cuGraph supports graph creation with Source and Destination being expressed as:
 * cuDF DataFrame
 * Pandas DataFrame
 
@@ -125,22 +136,14 @@ The amount of memory required is dependent on the graph structure and the analyt
 
 The use of managed memory for oversubscription can also be used to exceed the above memory limitations.  See the recent blog on _Tackling Large Graphs with RAPIDS cuGraph and CUDA Unified Memory on GPUs_:  https://medium.com/rapids-ai/tackling-large-graphs-with-rapids-cugraph-and-unified-virtual-memory-b5b69a065d4
 
+</br></br>
 
-## Getting cuGraph
-### Intro
-There are 3 ways to get cuGraph :
-1. [Quick start with Docker Demo Repo](#quick)
-2. [Conda Installation](#conda)
-3. [Build from Source](#source)
-
-
-
-
+---
 ## Quick Start <a name="quick"></a>
-Please see the [Demo Docker Repository](https://hub.docker.com/r/rapidsai/rapidsai/), choosing a tag based on the NVIDIA CUDA version you’re running. This provides a ready to run Docker container with example notebooks and data, showcasing how you can utilize all of the RAPIDS libraries: cuDF, cuML, and cuGraph.
+Please see the [Docker Repository](https://hub.docker.com/r/rapidsai/rapidsai/), choosing a tag based on the NVIDIA CUDA version you’re running. This provides a ready to run Docker container with example notebooks and data, showcasing how you can utilize all of the RAPIDS libraries: cuDF, cuML, and cuGraph.
 
 
-### Conda <a name="conda"></a>
+## Conda <a name="conda"></a>
 It is easy to install cuGraph using conda. You can get a minimal conda installation with [Miniconda](https://conda.io/miniconda.html) or get the full installation with [Anaconda](https://www.anaconda.com/download).
 
 Install and update cuGraph using the conda command:
@@ -160,7 +163,7 @@ conda install -c nvidia -c rapidsai -c numba -c conda-forge -c defaults cugraph
 Note: This conda installation only applies to Linux and Python versions 3.7/3.8.
 
 
-### Build from Source and Contributing <a name="source"></a>
+## Build from Source and Contributing <a name="source"></a>
 
 Please see our [guide for building cuGraph from source](SOURCEBUILD.md)</pr>
 

SOURCEBUILD.md

@@ -9,7 +9,7 @@ The cuGraph package include both a C/C++ CUDA portion and a python portion.  Bot
 __Compiler__:
 * `gcc`         version 5.4+
 * `nvcc`        version 10.0+
-* `cmake`       version 3.12+
+* `cmake`       version 3.18+
 
 __CUDA:__
 * CUDA 10.1+

ci/gpu/notebook_list.py

@@ -0,0 +1,48 @@
+# Copyright (c) 2021, NVIDIA CORPORATION.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import re
+import sys
+import glob
+
+from numba import cuda
+
+#
+# Not strictly true... however what we mean is
+# Pascal or earlier
+#
+pascal = False
+
+device = cuda.get_current_device()
+cc = getattr(device, 'COMPUTE_CAPABILITY')
+if (cc[0] < 7):
+    pascal = True
+
+for filename in glob.iglob('**/*.ipynb', recursive=True):
+    skip = False
+    for line in open(filename, 'r'):
+        if re.search('# Skip notebook test', line):
+            skip = True
+            print(f'SKIPPING {filename} (marked as skip)', file=sys.stderr)
+            break;
+        elif re.search('dask', line):
+            print(f'SKIPPING {filename} (suspected Dask usage, not currently automatable)', file=sys.stderr)
+            skip = True
+            break;
+        elif pascal and re.search('# Does not run on Pascal', line):
+            print(f'SKIPPING {filename} (does not run on Pascal)', file=sys.stderr)
+            skip = True
+            break;
+
+    if not skip:
+        print(filename)

ci/gpu/test-notebooks.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# Copyright (c) 2019-2020, NVIDIA CORPORATION.
+# Copyright (c) 2019-2021, NVIDIA CORPORATION.
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
@@ -20,11 +20,6 @@ LIBCUDF_KERNEL_CACHE_PATH=${WORKSPACE}/.jitcache
 cd ${NOTEBOOKS_DIR}
 TOPLEVEL_NB_FOLDERS=$(find . -name *.ipynb |cut -d'/' -f2|sort -u)
 
-# Add notebooks that should be skipped here
-# (space-separated list of filenames without paths)
-
-SKIPNBS="uvm.ipynb bfs_benchmark.ipynb louvain_benchmark.ipynb pagerank_benchmark.ipynb sssp_benchmark.ipynb release.ipynb nx_cugraph_bc_benchmarking.ipynb"
-
 ## Check env
 env
 
@@ -37,26 +32,14 @@ for folder in ${TOPLEVEL_NB_FOLDERS}; do
     echo "FOLDER: ${folder}"
     echo "========================================"
     cd ${NOTEBOOKS_DIR}/${folder}
-    for nb in $(find . -name "*.ipynb"); do
+    for nb in $(python ${WORKSPACE}/ci/gpu/notebook_list.py); do
         nbBasename=$(basename ${nb})
-        # Skip all NBs that use dask (in the code or even in their name)
-        if ((echo ${nb}|grep -qi dask) || \
-            (grep -q dask ${nb})); then
-            echo "--------------------------------------------------------------------------------"
-            echo "SKIPPING: ${nb} (suspected Dask usage, not currently automatable)"
-            echo "--------------------------------------------------------------------------------"
-        elif (echo " ${SKIPNBS} " | grep -q " ${nbBasename} "); then
-            echo "--------------------------------------------------------------------------------"
-            echo "SKIPPING: ${nb} (listed in skip list)"
-            echo "--------------------------------------------------------------------------------"
-        else
-            cd $(dirname ${nb})
-            nvidia-smi
-            ${NBTEST} ${nbBasename}
-            EXITCODE=$((EXITCODE | $?))
-            rm -rf ${LIBCUDF_KERNEL_CACHE_PATH}/*
-            cd ${NOTEBOOKS_DIR}/${folder}
-        fi
+        cd $(dirname ${nb})
+        nvidia-smi
+        ${NBTEST} ${nbBasename}
+        EXITCODE=$((EXITCODE | $?))
+        rm -rf ${LIBCUDF_KERNEL_CACHE_PATH}/*
+        cd ${NOTEBOOKS_DIR}/${folder}
     done
 done
 

ci/test.sh

@@ -1,5 +1,5 @@
 #!/bin/bash
-# Copyright (c) 2019-2020, NVIDIA CORPORATION.
+# Copyright (c) 2019-2021, NVIDIA CORPORATION.
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
@@ -64,7 +64,7 @@ else
     cd $WORKSPACE/ci/artifacts/cugraph/cpu/conda_work/cpp/build
 fi
 
-for gt in gtests/*; do
+for gt in tests/*_TEST; do
     test_name=$(basename $gt)
     echo "Running GoogleTest $test_name"
     ${gt} ${GTEST_FILTER} ${GTEST_ARGS}

conda/environments/cugraph_dev_cuda10.1.yml

@@ -23,7 +23,7 @@ dependencies:
 - cudatoolkit=10.1
 - clang=8.0.1
 - clang-tools=8.0.1
-- cmake>=3.12
+- cmake>=3.18
 - python>=3.6,<3.9
 - notebook>=0.5.0
 - boost
@@ -44,3 +44,6 @@ dependencies:
 - libcypher-parser
 - rapids-pytest-benchmark
 - doxygen
+- pytest-cov
+- gtest
+- gmock

conda/environments/cugraph_dev_cuda10.2.yml

@@ -23,7 +23,7 @@ dependencies:
 - cudatoolkit=10.2
 - clang=8.0.1
 - clang-tools=8.0.1
-- cmake>=3.12
+- cmake>=3.18
 - python>=3.6,<3.9
 - notebook>=0.5.0
 - boost
@@ -44,3 +44,6 @@ dependencies:
 - libcypher-parser
 - rapids-pytest-benchmark
 - doxygen
+- pytest-cov
+- gtest
+- gmock

conda/environments/cugraph_dev_cuda11.0.yml

@@ -23,7 +23,7 @@ dependencies:
 - cudatoolkit=11.0
 - clang=8.0.1
 - clang-tools=8.0.1
-- cmake>=3.12
+- cmake>=3.18
 - python>=3.6,<3.9
 - notebook>=0.5.0
 - boost
@@ -44,3 +44,6 @@ dependencies:
 - libcypher-parser
 - rapids-pytest-benchmark
 - doxygen
+- pytest-cov
+- gtest
+- gmock

conda/recipes/libcugraph/meta.yaml

@@ -1,4 +1,4 @@
-# Copyright (c) 2018, NVIDIA CORPORATION.
+# Copyright (c) 2018-2021, NVIDIA CORPORATION.
 
 # Usage:
 #   conda build -c nvidia -c rapidsai -c conda-forge -c defaults .
@@ -32,6 +32,8 @@ requirements:
     - nccl>=2.7
     - ucx-py {{ minor_version }}
     - ucx-proc=*=gpu
+    - gtest
+    - gmock
   run:
     - libcudf={{ minor_version }}
     - {{ pin_compatible('cudatoolkit', max_pin='x.x') }}