Add docs on new apis, per-task routing, and rooter

NicolasFR · Oct 17, 2021 · 374fcfc · 374fcfc
1 parent e38d480
commit 374fcfc
Show file tree

Hide file tree

Showing 12 changed files with 795 additions and 50 deletions.
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
@@ -21,12 +21,18 @@ nav:
   - System dependencies: installation/deps.md
   - Machinery modules: installation/machineries.md
   - VM Creation: installation/vmcreation.md
+  - Network routing: installation/routing.md
 
 - Configuration:
   - System settings: configuration/system.md
   - Cuckoo configuration:
     - Cuckoo.yaml: configuration/cuckooconfs/cuckooyaml.md
     - Distributed.yaml: configuration/cuckooconfs/distributedyaml.md
+    - Web configuration:
+      - Web.yaml: configuration/cuckooconfs/webconfs/webyaml.md
+    - Node configuration:
+      - Routing.yaml: configuration/cuckooconfs/nodeconfs/routingyaml.md
+
 
 - Usage:
   - Web API:

diff --git a/docs/src/configuration/cuckooconfs/analysissettingsyaml.md b/docs/src/configuration/cuckooconfs/analysissettingsyaml.md
@@ -0,0 +1,57 @@
+
+## What is this config
+
+The `$CWD/conf/analysissettings.yaml` is a settings files that contains submission settings limits and defaults.
+
+### Default config example
+
+```yaml
+# Limits on settings. Submissions will be denied if they exceed any
+# of these limits.
+limits:
+  max_timeout: 300
+  max_priority: 999
+  # The maximum amount of platforms a submission can have.
+  max_platforms: 3
+
+# The default settings that will be used if these are not given.
+default:
+  # The timeout in seconds that will be used for each task.
+  timeout: 120
+  # The priority that will be used when in scheduling. A higher number
+  # means a higher priority.
+  priority: 1
+  # The route that will be used for each task. Automatic network routing
+  # must be enabled and rooter must be running for this feature to work.
+  # See cuckoo.yaml.
+  route:
+    # The route type: internet, vpn, or drop.
+    type: null
+    # Route options such as 'country: somecountry' for a VPN route.
+    options:
+
+# Settings used to determine the platform to use if no platforms
+# are provided on submission.
+platform:
+  # The OS versions of a platform that should be added to settings for an
+  # identified platform. These versions are also used for the multi_platform
+  # and fallback_platforms settings. Multiple versions will result in a
+  # task for each version. Each platform must at least have a list of 1 version.
+  versions:
+    windows:
+    - 10
+
+  # Which of the supported platforms determined during the identification stage
+  # should actually be used if a target can run on multiple platforms.
+  # This should be a list of platform names.
+  # The OS versions used are the ones specified in the 'versions' setting.
+  multi_platform:
+  - windows
+
+  # Which platform(s) should be used if no platforms the target can run on were
+  # identified and no platforms were provided on submission?
+  # This should be a list of platform names.
+  # The OS versions used are the ones specified in the 'versions' setting.
+  fallback_platforms:
+  - windows
+```
diff --git a/docs/src/configuration/cuckooconfs/cuckooyaml.md b/docs/src/configuration/cuckooconfs/cuckooyaml.md
@@ -1,7 +1,7 @@
 
 ## What is this config
 
-The cuckoo.yaml is a "general" configuration file. It contains main settings such as
+The `$CWD/conf/cuckoo.yaml` is a "general" configuration file. It contains main settings such as
 the chosen machinery module(s).
 
 ### Default config example
@@ -32,26 +32,24 @@ tcpdump:
   enabled: True
   path: /usr/sbin/tcpdump
 
-platform:
-  # The default platform to use when no platforms are supplied on submission and no required platform is
-  # identified during the identification stage.
-  default_platform:
-    platform: windows
-    os_version: null
-
-  # Which of the supported platforms determined during the identification stage should be used if a target
-  # can run on multiple platforms. A task will be created for each of the specified platforms if it is present in the
-  # determined platforms.
-  multi_platform:
-    - windows
+# Automatic per-task network routing. Before a task runs, the rooter
+# is asked to apply a submitted or default network route. Cuckoo rooter
+# must be running and its unix socket path must be configured here.
+network_routing:
+  enabled: False
+  # Cuckoo rooter socket path. Must be writable and readable for the user
+  # that runs Cuckoo.
+  rooter_socket: null
 
+platform:
   # Use the machine tags determined during the identification phase for machine selection.
   # Enabling this means your analyis machines have been configured with the tags
   # listed in conf/processing/identification.yaml
   autotag: False
 
 state_control:
-  # Cancel an analysis if the file was not identified during the identification stage.
+  # Cancel an analysis if the submitted file was not identified during the
+  # identification stage.
   cancel_unidentified: False
 
 processing:
@@ -66,7 +64,11 @@ processing:
     # Post workers are responsible for the processing of data collected during each task.
     post: 1
 
+# The Cuckoo 'long term storage' host that reported analyses should be moved to when
+# starting 'cuckoocleanup remotestorage'. This host must have an import controller
+# instance and the Cuckoo API running.
 remote_storage:
+  # The API url of a remote running Cuckoo API.
   api_url: null
 
   # API key needs to have administrator privileges to export to

diff --git a/docs/src/configuration/cuckooconfs/nodeconfs/routingyaml.md b/docs/src/configuration/cuckooconfs/nodeconfs/routingyaml.md
@@ -0,0 +1,95 @@
+## What is this config
+
+The `$CWD/conf/node/routing.yaml` is a configuration that contains all settings used by Cuckoo rooter. Cuckoo
+rooter automatically applies per-task network routes and settings.
+
+This file contains things such as:
+
+- Outgoing interfaces and a routing table for internet routing.
+- Existing VPN interfaces and routing tables for VPN routing.
+- OpenVPN configuration files and up script paths for automatically starting VPNs.
+
+### Default config example
+
+```yaml
+# This is the configuration file for Cuckoo rooter. Rooter performs
+# automatic routing for analysis machines. This file contains the
+# routes that will be available to each Cuckoo node. Cuckoo rooter must be
+# running for these routes to work.
+
+# Internet/dirty line routing routes machine traffic directly over the
+# specified interface. The machine IP will be added to the specified
+# routing table. Note that this will route the malicious traffic over the
+# configured network. An example of internet route could be a preconfigured
+# VPN interface and routing table.
+internet:
+  # Enable or disable internet routing.
+  enabled: False
+  # The interface the network should be forwarded to to reach the internet.
+  interface: null
+  # The routing table id/name rooter should add machine IPs to. This table
+  # should have the routes that result in traffic being routed over the
+  # specified interface.
+  routing_table: main
+
+# Rooter can use preconfigured VPN interfaces and routing tables and can also
+# start OpenVPN VPNs if their configuration paths are specified. VPN
+# routing is used when a country is specified as a routing option.
+#
+# If both preconfigured and VPN pool is enabled, Cuckoo will choose the first
+# available VPN that matches the specified country or the first available
+# VPN if no country is specified.
+vpn:
+  # Preconfigured and running VPN interfaces and their routing tables.
+  preconfigured:
+    # Disable or enable the use of preconfigured VPNs.
+    enabled: False
+    # A mapping of one or more preconfigured VPNs that Cuckoo rooter can use.
+    vpns:
+      # The VPN name that used in logging.
+      example_vpn:
+        # The existing VPN tun interface.
+        interface: tun0
+        # The routing table for the existing VPN.
+        routing_table: vpn0
+        # The country of that the VPN IP is identified as. Any string can be
+        # used.
+        country: country1
+
+  # A pool of VPN providers with OpenVPN VPN configurations. Rooter can
+  # automatically start and stop these VPNs when needed. This feature is
+  # useful if you want to support a large amount of exit countries.
+  vpnpool:
+    # Disable or enable the automatic starting of available VPNs.
+    enabled: False
+
+    # The range of routing table IDs that rooter can use to pass to up scripts.
+    # These tables *must not* interfere with other ranges/existing tables.
+    # The size of the range limits the amount of automatically started VPNs
+    # that can be active at the same time.
+    routing_tables:
+      start_range: 100
+      end_range: 200
+
+    providers:
+      example_provider:
+        # The maximum amount of connections/devices the VPN provider allows.
+        # This is the maximum amount of configurations that rooter will
+        # use simultaneously for this provider.
+        max_connections: 5
+
+        # A list of dictionaries with all the VPN configuration for this
+        # provider that rooter can start.
+        vpns:
+          # The type of VPN. Only OpenVPN is currently supported.
+        - type: openvpn
+          # The VPN configuration file. OVPN file, for example.
+          config_path: /path/to/ovpns/country1.ovpn
+          # The up script that adds the required routes to the automatically
+          # determine routing table. Do not change unless your custom script
+          # also performs what the default script does.
+          up_script: /home/cuckoo/.cuckoocwd/rooter/scripts/openvpnroutes.sh
+          # The country of that the VPN IP is identified as. Any string can be
+          # used.
+          country: country1
+```
diff --git a/docs/src/configuration/cuckooconfs/webconfs/webyaml.md b/docs/src/configuration/cuckooconfs/webconfs/webyaml.md
@@ -0,0 +1,73 @@
+
+## What is this config
+
+The `$CWD/conf/web/web.yaml` is configuration file that contains settings for the web API and UI. It contains
+settings such as the enabling/disabling of sample downloading and statistics.
+
+### Default config example
+
+```yaml
+# Remote storage usage is the retrieval of analysis reports etc from
+# a remote Cuckoo 'long term storage' host.
+remote_storage:
+  enabled: False
+  api_url: null
+
+  # API key does not need administrator privileges
+  api_key: null
+
+elasticsearch:
+  # The Elasticsearch settings must be configured to be able to use any of
+  # the features in this section.
+
+  # Enable or disable the Cuckoo web results search functionality
+  web_search:
+    enabled: False
+
+  # Enable or disable Cuckoo web results statistics. Detected family, behavior
+  # graphs, amount of submissions, etc.
+  statistics:
+    enabled: False
+
+    # All enabled charts types and the time ranges over which they
+    # should display data. Available range: daily, weekly, monthly, yearly.
+    # Available chart examples: families_bar, families_line, targettypes_bar,
+    # categories_bar, categories_line, submissions_line
+    charts:
+    - chart_type: submissions_line
+      time_range: yearly
+    - chart_type: submissions_line
+      time_range: monthly
+    - chart_type: families_bar
+      time_range: weekly
+    - chart_type: families_line
+      time_range: weekly
+    - chart_type: targettypes_bar
+      time_range: monthly
+    - chart_type: categories_bar
+      time_range: monthly
+
+  # The Elasticsearch hosts where results are reported to during processing.
+  # Should be one ore more host:port combinations.
+  hosts:
+    - http://127.0.0.1:9200
+
+  indices:
+    # The names to use when searching Elasticsearch. Each name must be unique
+    # and should also be used in reporting.
+    names:
+      analyses: analyses
+      tasks: tasks
+      events: events
+
+  # The max result window that will be used in searches. The Elasticsearch default is 10000. This
+  # window has impact in how far back you can search with queries that match a large amount of documents.
+  max_result_window: 10000
+
+# Specific web features that can be disabled/enabled
+web:
+  downloads:
+    # Enable/disable submitted file downloading.
+    submitted_file: True
+
+```
diff --git a/docs/src/configuration/index.md b/docs/src/configuration/index.md
diff --git a/docs/src/installation/cuckoo.md b/docs/src/installation/cuckoo.md
@@ -83,7 +83,7 @@ of:
     task running nodes.
 
 * One or more task running nodes
-    * This node accepts, runs tasks, and stores the collected behavioral logs. It has an API that the main node uses to tell it to run a task or to download a result for a task. This node type is "dumb" it does not know about other nodes or even the main node.
+    * This node accepts, runs tasks, and stores the collected behavioral logs. It has an API that the main node uses to tell it to run a task or to download a result for a task. This node type is "dumb" it does not know about other nodes or even the main node. This node is also where Cuckoo rooter should be running if automatic network routing is desired.
 
 #### Task running node(s)