This document provides useful information and examples for using nProbe metrics with Elasticsearch and Kibana.
Requirements:
- nProbe Pro + Export Plugin for Elasticsearch Bulk
- nProbe Plugin(s) matching the desired Dashboard (ie: DNS, HTTP, etc)
- Elasticsearch 2.x (cluster or single server)
- Kibana 4.2+ (cluster or single server)
A script is provided to automatically load all nProbe demo dashboards and settings:
# ./load.sh -url http://localhost:9200
Once imported, Kibana objects should become available:
Before shipping nProbe metrics in Kibana, an index template is required to let Elasticsearch know which fields should be analyzed in which way. The provided template file can be adjusted and loaded with the following command:
curl -XPUT localhost:9200/_template/nprobe_template --data @template/nprobe-es-template.json
Once the metrics are indexed in ES, a matching Kibana index pattern should be created as follows:
Users can optionally generate scripted fields "on-the-fly" in Kibana when needed. The following example generate new flow fields with sum of IN/OUT Bytes and Packets:
The provided Dashboards are a work-in-progress and community contributions are extremely welcome!
Start nProbe exporting %DNS_* fields:
nprobe --elastic "dns;nprobe-%Y.%m.%d;http://localhost:9200/_bulk;" -T "%IPV4_SRC_ADDR %L4_SRC_PORT %IPV4_DST_ADDR %L4_DST_PORT %PROTOCOL %IN_BYTES %OUT_BYTES %FIRST_SWITCHED %LAST_SWITCHED %IN_PKTS %OUT_PKTS %IP_PROTOCOL_VERSION %APPLICATION_ID %L7_PROTO_NAME %ICMP_TYPE %SRC_IP_COUNTRY %DST_IP_COUNTRY %APPL_LATENCY_MS %DNS_QUERY %DNS_QUERY_ID %DNS_QUERY_TYPE %DNS_RET_CODE %DNS_NUM_ANSWERS %DNS_TTL_ANSWER %DNS_RESPONSE" ....
Start nProbe exporting %HTTP_* fields:
nprobe --elastic "http;nprobe-%Y.%m.%d;http://localhost:9200/_bulk;" -T "%IPV4_SRC_ADDR %L4_SRC_PORT %IPV4_DST_ADDR %L4_DST_PORT %PROTOCOL %IN_BYTES %OUT_BYTES %FIRST_SWITCHED %LAST_SWITCHED %IN_PKTS %OUT_PKTS %IP_PROTOCOL_VERSION %APPLICATION_ID %L7_PROTO_NAME %ICMP_TYPE %SRC_IP_COUNTRY %DST_IP_COUNTRY %APPL_LATENCY_MS %HTTP_URL %HTTP_METHOD %HTTP_RET_CODE %HTTP_REFERER %HTTP_UA %HTTP_MIME %HTTP_HOST %HTTP_SITE" ....
