Page tree

Contents

Setting Up ELK for Use with Data Connector

To set up ELK for use with Data Connector, complete the following:

  1. Depending on your setup, you may need to create a new input configuration file or update an existing file. Logstash supports different plugins to accept messages from Data Connector. Infoblox recommends to use TCP input with CEF codec. The TCP-port should match the TCP-port configured on the Syslog destination.

    Sample input configuration:

    input {
      tcp {
            port => 5534
            codec => cef{ }
            type => syslog
            tags => ["cdc"]
      }
    }
    
    filter {
      if "cdc" in [tags] {
      }
    }
    
    
    
    output {
      elasticsearch {
             hosts => ["127.0.0.1:9200"]
             index => "cdc-syslog"
      }
    }
  2. Restart Logstash.

  3. Configure Syslog destination on the Cloud Services Portal. You will need to provide the following information when configuring your destination: 

    • The Logstash IP address or hostname
    • TCP-port specified in the input configuration
    • CEF output format
    • Do not disable "Insecure" mode.

     

For more information on setting up a destination configuration, see Configuring Destinations.

Configuring a Traffic Flow

To push traffic to ELK, create a traffic flow and select the created destination as a destination for the traffic flow. For more information on configuring a traffic flow, see Configuring Traffic Flows.  

Checking Events in Kibana

To check events in Kibana, complete the following:

To view the configuration, under Kibana in the side menu navigate to Index Patterns (Kibana -> Management -> Index Patterns).

  1. In the Index Pattern field, enter cdc-syslog.

       2. Click Next step.
       3. For the Time Filter field name, select @timestamp.
       4. Click Create index pattern.


      5. Navigate to Discover and select cdc-syslog index.
      6. Continue sending the data to logstash, which will appear in this window as shown in the two images below.


Performance

Depending on Logstash, ELK configuration and Data Connector VM parameters, and Data Connector configuration and load, event types will be processed. The maximum Data Connector performance of Logstash via Syslog TCP, is up to 18.000 events per second (EPS). For performance testing to reach the maximum Logstash performance, CEF codec was disabled and output was configured to file (/dev/null).

To estimate EPS you need to summarize all event types received by Data Connector. For example, if on NIOS DNS Query where DNS Response and RPZ logs are enabled, then EPS will be calculated by doubling an average of DNS QPS and adding the average number of RPZ hits per second.

  • No labels

This page has no comments.