You can use the audit log, the replication status, the traffic capture tool, and the capacity report in a Grid or HA pair to monitor administrative activities and capture traffic for diagnostic purposes. You can also use CLI commands to monitor certain DNS transactions.
This section includes the following topics:
In addition, if Grid members manage Microsoft servers, Grid Manager creates a synchronization log file for each managed Microsoft server. For information, see Viewing Synchronization Logs.
The audit log contains a record of all Infoblox administrative activities. It provides the following detailed information:
The appliance logs the following successful operations:
When the audit log reaches its maximum size, which is 100 MB, the appliance automatically writes the file into a new file by adding a .0 extension to the first file and incrementing subsequent file extensions by 1. Files are compressed during the rotation process, adding a .gz
extension following the numerical increment (file.#.gz
). The sequential incrementation goes from zero through nine. When the eleventh file is started, the tenth log file (file.9.gz
) is deleted, and subsequent files are renumbered accordingly. For example, the current log file moves to file
.0.gz
, the previous file.0.gz
moves to file.1.gz
, and so on through file.9.gz
. A maximum of 10 log files (0-9) are kept. To list the audit log files and their sizes, log in to the Infoblox CLI and execute the show logfiles command.
To enable audit log rolling:
Select Detailed (default), or Brief, or WAPI Detailed audit log type as follows:
This option helps you to troubleshoot and monitor performance issues that impact specific WAPI calls and track WAPI usage. When you select this option, you can view additional columns such as URI, InData and Response Time in the Audit log.
The following example shows an audit log entry for a POST WAPI call: [2018-05-29 09:20:12.026Z] [admin]: Created(POST) v2.9/zone_auth {"fqdn":"foo.com"} 2.233 AuthZone foo.com DnsView=default: Set fqdn="foo.com"
In the example above:
POST
indicates the WAPI callv2.9/zone_auth
is the URI{"fqdn":"foo.com"}
represents InDataNote
There might be a slight impact on the device performance as the session log information, such as URI, InData, and response time, are captured for all the successful WAPI calls. The audit log file size increases as the log entries increase, which may impact the storage capacity. Infoblox recommends that you select the Copy Audit Log Messages to Syslog check box in the Grid Properties Editor to move audit log information to the syslog and to an external server for longer retention. For more information, see Specifying Syslog Servers. All Cloud WAPI, via Cloud Network Automation (CNA) or Cloud Platform (CP) appliances, related events are logged to syslog instead of the audit log. For more information, see Cloud Network Automation.
To view an audit log:
Note
The admin user displayed as $admin group name$ represents an internal user. You can create a admin filter with “matches expression” equals ^[^$] to filter out internal users.
You can also do the following in the log viewer:
Instead of paging through the audit log file to locate messages, you can have the appliance search for messages with certain text strings.
To search for specific messages:
The appliance searches through the audit log file and highlights the search value in the viewer. You can use the arrow keys next to the Search icon to locate the previous or next message that contains the search value.
You can download the audit log file to a specified directory, if you want to analyze it later. To download an audit log file:
Note
If your browser has a pop-up blocker enabled, you must turn off the pop-up blocker or configure your browser to allow pop-ups for downloading files.
The Replication Status panel reports the status of the database replication between Grid members and Grid Master, and between the two nodes in an independent HA pair. You can use this information to check the health of the Grid and HA pair activity.
To view the current replication status, from the Grid tab, select the Grid Manager tab -> Members tab, and then click Toggle Replication Status View.
Grid Manager can display the following replication information for each member:
You can do the following:
You can capture the traffic on one or all of the ports on a NIOS appliance, and then view it using a third-party network protocol analyzer application, such as the Wireshark – Network Protocol Analyzer™. Using this tool, you can capture traffic for a single member or multiple Grid members simultaneously. The NIOS appliance must have a minimum of 500 MB of free disk space to start the traffic capture; otherwise, the traffic capture might fail.
The NIOS appliance saves all the traffic it captures in a .cap file and compresses it into a .tar.gz file. Your management system must have a utility that can extract the .tar file from the .gzip file, and an application that can read the .cap (capture) file format. The size of the .cap file is limited to 2 GB for the Infoblox-4010, Infoblox-4030, Infoblox-4030-10GE, and PT-4000, and the size is limited to 1 GB for all other NIOS appliances. You can also transfer the traffic capture file to your local management system, a TFTP server, an FTP server, or a SCP server.
NIOS saves failed traffic capture file transfer attempts in the syslog. Traffic capture file transfers can fail due to remote server issues, such as an invalid server IP address, or an incorrect username, or a password, or an invalid path. Note that the traffic capture fails when the disk space on the member is insufficient, but the file transfer will be successful. The NIOS appliance logs a warning message in the syslog when the traffic capture fails. You can find the following information about traffic capture and file transfers in the audit log:
Note
This feature captures traffic of all the direct responses received from the cache accelerator on the IB-4030.
This section explains the process of capturing traffic, and how to download the traffic capture file to your management system. After that, you can extract the traffic capture file and view it with a third-party traffic analyzer application. The traffic capture file is shared between admin users.
You can also configure Grid Manager to trigger a traffic capture at set intervals and parameters. If Grid Manager detects that a parameter has breached a configured threshold or crossed the configured duration of time, it triggers a traffic capture. For more information about automated traffic capture, see Enabling Automated Traffic Capture.
Note
The NIOS appliance always saves a traffic capture file as <member name>_<timestamp>_tcpdumpLog.tar.gz. Example: infoblox.localdo_0_2018-10-15-03-47-53_tcpdumpLog.tar.gz. For FTP and TFTP transfers, the name of the file is added as a prefix. Example: filename.infoblox.localdo_0_2018-11-09-09-30-07_tcpdumpLog.tar.gz
For a single member, you can also capture traffic on the NIOS appliance through the Infoblox CLI using the set traffic_capture
command. However, you cannot use this command to capture traffic for multiple members. NIOS displays the traffic capture status and it allows you to download the captured traffic, irrespective of whether the traffic capture is initiated from the Infoblox CLI or from Grid Manager.
To capture traffic for a single member or multiple Grid members:
Members
Name: Click the Add icon to add either a single or multiple Grid members for which you want to capture traffic. When you click the Add icon, Grid Manager displays the Member Selector dialog box from which you can select one or multiple members. Use SHIFT+click to select multiple contiguous rows or use CTRL+click to select multiple non-contiguous rows. Click OK. The selected members are added to the list of members in the Members table. You cannot add offline members to the list or capture traffic on an offline member.
Note
Selecting members in the Grid Manager → Members tab does not capture traffic for the selected member. To capture traffic, you must select members from the Member Selector dialog box.
Interface: Select the port on which you want to capture traffic. You can view the selected interface while the traffic capture is in progress. Note that if you enabled the LAN2 failover feature, the LAN and LAN2 ports generate the same output and Grid Manager displays the interface as BOND while the traffic capture is in progress. By default the interface is set to ALL after the traffic capture process stops. For information about the LAN2 failover feature, see About Port Redundancy.
LANxnnnn: If you have configured VLANs on the LAN1 or LAN2 port, the appliance displays the VLANs in the format LANx nnnn, where x represents the port number and nnnn represents the associated VLAN ID.
3. Seconds to run: Specify the number of seconds you want the traffic capture tool to run.
4. Capture Control: Click the Start icon to start the capture. Note that the start action will overwrite the existing traffic capture file. You can click the Stop icon to stop the capture after you start it.
5. Transfer To: Select the destination to transfer the traffic capture file. You can select My Computer, TFTP, FTP, or SCP from the drop-down list.
My Computer: Transfer the traffic capture file to a local directory on your computer. This is the default.
Note
To avoid consumption of the Grid Master disk space, NIOS restricts downloading the traffic capture files from multiple members to a local directory on your computer.
/home/test/traffic_capture_filename
where traffic_capture_filename
is the name of the file. /home/test/traffic_capture_filename
where traffic_capture_filename
is the name of the file./home/test/
.6. Uncompressed Capture File Size: Select the members for which you want to download the traffic capture file and then click Download to download the captured traffic. You can download and save the file only after the capture stops, but not when the tool is running. You can rename the file if you want. NIOS updates the size of the report when the capture tool is running.
Note
The NIOS appliance must have free disk space of at least 500MB + size of the traffic capture file (2 GB/1 GB, depending on the appliance model) to download the traffic capture file.
7. Last updated: The timestamp of the last traffic capture process.
8. Use terminal window commands (Linux) or a software application (such as StuffIt™ or WinZip™) to extract the contents of the .tar.gz file.
9. When you see the traffic.cap file in the directory where you extract the .tar.gz file, open it with a third-party network protocol analyzer application.
You can view the capacity usage and object type information of an appliance in a capacity report. The capacity report displays capacity and object type information of an independent appliance, a Grid Master, or a Grid member. For an HA pair, the report displays information on the active node.
The top half of the panel displays a capacity summary, and the bottom half displays the object types the appliance supports and the total counts for each object type.
To view a capacity report:
The capacity summary contains the following information:
The capacity report filters object types you can manage through the appliance. You can configure the object types you want to see in the following table by completing the following in the Minimum Object Total filter:
The capacity report displays the following information:
Administrators with superuser accounts can configure a Grid Master or an independent appliance to email reports monthly and after each upgrade to Infoblox Technical Support and other specified recipients. The reports are also included in support bundles that you download.
The reports provide status and event information about the Grid or independent appliance and its services. The report is an XML document that includes the following information:
Note that if the appliance is configured to send email notifications to an SMTP relay server, as described in Notifying Administrators, the appliance sends the phone home reports to the relay server as well.
To configure the Grid Master to email status reports:
4. Save the configuration and click Restart if it appears at the top of the screen.
The NIOS appliance provides tools for monitoring DNS transactions and mitigating cache poisoning from UDP (User Datagram Protocol) traffic on source port 53. Cache poisoning can occur when a DNS server accepts maliciously created unauthentic data. The DNS server ends up locally caching the incorrect entries and serving them to users that make the same DNS requests. In a maliciously created situation, the attacker can redirect Internet traffic from the legitimate host to another host that the attacker controls.
You can configure the appliance to track invalid DNS responses for recursive DNS queries. The appliance tracks DNS responses that arrive on invalid ports or have invalid TXIDs (DNS transaction IDs). Both invalid ports and invalid TXIDs could be indicators of cache poisoning. An invalid port is a DNS response that arrives from UDP (User Datagram Protocol) port 53 with either one of the following conditions:
An invalid TXID is a DNS response that arrives from UDP port 53, and the TXID does not match the TXID of an outstanding DNS request. Figure 37.1 illustrates how the appliance detects an invalid port and an invalid TXID.
Figure 37.1 Invalid Port and Invalid TXID
Both invalid ports and invalid TXIDs could be indicators of DNS cache poisoning, although a small number of them is considered normal in situations where valid DNS responses arrive after the DNS queries have timed out. You can configure the appliance to track these indicators, and you can view their status. You can also configure thresholds for them. When the number of invalid ports or invalid TXIDs exceeds the thresholds, the appliance logs an event in the syslog file and sends an SNMP trap and e-mail notification, if you enable them. You can then configure rate limiting rules to limit incoming traffic or completely block connections from primary sources that send the invalid DNS responses.
Rate limiting is a token bucket system that accepts packets from a source based on the rate limit. You can configure the number of packets per minute that the Infoblox DNS server accepts from a specified source. You can also configure the number of packets for burst traffic, which is the maximum number of packets that the token bucket can accept. Once the bucket reaches the limit for burst traffic, it discards the packets and starts receiving new packets according to the rate limit.
The appliance monitors only UDP traffic from remote port 53 for the following reasons:
To monitor invalid ports and invalid TXIDs on the Infoblox DNS server, follow these procedures:
To mitigate cache poisoning, you can limit incoming traffic or completely block connections from specific sources, as follows:
You can verify the rate limiting rules after you configure them. For information, see Viewing Rate Limiting Rules
The appliance monitors only UDP traffic on port 53 for recursive queries, and then reports invalid DNS responses.
DNS alert monitoring is disabled by default. For an HA pair, you must enable DNS alert monitoring on both the active and passive nodes.
To enable DNS network monitoring and DNS alert monitoring:
set monitor dns on
The appliance displays the following:Turning on DNS Network Monitoring...
3. Enter the following command:
set monitor dns alert on
When you enable DNS alert monitoring and DNS network monitoring is disabled, the appliance automatically enables DNS network monitoring and displays the following:
DNS Network Monitoring is disabled. It must be enabled for alerting to function.
Enable DNS Monitoring now? (y or n):
You can also disable DNS network monitoring and DNS alert monitoring using the following commands:
set monitor dns off
set monitor dns alert off
Note
When you restart DNS network monitoring, you also reset the SNMP counters for DNS alerts.
You can then view the alert status to identify the primary source of invalid DNS responses. For information, see Viewing DNS Alert Indicator Status.
To view DNS alert indicator status:
show monitor dns alert status
The appliance displays historical alert counts and up to five primary sources that generate invalid DNS responses, as shown in the following example: Data last updated: Mon Oct 6 14:47:12 2008
DNS Alert1m5m15m60m24hEver
============================================
port8 12 1212 12 12
txid8 12 1212 12 12
There were 80 DNS responses seen in the last minute.
10% were to an invalid port.
10% had an invalid TXID.
Primary sources of invalid responses:
4.4.4.4 (unknown) sent 4
2.2.2.2 (unknown) sent 3
7.7.7.7 (unknown) sent 1
The appliance attempts to resolve the hostnames of the sources that sent invalid responses, if the DNS resolver is enabled. If the appliance cannot resolve a hostname, it displays "unknown" as the hostname of the invalid response.
You can configure thresholds for DNS alerts to control when the appliance tracks DNS attacks on UDP port 53 and issues SNMP traps and e-mail notifications.
Note
Ensure that you enable SNMP traps and e-mail notifications. For information, see Configuring SNMP.
You can configure thresholds for both invalid ports and invalid TXIDs. The default thresholds for both invalid ports and TXIDs are 50%. When the number of invalid ports or invalid TXIDs exceeds the thresholds, the appliance logs the event and sends SNMP traps and notifications. You can configure the thresholds either as absolute packet counts or as percentages of the total traffic during a one minute time interval.
To configure DNS alert thresholds:
set monitor dns alert modify port | txid over threshold_value packets | percent
whereport | txid
= Enter port
to set the threshold for invalid ports, or enter txid
to set the threshold for invalid TXIDs.threshold_value =
Enter the number of packets or percentage for the threshold.packets | percent =
Enter packets
if you want to track the total packet count, or enter percentage
if you want to track a percentage of the total traffic. For a percentage-based threshold, the appliance does not generate a threshold crossing event if the traffic level is less than 100 packets per minute.
For example, if you want the appliance to send a DNS alert when the percentage of DNS responses arriving on invalid ports from UDP port 53 exceeds 70% per minute, you can enter the following command:
set monitor dns alert modify port over 70 percent
If you want the appliance to send a DNS alert when the total number of packets with invalid TXIDs from UDP port 53 is over 100 packets per minute, you can enter the following command:
set monitor dns alert modify txid over 100 packets
When there is a DNS alert, the appliance logs an event in the syslog file and sends an SNMP trap and e-mail notification if enabled.
You can view the DNS alert thresholds. The appliance displays the current thresholds. If you have not configured new thresholds, the appliance displays the default thresholds, which are 50% for both invalid port and TXID.
To view the DNS alert thresholds:
show monitor dns alert
The appliance displays the threshold information as shown in the following example:
DNS Network Monitoring is enabled. Alerting is enabled.
DNS Alert Threshold (per minute)
===========================================
portover 70% of packets
txidover 100 packets
You can mitigate cache poisoning on your DNS server by limiting the traffic or blocking connections from UDP port 53. To enable rate limiting from sources:
set ip_rate_limit on
The appliance displays the following:
Enabling rate limiting will discard packets and may degrade performance.
Are you sure? (y or n):
Note
When you enable rate limiting, the appliance discards packets based on the configured rate limiting rules.
This might affect the DNS performance when the appliance discards valid DNS responses.
3. Enter y to enable rate limiting.
When you enable rate limiting, the appliance applies the rate limiting rules that you configured. You might want to configure the rate limiting rules before enabling rate limiting. For information on how to configure rate limiting rules, see Configuring Rate Limiting Rules.
You can also disable rate limiting by entering the following command:
set ip_rate_limit off
When you disable rate limiting, the appliance stops applying the rate limiting rules.
You configure rate limiting rules to limit access or block connections from UDP port 53. The rules take effect when you enable rate limiting.
When adding rules, ensure that you do not include an IP address that matches the IP address of either the Grid Master or Grid member. Doing this could affect VPN connectivity. To configure rate limiting rules:
set ip_rate_limit add source all | ip_address [/mask] limit packets/m [burst burst_packets]
where
all | ip_address
= Enter all
or 0.0.0.0
if you want to limit all traffic from all sources, or enter the IP
address from which you want to limit the traffic.
[/mask]
= Optionally, enter the netmask of the host from which you want to limit the traffic.
packets
= Enter the number of packets per minute that you want to receive from the source.
[burst burst_packets]
= Optionally, enter burst
and the number of packets for burst traffic. This is the maximum number of packets accepted.
The following are sample commands and descriptions for rate limiting rules:
set ip_rate_limit add source 10.10.1.1 limit 0
set ip_rate_limit add source 10.10.1.2 limit 5/m
set ip_rate_limit add source 10.10.2.1/24 limit 5/m burst 10
set ip_rate_limit add source all limit 5000/m
You can remove the existing rate limiting rules that limit access or block connections from UDP port 53. To remove all the existing rules:
set ip_rate_limit remove source all
or
set ip_rate_limit remove all
To remove one of the existing rules for an existing host:
set ip_rate_limit remove source ip-address[/mask]
You can view the existing rate limiting rules that limit access or block connections from UDP port 53. To view rate limiting rules:
show ip_rate_limit
The appliance displays the rules, as shown in the following example:
IP rate limiting is enabled.
Source Limit Burst
============================================
10.10.1.1 0 packets/minute 0 packets
10.10.1.2 5 packets/minute 5 packets
10.10.2.1/24 5 packets/minute 10 packets
all 5000packets/minute 5000 packets
* Note: Copy audit to syslog feature should not alter. But increases the syslog file size.
This page has no comments.