Written by: Steve McMaster
At Hurricane Labs, we like to know as much about our network traffic as possible. Not only does this allow us to plan things (like QoS or network hardware requirements) better, it can also help with a number of other things, from identifying infected hosts (though that’s not really a problem we have often, being a 99% Linux shop) to reprimanding users of our hosting network for using the 3rd highest amount of bandwidth. ntop is an open source application that allows us to view network traffic statistics, such as which hosts or which protocols consume the most bandwidth, via several methods. The methods we’re currently using are Cisco SPAN ports and Cisco Netflow. To do this, you’ll need a server with a network interface for management and receiving Netflow data, plus another network interface for each switch you’re monitoring (unless you’re using RSPAN, which will not be covered in this article).
According to their website, ntop is “a network traffic probe that shows the network usage”. ntop is available for both Windows and *nix from their website, and is also available in the repositories of our preferred Linux distro, Ubuntu. Installation may vary from platform to platform, but on Ubuntu it was especially easy. Simply run ‘sudo apt-get install ntop’ from the terminal on one of your servers, and Ta Da! It’s installed. If you were to now start ntop (via the standard ‘sudo /etc/init.d/ntop start’) you would see a message that says ntop has started. However, if you tried to go to the ntop web interface (http://example.com:3000/), you would see that the server could not be reached. This is because ntop has not been configured with an administrator password. To do this, simply run ‘sudo ntop’ from your terminal. ntop will start, and then ask you to set the admin password. Once set and verified, ntop will start running. At this point, you could connect to the web interface at the URL. However, since we generally like ntop to be running in the background as a daemon to enable logging and it doesn’t require us to be logged into the system’s terminal, we’re going to quit out of ntop by pressing the Conrol key and the C key simultaneously. Now we can run ‘sudo /etc/init.d/ntop start’ and ntop will start correctly. If you want to take a look around ntop’s web interface, point your browser at the URL above. But before you go on, be sure to stop ntop by running ‘sudo /etc/init.d/ntop stop’.
Before we will see any real traffic on this server, we’re going to need a way to get information about the traffic to our server. There are two ways to do this – Cisco SPAN ports (called mirror or monitor ports by other switch manufacturers) and Cisco Netflow. There is also another protocol that ntop supports, called sFlow, that other hardware vendors may support, but this is beyond the scope of this article. Cisco SPAN ports allow traffic received on a switchport or VLAN to be “mirrored” to another port. This method is supported on most modern Cisco IOS switches, but we’ll use Netflows for Cisco routers later on. We generally use SPAN ports to mirror traffic from all VLANs on a switch, which can be done fairly simply. However, you need to make sure you know all of the VLANs on your switch. Check the output of ‘show vlan’ on your switch if you don’t already know. To do this, you must have access to your switch, as well as access to enable mode on the switch. Run the following to enable a SPAN port on GigabitEthernet0/1 (the very first port in your switch), assuming you use VLANs 1 through 10 in your network. Pay careful attention to the syntax – you are limited to 5 VLANs per line, and you need a space before and after the commas.
Switch> enable
Switch# configure terminal
Switch(config)# monitor session 1 source vlan 1 , 2 , 3 , 4 , 5 rx
Switch(config)# monitor session 1 source vlan 6 , 7 , 8 , 9 , 10 rx
Switch(config)# monitor session 1 destination GigabitEthernet0/1
Switch(config)# exit
Switch# copy running-config startup-config
After connecting GigabitEthernet0/1 to one of the extra interfaces on your ntop server, you need to make just a few more changes on your ntop box to get it listening on this interface. First, you need to set the interface into promiscuous mode. To do this as a one time thing, run ‘sudo ifconfig eth1 promisc’, assuming that eth1 is the interface you’ve connected to the monitor port. If you’d like to make this more permanent, you’ll need to add configuration to the network configuration file. Assuming that this is an Ubuntu or other Debian-based server, and that eth1 is the interface you’re using, you’ll need to remove any lines from the network configuration file (‘/etc/network/interfaces’) that are for eth1, and add the following lines. Edit the file by running ‘sudo editor /etc/network/interfaces’. You can replace ‘editor’ with your editor of choice if you have one, otherwise the system default will be used:
auto eth1
iface eth1 inet manual
up ifconfig eth1 up promisc
down ifconfig eth1 down -promisc
After doing this, eth1 will always be set to promiscuous mode upon startup. The last thing you need to do is to change the ntop configuration file to listen on this interface. To do this, run ‘sudo editor /var/lib/ntop/init.cfg’ and change the line that reads ‘INTERFACES=”eth0”’ to say ‘INTERFACES=”eth1”’. Once this is done, you’re ready to start ntop with the ‘sudo /etc/init.d/ntop start’ command.
The last step is to configure Netflows to send to this ntop server. This is probably the most involved step of the journey, as it involves changes to the router, firewall rules, possibly NAT, and the ntop interface. However, Netflows will allow you to see traffic as your edge routers see it, which is more than likely a little different than how your internal switches will see it. An example is that on the inside, one host may be the highest bandwidth user, but on the outside your firewall (or firewall cluster) is the highest user. This would be the case if you have a lot of VPN traffic, which will appear to be originating from your firewall. The first step to do this is to configure your firewall to allow traffic from your routers to your ntop box, including any necessary NAT rules, on the ports you designate for Netflow destinations. The “standard” port is 2055 (UDP). A good practice would be to start at 2055 and work your way up for each additional router. To configure Netflow, you’ll need access to your router, as well as enable mode access. Run through the following to enable Netflow – we assume 192.168.0.100 is the IP of your ntop server, and that you’re using port 2055. You’ll also want to replace GigabitEthernet0/0 in the ‘ip flow-export source’ line with whichever interface you want the traffic to be sourced from; the IP of this interface will be the one you should use in your firewall rule(s). This also assumes you want to send flows for the traffic traversing GigabitEthernet0/0 and GigabitEthernet0/1:
Router> enable
Router# configure terminal
Router(config)# ip cef
Router(config)# ip flow-export version 5
Router(config)# ip flow-export destination 192.168.0.100 2055
Router(config)# ip flow-export source GigabitEthernet0/0
Router(config)# interface GigabitEthernet0/0
Router(config-if)# ip route-cache flow
Router(config-if)# exit
Router(config)# interface GigabitEthernet0/1
Router(config-if)# ip route-cache flow
Router(config-if)# exit
Router(config)# exit
Router# copy running-config startup-config
Now that you have your firewall changes made (give Hurricane Labs a call if you need some assistance with this), all that is left is to configure ntop. Open up the web interface (http://example.com:3000/). You should see some nice, fancy menus across the top. Hover over the Plugins Menu, then move the mouse to Netflow. In the Netflow submenu, choose “Activate”. This will take you to a page that describes the NetFlow plugin. From this page, if you click “NetFlow” in the table shown, you will be taken to the NetFlow Device Configuration page. From here, click “Add NetFlow Device”. ntop uses virtual devices to receive the Netflow traffic, and these interfaces should be unique per router. When adding a device, you’ll want to change three settings: the Device name, the UDP port, and the Virtual address. The device name is arbitrary and should be set to something meaningful, like the name of the router it corresponds to. The UDP port should be the port you directed the router at and allowed through your firewall. The virtual address should be the address of one of the interfaces on the router using this device. After changing each of these settings, click the button next to it labelled “Set Interface Name”, “Set Port”, or “Set Interface Address”, accordingly.
Now that you have all of your interfaces set up, return to the ntop main screen (point your browser back to the main URL, http://your.server.com:3000). From there, poke around in the menus at some of the ways ntop lets you break down your network traffic. Right off the bat, you won’t have a lot of data to look at or depending on your network usage, maybe you will. If you let ntop run through a day or two of regular network usage, you’ll start to see patterns of usage appear. In a future article, expect to see an explanation of what some of ntop’s graphs mean, and what you can do with the data you’ve collected. If nothing else, this whole endeavor will leave you with a clearer understanding of what your bandwidth is being used for. It might even be enough to convince the higher-ups that you need more bandwidth (although I can’t really guarantee that). And remember, ntop is available as a managed service for all Hurricane Defense customers who want it.
Posts
0 comments:
Post a Comment