Netdb
From Sanet
Contents |
Introduction
In a building or campus network, switches and routers can supply a lot of information about what they connect: MAC addresses, ports, IP addresses, etc. Netdb collects all this information and keeps an history of the relations among then.
Netdb collects information about this entities:
- MAC addresses
- IPv4 addresses
- Switch ports
- Netbios names
- DNS names (direct and reverse)
The collected information is:
- ARP entries (node, interface, MAC, IPv4)
- FDB entries (node, interface, MAC)
- Single-mac FDB entries: An FDB entry is single if it is the only MAC on that port in that moment: this usually means that the device is connected to that interface. Single-mac FDB entries are kept together with other FDB entries, and also kept in a specific table.
- Switch port correspondence between bridge instance numbers (brinst) used in the FDB and ifindexes used in ARP and MAC tables
- Switch port in use (i.e. link is up) (node, interface)
- IP address assigned to a node (IPv4, node, interface)
- MAC address assigned to a node (MAC, node, interface)
- Netbios name (IPv4, name, type of name, unique or group)
- DNS direct (name, IPv4)
- DNS reverse (IPv4, name)
All this tuples of information are kept together with a time reference in the form of a (begin, end) interval of validity, so you can see the current relations among addresses, names and switches, and also see the complete history since netdb was started. Netdb collects this information periodically and it can take some hourse to cycle through all the nodes. Having a very big amount of small entries would put a great load on the database in a big network. For these reasons, this time reference is somewhat approximated: although the begin and end are expressed with seconds, you should only look at the day.
Configuration
In order to install and enable netdb you should perform the following operations in this order:
- Enable netdb in settings.py and tune more settings if desired
- Configure switches, routers and subnets from the CLI
- Issue "python manage.py syncdb" and restart the poller
Enabling Netdb
You can enable Netdb in settings.py with:
if not 'netdb' in INSTALLED_APPS:
INSTALLED_APPS.append('netdb')
If you want to collect NetBIOS names from PCs, you must tell netdb the command to use: nbtscan is faster, but for a small network nmblookup can suffice. Give netdb the command to use, with %% where the IP address should be placed, e.g.:
NETBIOS_NODE_STATUS_COMMAND = '/usr/bin/nbtscan -t 500 -v -r %%' # or: # NETBIOS_NODE_STATUS_COMMAND = '/usr/local/bin/nmblookup -U %% -S -A %%'
Add "-r" to nmblookup or nbtscan if you have no nmbd running on the server and you want to see NetBIOS name from Win95 machines.
Netdb collects information with a configurable number of threads: more threads make more collections per day, but also more load on the server. The default is 5, you can change it with:
COLLECTOR_THREADS=5
Netdb uses an SNMP timeout computed from the observed RTT of the node, and a fixed configurable number of retries, default 2. You can change the number of retries with:
COLLECTOR_SNMP_TRIES=2
Switches, routers and subnets
Netdb collects ARP information from Sanet nodes that have the "router" flag in their node-category. Check with show categories node-categories for categories having that flag, and check with show nodes that nodes belong to the intended category. Use the router or no router commands in the node-category configuration prompt to change the flag.
Netdb collects FDB information from Sanet nodes that have the "switch" flag in their node category. Check and manage this flag exactly as you do for the "router" flag. The two flags are independent from each other.
You can define subnets to limit the IPv4 addresses from which Netdb collects NetBIOS name information. Querying for NetBIOS names is an intrusive operation that can be seen as suspicious, so you should do it only on devices that you manage or for which you are authorized in some way.
Sanet maintains an ordered list of IPv4 subnets, expressed as prefixes, and for each of them is defined a behaviour:
- deny do not probe the device in any way
- allow probe the device as needed without limitation
- noscan do only mild probing operations
Currently Netdb probes only for NetBIOS names, and does this for prefixes with allow, so there is currently no difference between deny and noscan.
When Netdb is about to probe an IP address for NetBIOS names, it walks the ordered list until it finds a prefix that matches the address, and if the prefix is permit it does the probing. If it is deny or noscan netdb skips that address. Walking through the end of the list is considered an implicit deny.
You can populate the list with the command:
subnet sequence number base/len allow|deny|noscan
and you can remove an item in the list with:
no subnet seq
You can verify the list with the show subnets or show configuration subnets commands.
The subnets are treated as pure IPv4 ranges in order to verify if the address fits in it. There is no special treatment of base, broadcast, etc.
The collector will probe only IPv4 addresses that it found in some other way, it will not sequentially probe the whole subnet.
The collector process
When you have installed netdb in settings.py and optionally tuned the other relevant settings, you can update the database and restart the poller with the commands:
- etc/rc.sanet stop
- python manage.py syncdb
- etc/rc.sanet start
You will see with ps aux | fgrep poller that in addition to the supervisor and the checker processes, now there is also the collector process.
You can verify its operation by grepping collector in var/poller_log. You will see messages when it is unable to collect some information from a node via SNMP.
Doing a complete collection cycle can takes minutes or hours, depending on the size of the network. Collecting NetBIOS names is usually the more time comsuming phase, so define your subnets with care.
You can see when a cycle ends, by grepping pass in the log. This is an example from a network with a few thousands PCs:
[collector] pass 1 completed, times: nodes 1668, direct and reverse names 561, netbios names 22188, total 24417
The line means that the first pass (from poller restart) took:
- 1668 seconds to query nodes for FDB, ARP, defined IPv4 and MAC addresses and used ports
- 561 seconds to query DNS direct and reverse
- 22188 to query netbios names
- 24417 seconds total
If you want your passes to end in less seconds, raise the number of threads, but be careful because you will also increase the load on the system, or restrict the subnets where NetBIOS name scanning is allowed. The collector should be able to do at least two or three passes a day to collect meaningful data.
Displaying collected data
The collected data can be displayed with the CLI or with the advanced search page in the web UI. The information displayed includes begin and end of validity for each entry with second precision, but due to the cyclic nature of netdb polling they should be regarded as being approximated to the day. Each entry also has an arbitrary numeric id, which can be used in some cases to display more information about it.
MAC address are always written in uppercase hex with no separator among bytes, and IPv4 addresses are written as dotted quad. Search patterns for MAC addresses, IPv4 addresses and names don't have to be at the beginning of the MAC address, so e.g. 005E4F matches both 005E4F0A0D43 and 00005E4F0197.
Searching collected data in the Web UI
In the Advanced search page the data collected with netdb can be searched using the checkmarks for IPv4, MAC, DNS or NetBIOS names, used ports in switches, bridge ports in switches. Selecting the checkmarks enables searching in all the objects where the field is present.
This feature is available only to users having superuser privileges.
The search string is matched as a case insensitive substring. The different kind of data displayed can be sorted with the controls in the table headers.
A too small search string can return a huge amount of data, which can make the search very long: try to avoid too generic substrings (e.g. "10."). There is currently no pagination nor limit on the results returned.
CLI commands
When netdb is installed, the following CLI commands are available to query the collected information. When the keyword detail is available, it enables a more detailed multiline output.
The database can contain many entries with essentially the same data, differing only for the time interval. When an entry is not seen in the tables for a lot of time (more than a week), and then reappears, the database will contain two entries: the first for the period before the interruption, and the second for the period after it. Suffixing the keyword with -history, when allowed, includes in the output all the entries with the same data, referring to different disjoint intervals in time. Without that suffix, only the newest entry is shown.
show ip summary ip
Show summary information about a single IP address. This will include any direct, reverse and NetBIOS names, nodes having that address configured on an interface, ARP entries with corresponding FDB entries where the MAC address was the only one on the port.
This command is intended for everyday use, to associate IP addresses with switch ports (other commands give much more detail).
Example output:
sanet# sh ip summ 192.168.0.1 Nodes having 192.168.0.1 assigned: 2012-06-14 fw-vrrp, ifindex 4, 192.168.0.1/26 (255.255.255.192) 2012-06-14 fw1, ifindex 4, 192.168.0.1/26 (255.255.255.192) DNS entries for 192.168.0.1: 2012-06-14 fw1 -> 192.168.0.1 2012-06-14 fw1.example.com -> 192.168.0.1 2012-06-14 192.168.0.1 -> fw1.example.com ARP entries for 192.168.0.1: 2012-06-14 001C7E251E1E, seen at switch-core-1, ifindex 627 Switch ports for 001C7E251E1E in the same day: 2012-06-14: 192.168.0.1 -> 001C7E251E1E -> switch-dc-1 port 25 (ifindex 25) (port now b1)
show mac [ assigned[-history] pattern | fdb[-history] [ mac pattern ] | fdb-alone[-history] [ mac pattern ] ]
Show diverse information about MAC addresses matching pattern. Keywords can be used to select the type of MAC address collected to show: assigned (the mac address of an interface of a sanet node), fdb (present in the forwarding database of a switch), fdb-alone (the only MAC present on a switch port).
show mac collected[-history] pattern
Show all the collected information (assigned, FDB, etc.) about MAC addresses matching pattern.
show ip assigned[-history] [ ip pattern | id number | detail ]
Show information about IPv4 addresses assigned to sanet nodes, matching an IPv4 address pattern or having a specific numeric id.
show ip arp[-history] [ ip pattern | mac pattern | id number | detail ]
Show information about ARP entries, matching an ip or mac pattern, or having a specific numeric id.
show ip reverse-dns[-history] ip pattern | name pattern
Show information about reverse (IPv4 to name, i.e. PTR records) DNS entries, matching by ip or name.
show ip direct-dns[-history] ip pattern | name pattern
Show information about direct (name to IPv4, i.e. A records) DNS entries, matching by ip or name.
show ip netbios-names[-history] ip pattern | name pattern
Show NetBIOS names collected from nodes, matching by ip or name.
show ip collected[-history] pattern
Show all the information collected about IPv4 addresses matching pattern.
show nodes name used-switch-ports[-history]
Show the collected information about usage of ports of a sanet node marked as switch.
show nodes name bridge-ports[-history]
Show the collected information about bridge instance numbers (brinst) and corresponding ifindex, for ports of a sanet node marked as switch.
collector plug [ ip-assigned | mac-assigned | ip-arp | mac-fdb | used-switch-ports | direct-dns | reverse-dns | netbios-names | bridge-ports | all ] begin end
Combine all collection entries having compatible information and meeting a begin end interval into a single entry.
This command is used when for some reason there are global "holes" in the collected entries that you want to "plug", e.g. because the poller was stopped or because of a major outage.
The command will search all entries of a given kind (select with one of the available keywords, or all to plug holes of all kinds) having a compatible content and meeting the interval, i.e. ending after begin and starting before end. The command will then combine all the compatible entries it found in a single entry, the minimum of the begin values as its begin, and the maximum of the end values as its end.
Example of usage and effects:
sanet# sh mac fdb mac E0CB4E29550C ID MAC BRINST BEGIN END NODE NODE_ID --- ------------ ------- ------------------- ------------------- ------------- ------- 58 E0CB4E29550C 26 2011-06-20 10:45:22 2011-06-21 23:48:14 labs-sw2524-2 9 57 E0CB4E29550C 1 2011-06-20 10:45:20 2011-06-21 23:48:16 labs-sw2524-1 8 116 E0CB4E29550C 26 2011-07-06 06:45:41 2011-07-06 18:55:17 labs-sw2524-2 9 21 E0CB4E29550C 26 2011-06-16 14:56:36 2011-06-18 18:06:14 labs-sw2524-2 9 18 E0CB4E29550C 1 2011-06-16 14:56:36 2011-06-18 18:06:14 labs-sw2524-1 8 77 E0CB4E29550C 1 2011-06-27 20:26:49 2011-06-29 22:16:31 labs-sw2524-1 8 76 E0CB4E29550C 26 2011-06-27 20:26:47 2011-06-29 22:16:34 labs-sw2524-2 9 91 E0CB4E29550C 14 2011-06-28 10:08:15 2011-06-28 10:08:15 labs-sw2524-2 9 92 E0CB4E29550C 26 2011-06-28 10:08:17 2011-06-28 10:08:17 labs-sw2524-1 8 115 E0CB4E29550C 1 2011-07-06 06:45:41 2011-07-06 18:55:18 labs-sw2524-1 8 sanet# collector plug mac-fdb "2011-06-16 16:00:00" "2011-06-30 00:00:00" sanet# sh mac fdb mac E0CB4E29550C ID MAC BRINST BEGIN END NODE NODE_ID --- ------------ ------- ------------------- ------------------- ------------- ------- 116 E0CB4E29550C 26 2011-07-06 06:45:41 2011-07-06 18:55:17 labs-sw2524-2 9 77 E0CB4E29550C 1 2011-06-16 14:56:36 2011-06-29 22:16:31 labs-sw2524-1 8 76 E0CB4E29550C 26 2011-06-16 14:56:36 2011-06-29 22:16:34 labs-sw2524-2 9 91 E0CB4E29550C 14 2011-06-28 10:08:15 2011-06-28 10:08:15 labs-sw2524-2 9 92 E0CB4E29550C 26 2011-06-28 10:08:17 2011-06-28 10:08:17 labs-sw2524-1 8 115 E0CB4E29550C 1 2011-07-06 06:45:41 2011-07-06 18:55:18 labs-sw2524-1 8
The collector plug performs a complex operation on an amount of data that can be extremely large, so it can take an extremely long time to complete (days have been observed in extreme cases). It can be run in parallel with the poller, the only issue being the load on CPU, RAM and I/O: it will not "lock out" the poller. If the command is interrupted with ctrl-c or by other means, no plugging will be done, i.e. plugging holes is an atomic operation.
collector statistics node node-name
Display statistics abount the collection of information from a sanet node. This command is used to troubleshoot slow collector passes or missing information from a node.
For each category of information collected from a sanet node (assigned ip4, port information (usage and mac addresses), arp, fdb) the command will display:
- count of collected items (MAC addresses, ARP entries, ports, etc.)
- seconds elapsed
- time of completion
- Last exception occurred
- Time of last exception
Empty strings or "None" mean that the information is not available.
collector lookup netbios IPv4 address
This command performs an immediate lookup of NetBIOS names from a given IPv4 address, and display the results. It doesn't store the information in any way.
