From BroWiki

In this chapter we detail the different analyzers that Bro provides. Some analyzers look at traffic in fairly generic terms, such as at the level of TCP or UDP connections. Others delve into the specifics of a particular application that is carried on top of TCP or UDP.

As we use the term here, analyzer primarily refers to Bro's event engine. We use the term script to refer to a set of event handlers (and related functions and variables) written in the Bro language; module to refer to a script that serves primarily to provide utility (helper) functions and variables, rather than event handlers; and handler to denote an event handler written in the Bro language. Furthermore, the standard script is the script that comes with the Bro distribution for handling the events generated by a particular analyzer.

Note: However, we also sometimes use "analyzer" to refer to the event handler that processes events generated by the event engine.

We characterize the analyzers in terms of what events they generate, but don't here go into the details of how they generate the events (i.e., the nitty gritty C++ implementations of the analyzers).

Contents 1 Activating an Analyzer 1.1 Loading Analyzers 1.2 Filtering 2 Module Facility 3 General Processing Events 4 Generic Connection Analysis 4.1 The connection record 4.2 Definitions of connections 4.3 Generic TCP connection events 4.4 The tcp analyzer 4.5 The udp analyzer 4.6 Connection summaries 4.7 Connection functions 5 Site-specific information 5.1 Site variables 5.2 Site-specific functions 6 The hot Analyzer 6.1 hot variables 6.2 hot functions 7 The scan Analyzer 7.1 scan variables 7.2 scan functions 7.3 scan event handlers 8 The port-name Analysis Script 9 The brolite Analysis Script 10 The alarm Analysis Script 11 The active Analysis Script 12 The demux Analysis Script 13 The dns Analysis Script 13.1 The dns_mapping record 13.2 dns variables 13.3 dns event handlers 14 The finger Analyzer 14.1 finger variables 14.2 finger event handlers 15 The frag Analysis Script 16 The hot-ids Analysis Script 17 The ftp Analyzer 17.1 The ftp_session_info record 17.2 ftp variables 17.3 ftp functions 17.4 ftp event handlers 17.5 ftp notices 18 The http Analyzer 18.1 http variables 18.2 http event handlers 19 The ident Analyzer 19.1 ident variables 19.2 ident event handlers 20 The irc Analyzer 20.1 irc records 20.2 irc variables 20.3 irc event handlers 21 The login Analyzer 21.1 login analyzer confusion 21.2 login variables 21.3 login functions 21.4 login event handlers 22 The pop3 Analyzer 22.1 The pop3_session_info record 22.2 pop3 variables 22.3 pop3 event handlers 23 The portmapper Analyzer 23.1 portmapper variables 23.2 portmapper functions 23.3 portmapper event handlers 24 The analy Analyzer 25 The signature Analysis Script 26 The SSL Analyzer 26.1 The x509 record 26.2 The ssl_connection_info record 26.3 SSL variables 26.4 SSL event handlers 27 The weird Analysis Script 27.1 Actions for "weird" events 27.2 weird variables 27.3 weird functions 27.4 Events handled by conn_weird 27.5 Events handled by conn_weird_addl 27.6 Events handled by flow_weird 27.7 Events handled by net_weird 27.8 Events generated by the standard scripts 27.9 Additional handlers for ``weird events 28 The icmp Analyzer 29 The stepping Analyzer 30 The ssh-stepping Analysis Script 31 The backdoor Analyzer 32 The interconn Analyzer

Activating an Analyzer

In general, Bro will only do the work associated with a particular analyzer if your policy script defines one or more event handlers associated with the analyzer. For example, Bro will instantiate an FTP analyzer only if your script defines an ftp_request or ftp_reply handler. If it doesn't, then when a new FTP connection begins, Bro will only instantiate a generic TCP analyzer for it. This is an important point, because some analyzers can require Bro to capture a large volume of traffic (See Filtering) and perform a lot of computation; therefore, you need to have a way to trade off between the type of analysis you do and the performance requirements it entails, so you can strike the best balance for your particular monitoring needs.

Deficiency: While Bro attempts to instantiate an analyzer if you define a handler for any of the events the analyzer generates, its method for doing so is incomplete: if you only define an analyzer's less mainstream handlers, Bro may fail to instantiate the analyzer.

Loading Analyzers

The simplest way to use an analyzer is to @load the standard script associated with the analyzer. (See load directive for a discussion of @load). However, there's nothing magic about these scripts; you can freely modify or write your own. The only caveat is that some scripts @load other scripts, so the original version may wind up being loaded even though you've also written your own version.

Deficiency: It would be useful to have a mechanism to fully override one script with another.

In this chapter we discuss each of the standard scripts as we discuss their associated analyzers.

Filtering

Most analyzers require Bro to capture a particular type of network traffic. These traffic flows can vary immensely in volume, so different analyzers can cost greatly differing amounts in terms of performance. Bro declares two redefinable tables in pcap.bro that have special interpretations with regard to filtering:

   global capture_filters: table[string] of string &redef;
   global restrict_filters: table[string] of string &redef;

The key strings serve as a user-definable identifier for the filter strings they are associated with. The entries of the capture_filters table define what traffic Bro should capture, while restrict_filters' entries limit what traffic Bro captures. Bro builds the following tcpdump filter from both tables:

('OR' of capture_filters' entries) and ('AND' of restrict_filters' entries)

Thus, repeated Refinements of capture_filters using the += initializer are combined using logical "OR"s, whereas for restrict_filters "AND"s are used. This follows from the tables' respective purposes---capture_filters permits any of its components, while restrict_filters rejects everything that does not comply with all of its components.

If you do not define capture_filters, then its value is set to tcp or udp; if you do not define restrict_filters, then no restriction is in effect.

Here is an example. If you specify:

   redef capture_filters = { ["HTTP"] = "port http" };
   redef restrict_filter = { ["mynet"] = "net 128.3" };

then the corresponding tcpdump filter will be:

   (port http) and (net 128.3)

which will capture only the TCP port 80 traffic that has either a source or destination address belonging to the 128.3 network (i.e., 128.3/16). A more complex example:

   redef capture_filters += { ["DNS"] = "udp port 53" };
   redef capture_filters += { ["FTP"] = "port ftp" };

   redef restrict_filters += { ["foonet"] = "net 128.3" };
   redef restrict_filters += { ["noflood"] = "not host syn-flood.magnet.com" };

yields this tcpdump filter:

   ((udp port 53) or (port ftp)) and ((net 128.3) and (not host syn-flood.magnet.com))

As you add analyzers, the final tcpdump filter can become quite complicated. You can load the predefined print-filter script to print out the resulting filter. This script handles the bro_init event and exits Bro after printing the filter. Its intended use is that you can add it to the Bro command line (bro my-own-script print-filter) when you want to see what filter the script my-own-script winds up using.

There are two particular uses for print-filter. The first is to debug filtering problems. Unfortunately, Bro sometimes uses sufficiently complicated expressions that they tickle bugs in tcpdump's optimizer. You can take the filter printed out for your script and try running it through tcpdump by hand, and then also try using tcpdump's -O option to see if turning off the optimizer fixes the problem. The second use is to provide a shadow backup to Bro: that is, a version of tcpdump running either on the same machine or a separate machine that uses the same network filter as Bro. While tcpdump can't perform any analysis of the traffic, the shadow guards against the possibility of Bro crashing, because if it does, you will still have a record of the subsequent network traffic which you can run through Bro for post-analysis.

Module Facility

The module facility implements namespaces. Everything is in some namespace or other. The default namespace is called "GLOBAL" and is searched by default when doing name resolution. The scoping operator is "::" as in C++. You can only access things in the current namespace, things in the GLOBAL namespace, or things that have been explicitly exported from a different namespace. Exported variables and functions still require fully-qualified names. The syntax is as follows:

module foo;  # Sets the current namespace to "foo"
 export {
   int i;
   int j;
 }
 int k;

module bar;
 int i;

 foo::i = 1;
 bar::i = 2;
 print i;    # bar::i (since we're currently in module bar)
 j = 3;      # ERROR: j is exported, but the fully qualified name
             #        foo::j is required
 foo::k = 4; # ERROR: k is not exported

The same goes for calling functions.

One restriction currently in place is that variables not in the "GLOBAL" namespace can't shadow those in GLOBAL, so you can't have:

   module GLOBAL;
   global i: int;

   module other_module;
   global i: int;

It is a little confusing that the "global" declaration really only means that the variable i is global to the current module, not that it is truly global and thus visible everywhere (that would require that it be in GLOBAL, or if using the full name is okay, that it be exported). Perhaps there will be a change to the syntax in the future to address this.

The "module" statement cuts across @load commands, so that if you say:

   module foo;
   @load other_script;

then other_script will be in module foo. Likewise if other_script changes to module bar, then the current module will be module bar even after other_script is done. However, this functionality may change in the future if it proves problematic.

The policy scripts in the Bro distribution have not yet been updated to use it, but there is a backward-compatibility feature so that existing scripts should work without modification. In particular, everything is put in GLOBAL by default.

General Processing Events

Bro provides the following events relating to its overall processing:

• bro_init () is generated when Bro first starts up. In particular, after Bro has initialized the network (or initialized to read from a save file) and executed any initializations and global statements, and just before Bro begins to read packets from the network input source(s).

• net_done (t: time) generated when Bro has finished reading from the network, due to either having exhausted reading the save file(s), or having received a terminating signal (See General Processing Events). t gives the time at which network processing finished. This event is generated before bro_done. Note: If Bro terminates due to an invocation of exit, then this event is not generated.

Deficiency: This event is generated on a terminating signal even if Bro is not reading network traffic.

• bro_done () generated when Bro is about to terminate, either due to having exhausted reading the save file(s), receiving a terminating signal (See General Processing Events), or because Bro was run without the network input source and has finished executing any global statements.
  This event is generated after net_done. If you have cleanup that only needs to be done when processing network traffic, it likely is better done using net_done. Note: If Bro terminates due to an invocation of exit, then this event is not generated.

• bro_signal (signal: count) generated when Bro receives a signal. Currently, the signals Bro handles are SIGTERM, SIGINT, and SIGHUP.
  Receiving either of the first two terminates Bro, though if Bro is in the middle of processing a set of events, it first finishes with them before shutting down. The shutdown leads to invocations of net_done and bro_done, in that order. Upon receiving SIGHUP, Bro invokes flush_all (in addition to your handler, if any).

Deficiency: When receiving a signal during event processing, Bro fails to invoke bro_signal, clearly a bug.

• net_stats_update (t: time, ns: net_stats) This event includes two arguments, t, the time at which the event was generated, and ns, a net_stats record, as defined in the example below. Regarding this second parameter, the pkts_recvd field gives the total number of packets accepted by the packet filter so far during this execution of Bro; pkts_dropped gives the total number of packets reported dropped by the kernel; and interface_drops gives the total number of packets reported by the kernel as having been dropped by the network interface.
  Note: An important consideration is that, as shown by experience, the kernel's reporting of these statistics is not always accurate. In particular, the $pkts_dropped statistic is sometimes missing actual packet drops, and some operating systems do not support the interface_drops statistic at all. See the ack_above_hole event for an alternate way to detect if packets are being dropped.

type net_stats: record {
   # All counts are cumulative.
   pkts_recvd: count;       # Number of packets received so far.
   pkts_dropped: count;     # Number of packets *reported* dropped.
   interface_drops: count;  # Number of drops reported by interface(s).
};

Generic Connection Analysis

The conn analyzer performs generic connection analysis: connection start time, duration, sizes, hosts, and the like. You don't in general load analyzer directly, but instead do so implicitly by loading the tcp, udp, or icmp analyzers. Consequently, analyzer doesn't configure capture_filters by itself, but instead uses whatever is set up by these more specific analyzers.

conn analyzes a number of events related to connections beginning or ending. We first describe the connection record data type that keeps track of the state associated with each connection (See connection record), and then we detail the events in Generic TCP connection events. The main output of its analysis are one-line connection summaries, and in Connection functions we give an overview of the different callable functions provided by conn.

conn also loads three other Bro modules: the hot and scan analyzers, and the port_name utility module.

The connection record

type conn_id: record {
   orig_h: addr;  # Address of originating host.
   orig_p: port;  # Port used by originator.
   resp_h: addr;  # Address of responding host.
   resp_p: port;  # Port used by responder.
};

type endpoint: record {
   size: count;  # Bytes sent by this endpoint so far.
   state: count; # The endpoint's current state.
};

type connection: record {
   id: conn_id;        # Originator/responder addresses/ports.
   orig: endpoint;     # Endpoint info for originator.
   resp: endpoint;     # Endpoint info for responder.
   start_time: time;   # When the connection began.
   duration: interval; # How long it was active (or has been so far).
   service: string;    # The service we associate with it (e.g., "http").
   addl: string;       # Additional information associated with it.
   hot: count;         # How many times we've marked it as sensitive.
};

A connection record record holds the state associated with a connection, as shown in the example above. Its first field, id, is defined in terms of the conn_id record, which has the following fields:

• orig_h: The IP address of the host that originated (initiated) the connection. In "client/server" terminology, this is the "client."

• orig_p: The TCP or UDP port used by the connection originator (client). For ICMP "connections," it is set to 0 icmp Analyzer.

• resp_h: The IP address of the host that responded (received) the connection. In "client/server" terminology, this is the "server."

• resp_p: The TCP or UDP port used by the connection responder (server). For ICMP "connections," it is set to 0 icmp Analyzer.

The orig and resp fields of a connection record both hold endpoint record values, which consist of the following fields:

• size: How many bytes the given endpoint has transmitted so far. Note that for some types of filtering, the size will be zero until the connection terminates, because the nature of the filtering is to discard the connection's intermediary packets and only capture its start/stop packets.

• state: The current state the endpoint is in with respect to the connection. The table below defines the different possible states for TCP and UDP connections.

Deficiency: The states are currently defined as count, but should instead be an enumerated type; but Bro does not yet support enumerated types.

Note: UDP "connections" do not have a well-defined structure, so the states for them are quite simplistic. See Definitions of connections for further discussion.

The remaining fields in a connection record are:

• start_time: The time at which the first packet associated with this connection was seen.

• duration : How long the connection lasted, or, if it is still active, how long since it began.

• service: The name of the service associated with the connection. For example, if $id$resp_p is tcp/80, then the service will be "http". Usually, this mapping is provided by the global variable, perhaps via the endpoint_id function; but the service does not always directly correspond to $id$resp_p, which is why it's a separate field. In particular, an FTP data connection can have a service of "ftp-data" even though its $id$resp_p is something other than tcp/20 (which is not consistently used by FTP servers). If the name of the service has not yet been determined, then this field is set to an empty string.

• addl: Additional information associated with the connection. For example, for a login connection, this is the username associated with the login. If no additional information is yet associated with this connection, then this field is set to an empty string.

Deficiency: A significant deficiency associated with the addl field is that it is simply a string without any further structure. In practice, this has proven too restrictive. For example, we may well want to associate an unambiguous username with a login session, and also keep track of the names associated with failed login attempts. (See the login analyzer for an example of how this is implemented presently.) What's needed is a notion of union types which can then take on a variety of values in a type-safe manner.

• hot: How many times this connection has been marked as potentially sensitive or reflecting a break-in. The default value of 0 means that so far the connection has not been regarded as ``hot.

Note: Bro does not presently make fine-grained use of this field; the standard scripts alarm on connections with a non-zero hot field, and do not in general alarm on those that do not, though there are exceptions. In particular, the hot field is not rigorously maintained as an indicator of trouble; it instead is used loosely as an indicator of particular types of trouble (access to sensitive hosts or usernames).

Definitions of connections

Connections for TCP are well-defined, because establishing and terminating a connection plays a central part of the TCP protocol. Beyond those, Bro enforces a hard connection timeout after the period of time specified through the tcp_inactivity_timeout variable, defined in bro.init.

For UDP, a connection begins when host A sends a packet to host B for the first time, B never having sent anything to A. This transmission is termed a request, even if in fact the application protocol being used is not based on requests and replies. If B sends a packet back, then that packet is termed a reply. Each packet A or B sends is another request or reply. UDP connection timeouts are specified through the udp_inactivity_timeout variable, defined in bro.init.

For ICMP, Bro likewise creates a connection the first time it sees an ICMP packet from A to B, even if B previously sent a packet to A, because that earlier packet would have been for a different transport connection than the ICMP itself---the ICMP will likely refer to that connection, but it itself is not part of the connection. For simplicity, this holds even for ICMP ECHOs and ECHO_REPLYs; if you want to pair them up, you need to do so explicitly in the policy script. ICMP connection timeouts are specified through the icmp_inactivity_timeout variable, defined in bro.init.

Generic TCP connection events

There are a number of generic events associated with TCP connections, all of which have a single connection record as their argument:

• new_connection: Generated whenever state for a new (TCP) connection is instantiated.

Note: Handling this event is potentially expensive. For example, during a SYN flooding attack, every spoofed SYN packet will lead to a new new_connection event.

• connection_established: Generated when a connection has become established, i.e., both participating endpoints have agreed to open the connection.

• connection_attempt: Generated when the originator (client) has unsuccessfully attempted to establish a connection. "Unsuccessful" is defined as at least ATTEMPT_INTERVAL seconds having elapsed since the client first sent a connection establishment packet to the responder (server), where ATTEMPT_INTERVAL is an internal Bro variable which is presently set to 300 seconds.

Deficiency: This variable should be user-settable. If you want to immediately detect that a client is attempting to connect to a server, regardless of whether it may soon succeed, then you want to handle the new_connection event instead.

Note: Handling this event is potentially expensive. For example, during a SYN flooding attack, every spoofed SYN packet will lead to a new connection_attempt event, albeit delayed by ATTEMPT_INTERVAL.

• partial_connection: Generated when both connection endpoints enter the TCP_PARTIAL state This means that we have seen traffic generated by each endpoint, but the activity did not begin with the usual connection establishment.

Deficiency: For completeness, Bro's event engine should generate another form of partial_connection event when a single endpoint becomes active (see new_connection below). This hasn't been implemented because our experience is network traffic often contains a great deal of "crud", which would lead to a large number of these really-partial events. However, by not providing the event handler, we miss an opportunity to detect certain forms of stealth scans until they begin to elicit some form of reply.

TCP and UDP connection states, as stored in an endpoint record

State	Meaning
TCP_INACTIVE	The endpoint has not sent any traffic.
TCP_SYN_SENT	It has sent a SYN to initiated a connection.
TCP_SYN_ACK_SENT	It has sent a SYN ACK to respond to a connection request.
TCP_PARTIAL	The endpoint has been active, but we did not see the beginning of the connection.
TCP_ESTABLISHED	The two endpoints have established a connection.
TCP_CLOSED	The endpoint has sent a FIN in order to close its end of the connection.
TCP_RESET	The endpoint has sent a RST to abruptly terminate the connection.
UDP_INACTIVE	The endpoint has not sent any traffic.
UDP_ACTIVE	The endpoint has sent some traffic.

• connection_finished: Generated when a connection has gracefully closed.

Note: This event is triggered after the second FIN has been seen, not the potential final ACK following it. In case you see new_connection events for the final ACK, make sure that tcp_close_delay is large enough (for example loading the heavy-analysis.bro policy, which bumps up several timeout values).

• connection_rejected: Generated when a server rejects a connection attempt by a client.

Note: This event is only generated as the client attempts to establish a connection. If the server instead accepts the connection and then later aborts it, a connection_reset event is generated (see below). This can happen, for example, due to use of TCP Wrappers.

Note: Per the discussion above, a client attempting to connect to a server will result in one of connection_attempt, connection_established, or connection_rejected; they are mutually exclusive.

• connection_half_finished : Generated when Bro sees one endpoint of a connection attempt to gracefully close the connection, but the other endpoint is in the TCP_INACTIVE state. This can happen due to split routing, in which Bro only sees one side of a connection.

• connection_reset: Generated when one endpoint of an established connection terminates the connection abruptly by sending a TCP RST packet.

• connection_partial_close : Generated when a previously inactive endpoint attempts to close a connection via a normal FIN handshake or an abort RST sequence. When it sends one of these packets, Bro waits PARTIAL_CLOSE_INTERVAL (an internal Bro variable set to 10 seconds) prior to generating the event, to give the other endpoint a chance to close the connection normally.

• connection_pending: Generated for each still-open connection when Bro terminates.

The tcp analyzer

The general tcp analyzer lets you specify that you're interested in generic connection analysis for TCP. It simply @load's conn and adds the following to :

   tcp[13] & 0x7 != 0

which instructs Bro to capture all TCP SYN, FIN and RST packets; that is, the control packets that delineate the beginning (SYN) and end (FIN) or abnormal termination (RST) of a connection.

The udp analyzer

The general udp analyzer lets you specify that you're interested in generic connection analysis for UDP. It @load's both hot and conn, and defines two event handlers:

• udp_request (u: connection): Invoked whenever a UDP packet is seen on the forward (request) direction of a UDP connection. See Definitions of connections for a discussion of how Bro defines UDP connections.
  The analyzer invokes check_hot with a mode of CONN_ATTEMPTED and then record_connections to generate a connection summary (necessary because Bro does not time out UDP connections, and hence cannot generate a connection-attempt-failed event).

• udp_reply (u: connection): Invoked whenever a UDP packet is seen on the reverse (reply) direction of a UDP connection. See Definitions of connections for a discussion of how Bro defines UDP connections.
  The analyzer invokes check_hot with a mode of CONN_ESTABLISHED and then again with a mode of CONN_FINISHED to cover the general case that the reply reflects that the connection was both established and is now complete. Finally, it invokes to generate a connection summary.

Note: The standard script does not update capture_filters to capture UDP traffic. Unlike for TCP, where there is a natural generic filter that captures only a subset of the traffic, the only natural UDP filter would be simply to capture all UDP traffic, and that can often be a huge load.

Connection summaries

The main output of conn is a one-line ASCII summary of each connection. By tradition, these summaries are written to a file with the name conn.tag.log, where tag uniquely identifies the Bro session generating the logs.

The summaries are produced by the record_connection function, and have the following format (all reported on a single line):

 <start> <duration> <local IP> <remote IP> <service> <local port> \
 <remote port> <protocol> <org bytes sent>, <res bytes sent> <state> \
 <flags> <tag>

• start: corresponds to the connection's start time, as defined by start_time. * duration: gives the connection's duration, as defined by duration. * local IP, remote IP: correspond to the local and remote addresses that participated in the connection, respectively. The notion of which addresses are local is controlled by the global variable local_nets, which has a default value of empty. If local_nets has not been redefined, then local IP is the connection responder and remote IP is the connection originator. * service: is the connection's service, as defined by service. * local port, remote port: are the ports used by the connection. * org bytes sent res bytes sent: give the number of bytes sent by the originator and responder, respectively. These correspond to the size fields of the corresponding endpoint records. * state: reflects the state of the connection at the time the summary was written (which is usually either when the connection terminated, or when Bro terminated). The different states are summarized in the table below.

Summaries of connection states, as reported in conn.log files

Name	Meaning
S0	Connection attempt seen, no reply.
S1	Connection established, not terminated.
SF	Normal establishment and termination. Note that this is the same symbol as for state S1. You can tell the two apart because for S1 there will not be any byte counts in the summary, while for SF there will be.
REJ	Connection attempt rejected.
S2	Connection established and close attempt by originator seen (but no reply from responder).
S3	Connection established and close attempt by responder seen (but no reply from originator).
RSTO	Connection established, originator aborted (sent a RST).
RSTR	Established, responder aborted.
RSTOS0	Originator sent a SYN followed by a RST, we never saw a SYN-ACK from the responder.
RSTRH	Responder sent a SYN ACK followed by a RST, we never saw a SYN from the (purported) originator.
SH	Originator sent a SYN followed by a FIN, we never saw a SYN ACK from the responder (hence the connection was "half" open).
SHR	Responder sent a SYN ACK followed by a FIN, we never saw a SYN from the originator.
OTH	No SYN seen, just midstream traffic (a "partial connection" that was not later closed).

The ASCII Name given in the Table is what appears in the conn.tag.log log file; it is returned by the conn_state function. The Symbol is used when generating human-readable versions of the file---see hot-report script.

For UDP connections, the analyzer reports connections for which both endpoints have been active as SF; those for which just the originator was active as S0; those for which just the responder was active as SHR; and those for which neither was active as OTH (this latter shouldn't happen!).

• flags: reports a set of additional binary state associated with the connection: ** L indicates that the connection was initiated locally, i.e., the host corresponding to A_l initiated the connection. If L is missing, then the host corresponding to A_r initiated the connection. ** U indicates the connection involved one of the networks listed in the neighbor_nets variable. The use of U for this indication (rather than N, say) is historical, as for the most part is the whole notion of "neighbor network." Note that connection can have both L and U set (see next item). ** X is used to indicate that neither the L or U flags is associated with this connection. * tag: Reference tag to log lines containing additional information associated with the connection in other log files, (e.g.: http.log).

Putting all of this together, here is an example of a conn.log connection summary:

931803523.006848 54.3776 http 7320 38891 206.132.179.35 128.32.162.134 RSTO X %103

The connection began at timestamp 931803523.006848 (18:18:43 hours GMT on July 12, 1999; see the cf utility for how to determine this) and lasted 54.3776 seconds. The service was HTTP (presumably; this conclusion is based just on the responder's use of port 80/tcp). The originator sent 7,320 bytes, and the responder sent 38,891 bytes. Because the L flag is absent, the connection was initiated by host 128.32.162.134, and the responding host was 206.132.179.35. When the summary was written, the connection was in the RSTO state, i.e., after establishing the connection and transferring data, the originator had terminated it with a RST (this is unfortunately common for Web clients). The connection had neither the L or U flags associated with it, and there was additional information, summarized by the string %103 (see the http analyzer for an explanation of this information).

Connection functions

We finish our discussion of generic connection analysis with a brief summary of the different Bro functions provided by the conn analyzer:

• conn_size (e: endpoint, is_tcp: bool): string returns a string giving either the number of bytes the endpoint sent during the given connection, or "?" if from the connection state this can't be determined. The is_tcp parameter is needed so that the function can inspect the endpoint's state to determine whether the connection was closed.

• conn_state (c: connection, is_tcp: bool): string returns the name associated with the connection's state, as given in the above table.

• determine_service (c: connection): bool sets the service field of the given connection, using port_names. If you are using the ftp analyzer, then it knows about FTP data connections and maps them to port_names[20/tcp], i.e., "ftp-data".

• full_id_string (c: connection): string returns a string identifying the connection in one of the two following forms. If the connection is in state S0, S1, or REJ, then no data has been transferred, and the format is:

 A_o <state> A_r/<service> <addl>

where A_o is the IP address of the originator ($id$orig_h), state is as given in the Symbol column of the above table. A_r is the IP address of the responder ($id$resp_h), service gives the application service ($service) as set by determine_service, and addl is the contents of the $addl field (which may be an empty string).

Note that the ephemeral port used by the originator is not reported. If you want to display it, use id_string.

So, for example:

 128.3.6.55 > 131.243.88.10/telnet "luser"

identifies a connection originated by 128.3.6.55 to 131.243.88.10's Telnet server, for which the additional associated information is "luser", the username successfully used during the authentication dialog as determined by the analyzer. From the table above we see that the connection must be in state S1, as that's the only state of S0, S1, or REJ that has a > symbol. (We can tell it's not in state SF because the format used for that state differs---see below.)

For connections in other states, Bro has size and duration information available, and the format returned by full_id_string is:

 A_o S_o <state> A_r/<service> S_r D_s <addl>

where A_o, A_r, state, service, and addl are as before, S_o and S_r give the number of bytes transmitted so far by the originator to the responder and vice versa, and D gives the duration of the connection in seconds (reported with one decimal place) so far.

An example of this second format is:

   128.3.6.55 63b > 131.243.88.10/telnet 391b 39.1s "luser"

which reflects the same connection as before, but now 128.3.6.55 has transmitted 63 bytes to 131.243.88.10, which has transmitted 391 bytes in response, and the connection has been active for 39.1 seconds. The ">" indicates that the connection is in state SF.

• id_string (id: conn_id): string returns a string identifying the connection by its address/port quadruple. Regardless of the connection's state, the format is:

 A_o / P_o  > A_r / P_r

where A_o and A_r are the originator and responder addresses, respectively, and P_o and P_r are representations of the originator and responder ports as returned by the port-name module, i.e., either or a string like "http" for a well-known port such as 80/tcp.

An example:

 128.3.6.55/2244 > 131.243.88.10/telnet

Note, id_string is implemented using a pair of calls to endpoint_id.

Deficiency: It would be convenient to have a form of id_string that can incorporate a notion of directionality, for example 128.3.6.55/2244 < 131.243.88.10/telnet to indicate the same connection as before, but referring specifically to the flow from responder to originator in that connection (indicated by using "<" instead of ">").

• log_hot_conn (c: connection) logs a real-time SensitiveConnection alarm of the form:

 hot:  < connection-id >

where connection-id is the format returned by full_id_string. log_hot_conn keeps track of which connections it has logged and will not log the same connection more than once.

• record_connection (c: connection, disposition: string) Generates a connection summary to the @file{conn} file in the format described in Connection summaries. If the connection's hot field is positive, then also logs the connection using log_hot_conn. The disposition is a text description of the connection's state, such as "attempt" or "half_finished"; it is not presently used.

• service_name (c: connection): string returns a string describing the service associated with the connection, computed as follows. If the responder port ($id$resp_p), p, is well-known, that is, in the port_names table, then p's entry in the table is returned (such as "http" for TCP port 80). Otherwise, for TCP connections, if the responder port is less than 1024, then priv-p is returned, otherwise other-p. For UDP connections, the corresponding service names are upriv-p and uother-p.

• terminate_connection (c: connection) Attempts to terminate the given connection using the rst utility in the current directory. It does not check to see whether the utility is actually present, so an unaesthetic shell error will appear if the utility is not available. rst terminates connections by forging RST packets. It is not presently distributed with Bro, due to its potential for disruptive use.

If Bro is reading a trace file rather than live network traffic, then terminate_connection logs the rst invocation but does not actually invoke the utility. In either case, it finishes by logging that the connection is being terminated.

Site-specific information

The site analyzer is not actually an analyzer but simply a set of global variables (and Updateme: one function) used to define a site's basic topological information.

Site variables

The site module defines the following variables, all redefinable:

• local_nets set[net]: Defines which net's Bro should consider as reflecting a local address. Default: empty.

• local_16_nets set[net]: Defines which /16 prefixes Bro should consider as reflecting a local address. Default: empty.

Deficiency: Bro currently is inconsistent regarding when it consults local_nets versus local_16_nets, so you should ensure that this variable and the previous one are always consistent.

• local_24_nets set[net]: The same, but for /24 addresses. Default: empty.

• neighbor_nets set[net]: Defines which net's Bro should consider as reflecting a "neighbor." Neighbors networks can be treated specially in some policies, distinct from other non-local addresses. In particular, will not drop connectivity to an address belonging to a neighbor.
  The notion is somewhat historical, as is the use of ``U to mark neighbors in connection summaries (See Connection summaries). Default: empty.

• neighbor_16_nets set[addr]: Defines which /16 addresses Bro should consider as reflecting a neighbor; the only use of this variable in the standard scripts is that a scan originating from an address with one of these prefixes will not be dropped. Default: empty.

Deficiency: The name is poorly chosen and should be changed to better reflect this use.

Deficiency: In addition, this variable should be kept consistent with neighbor_nets, until the fine day when the processing is rectified to only use one variable.

• neighbor_24_nets set[net]: The same, but for /24 addresses. Default: empty.

Site-specific functions

Currently, the site module only defines one function:

• is_local_addr (a: addr): bool returns true if the given address belongs to one of the "local" networks, false otherwise.

Fixme: Currently, the test is made by masking the address to /16 and /24 and comparing it to local_16_nets and local_24_nets.

The hot Analyzer

The standard hot script defines policy relating to fairly generic notions of allowed and prohibited connections. It defines a number of variables that you will need to refine to customize your site's policies. It also provides two functions for checking connections against the policies, which can be used by other of the standard scripts.

hot variables

The standard hot script defines the following variables, all redefinable:

• same_local_net_is_spoof : bool If true, then a connection with a local originator address and a local responder address is considered by to have been spoofed.

Deficiency: The name is poorly chosen (and may be changed in the future) to something more accurate like both_local_nets_is_spoof.

In general, you want to use true for a Bro that is monitoring Internet access links (DMZs) and false for internal monitors. Default: F.

• allow_spoof_services : set[port]: Defines a set of services (responder ports) for which Bro should not generate notices if it sees apparent spoofed traffic. Default: 110/tcp (POP version 3; RFC-1939). This default was chosen because in our experience one common form of benign spoof is an off-site laptop attempting to read mail while still configured to use its on-site address.

• allow_pairs : set[addr, addr] Defines pairs of source and destination addresses for which the source is allowed to connect to the destination. The intent with this variable is that the source or destination address will be a sensitive host (such as defined with host_src or host_dsts), for which this particular access should be allowed. Default: empty.

• allow_16_net_pairs : set[addr, addr] Defines pairs of source and destination /16 networks for which the source is allowed to connect to the destination, similar to allow_pairs. Default: empty.

Note: The set is defined in terms of addr's and not net's. So, for example, rather than specifying 128.32., which is a net constant, you'd use 128.32.0.0 (an addr constant).

• hot_srcs : table[addr] of string Defines source addresses that should be considered "hot." A successfully established connection from such a source address generates an alarm, unless one of the access exception variables such as allow_pairs also matches the connection. The value of the table gives an explanatory message as to why the source is hot; for example, "known attacker site". Default: empty.

Note: This value is not currently used, though it aids in documenting the policy script.

Example: redefining hot_srcs using

redef hot_srcs: table[addr] of string = {
   [ph33r.the.eleet.com] = "script kideez",
};

would result in Bro noticing any traffic coming ph33r.the.eleet.com.

• hot_dsts : table[addr] of string Same as hot_srcs, except for destination addresses. Default: empty.

• hot_src_24nets : table[addr] of string Defines /24 source networks should be considered "hot," similar to hot_srcs. Default: empty.

Deficiency: Other network masks, particularly /16, should be provided.

Example: redefining hot_src_24nets using

redef hot_src_24nets: table[addr] of string = {
   [198.81.129.0] = "CIA incoming!",
};

would result in Bro noticing any traffic coming from the 198.81.129/24 network.

• hot_dst_24nets : table[addr] of string same as hot_src_24nets, except for destination networks. Default: empty.

• allow_services : set[port] Defines a set of services that are always allowed, regardless of whether the source or destination address is "hot." Default: ssh, http, gopher ident, smtp, 20/tcp (FTP data).

Note: The defaults are a bit unusual. They are intended for a quite open site with many services.

• allow_services_to : set[addr, port] Defines a set of services that are always allowed if the server is the given host, regardless of whether the source or destination address is "hot." Default: empty.

Example: redefining allow_services_to using

redef allow_services_to: set[addr, port] += {
   [ns.mydomain.com, [domain, 123/tcp]],
} &redef;

would result in Bro not noticing any TCP DNS or NTP traffic heading to ns.mydomain.com. You might add this if ns.mydomain.com is also in hot_dsts, because in general you want to consider any access (other than DNS or NTP) as sensitive.

• allow_services_pairs : set[addr, addr, port] Defines a set of services that are always allowed if the connection originator is the first address and the responder (server) the second address. Default: empty.

Example: redefining allow_services_pairs using

redef allow_services_pairs: set[addr, addr, port] += {
   [ns2.mydomain.com, ns.mydomain.com, [domain, 123/tcp]],
} &redef;

would result in Bro not noticing any TCP DNS or NTP traffic initiated from ns2.mydomain.com to ns.mydomain.com.

• flag_successful_service : table[port] of string The opposite of allow_services. Defines a set of services that should always be flagged as sensitive, even if neither the source nor the destination address is "hot." The string value in the table gives the reason for why the service is considered hot. Note: Bro currently does not use these explanatory messages. Default: 31337/tcp (a popular backdoor because in stylized lettering it spells ELEET) and 2766/tcp (the Solaris listen service, in our experience rarely used legitimately in wide-area traffic).

Note: Bro can flag these services erroneously when a server happens to run a different service on the same port. For example, if you're not running the FTP analyzer, then Bro won't know that FTP data connections using ephemeral ports in fact belong to legitimate FTP traffic, and will flag any that coincide with these services. A related problem arises when a user has configured their SSH access to tunnel FTP control channels through the FTP connection, but not the corresponding data connections (so they don't pay the expense of encrypting the data transfers), so again Bro can't recognize that the ephemeral ports used for the data connections does not reflect the presumed sensitive service.

Example: redefining flag_successful_service using

redef flag_successful_service: table[port] of string += {
       [1524/tcp] = "popular backdoor",
};

would result in Bro also noticing any successful connection to a server running on TCP port 1524.

• flag_successful_inbound_service : table[port] of string The same as flag_successful_service, except only applies to connections with a remote initiator and a local responder (determined by finding the responder address in local_nets). Default: 1524/tcp (ingreslock, a popular backdoor because an attacker can place an entry for the backdoor in /etc/inetd.conf using a service name rather than a raw port number, and hence more likely to appear legitimate to casual inspection). Note: There's no compelling reason why ingreslock is in this table rather than the more general flag_successful_service, though it does tend to result in a few more false hits than the others, presumably because it's a lower port number, and hence more likely on some systems to be chosen for an ephemeral port.

Note: Symmetry would call for flag_successful_outbound_service. This hasn't been implemented in Bro yet simply because the Bro development site has a threat model structured primarily around external threats.

• terminate_successful_inbound_service : table[port] of string The same as flag_successful_inbound_service, except invokes in an attempt to terminate the connection. Default: empty.

Note: As for flag_successful_inbound_service, it would be symmetric to have terminate_successful_outbound_service, and also to have a more general terminate_successful_service.

• flag_rejected_service table[port] of string Similar to flag_successful_service, except applies to connections that a server rejects. For example, you could detect a particular, failed Linux "mountd" attack by adding 10752/tcp to this table, since that happens to be the port used by the commonly available version of the exploit for its backdoor if the attack succeeds. Note: You would of course likely also want to put 10752/tcp in flag_successful_service; or put the entire flag_rejected_service table into flag_successful_service, as discussed in Inserting tables into tables. Default: none.

Deficiency: It might make sense to have flag_attempted_service, which doesn't require that a server actively reject the connection, but Bro doesn't currently have this.

hot functions

The hot module defines two functions for external use:

• check_spoof (c: connection): bool checks the originator and responder addresses of the given connection to determine if they are both local (and the connection is not explicitly allowed in allow_spoof_services). If so, and if same_local_net_is_spoof is true, then marks the connection as "hot".

The function also checks for a specific denial of service attack, the "Land" attack, in which the addresses are the same and so are the ports. If so, then it generates a event with a name of "Land_attack". It makes this check even if is false.

Returns: true if the connection is now hot (or was upon entry), false otherwise.

• check_hot (c: connection, state: count): bool checks the given connection against the various policy variables discussed above, and bumps the connection's hot field if it matches the policies for being sensitive, and does not match the various exceptions. It also uses check_spoof to see if the connection reflects a possible spoofing attack; and terminates the connection if terminate_successful_service indicates so.
  The caller indicates the connection's state in the second parameter to the function, using one of the values given in the Table below. As noted in the Table, the processing differs depending on the state.

Different connection states to use when calling check hot

State	Meaning	Tests
CONN_ATTEMPTED	Connection attempted, no reply seen. Note that you should also use this value for scans with undetermined state, such as possible stealth scans. For example, connection half_finished does this.	check_spoof
CONN_ESTABLISHED	Connection established. Also used for connections apparently established, per partial_connection.	check_spoof, flag_successful_service, flag_successful_inbound service, allow_services_to, terminate_successful_inbound_service
APPL_ESTABLISHED	The connection has reached application-layer establishment. For example, for Telnet or Rlogin, this is after the user has authenticated.	allow_services_to, allow_service_pairs, allow_pairs, allow_16_net_pairs, hot_srcs, hot_dsts, hot_src_24nets, hot_dst_24nets
CONN_FINISHED	The connection has finished, either cleanly or abnormally (for example, connection_reset.	Same as APPL_ESTABLISHED, if the connection exchanged non-zero amounts of data in both directions, and if the service wasn't one of the ones that generates APPL_ESTABLISHED
CONN_REJECTED	The connection attempt was rejected by the server.	check_spoof, flag_rejected_service

In general, the pattern is to make one call when the connection is first seen, either CONN_ATTEMPTED, CONN_ESTABLISHED, or CONN_REJECTED. If the application is one for which connections should only be considered "established" after a successful pre-exchange between originator and responder, then a subsequent call is made with a state of APPL_ESTABLISHED. The idea here is to provide a way to filter out what are in fact not really successful connections so that they are not analyzed in terms of successful service. Finally, for services that don't use APPL_ESTABLISHED, a call is made instead when the connection finishes for some reason, using state CONN_FINISHED.

Note: This approach delays noticing until the connection is over, which might be later than you want, in which case you may need to edit check_hot to provide the desired functionality.

Returns: true if the connection is now hot (or was upon entry), false otherwise.

The scan Analyzer

The scan analyzer detects connection attempts to numerous machines (address scanning), connection attempts to many different services on the same machine (port scanning), and attempts to access many different accounts (password guessing). The basic methodology is to use tables to keep track of the distinct addresses and ports to which a given host attempts to connect, and to trigger notices when either of these reaches a specified size.

Deficiency: As currently written, the analyzer will not detect distributed scans, i.e., when many sites are used to probe individually just a few, but together a large number, of ports or addresses.

A powerful technique that Bro potentially provides is dropping border connectivity with remote scanning sites, though you must supply the magic script to talk with your router and effect the block. See drop_address below for a discussion of the interface provided.

Note: Naturally, providing this capability means you might become vulnerable to denial-of-service attacks in which spoofed packets are used in an attempt to trigger a block of a site to which you want to have access.

scan variables

In addition to internal variables for its bookkeeping, the analyzer provides the following redefinable variables:

report_peer_scan : set[count] Generate an alarm whenever a remote host (as determined by is_local_address) has attempted to connect to the given number of distinct hosts.

Default: { 100, 1000, 10000, }. So, for example, if a remote host attempts to connect to 3,500 different local hosts, a report will be generated when it makes the 100th attempt, and another when it makes the 1,000th attempt.

• report_outbound_peer_scan : set[count] The same as report_peer_scan, except for connections initiated locally.

Default: { 1000, 10000, }.

• possible_port_scan_thresh : count Initially, port scan detection is done based on how many different ports a given host connects to, regardless of on which hosts. Once this threshold is reached, however, then the analyzer begins tracking ports accessed per-server, which is important for reducing false positives. Note: The reason this variable exists is because it is very expensive to track per-server ports accessed for every active host; this variable limits such tracking to only active hosts contacting a significant number of different ports. Default: 25.

• report_accounts_tried : set[count] Whenever a remote host has attempted to access a number of local accounts present in this set, generate an alarm. Each distinct username/password pair is considered a different access. Default: { 25, 100, 500, }.

• report_remote_accounts_tried : set[count] The same, except for access to remote accounts rather than local ones. Default: { 100, 500, }.

• skip_accounts_tried : set[addr] Do not do bookkeeping for account attempts for the given hosts. Default: empty.

• skip_outbound_services : set[port] Do not do outbound-scanning bookkeeping for connections involving the given services. Default: allow_services, ftp, addl_web (see next item).

• addl_web : set[port] Additional ports that should be considered as Web traffic (and hence skipped for outbound-scan bookkeeping). Default: { 81/tcp, 443/tcp, 8000/tcp, 8001/tcp, 8080/tcp, }.

• skip_scan_sources : set[addr] Hosts that are allowed to address-scan without complaint. Default: scooter.pa-x.dec.com, scooter2.av.pa-x.dec.com (AltaVista crawlers; you get the idea.)

• skip_scan_nets_24 : set[addr, port] /24 networks that are allowed to address scan for the given port without complaint. Default: empty.

• can_drop_connectivity : bool True if the Bro has the capability of dropping connectivity, per drop_address. Default: false.

• shut_down_scans : set[port] Scans of these ports trigger connectivity-dropping (if the Bro is capable of dropping connectivity), unless shut_down_all_scans is defined (next item). Default: empty.

• shut_down_all_scans : bool Ignore shut_down_scans and simply drop all scans regardless of service. Default: false.

• shut_down_thresh : count Shut down connectivity after a host has scanned this many addresses. Default: 100.

• never_shut_down : set[addr] Purported scans from these addresses are never shut down. Default: the root name servers (a.root-servers.net through m.root-servers.net).

scan functions

The standard scan script provides the following functions:

• drop_address (a: addr, msg: string) Drops external connectivity to the given address and generates a notification using the given message.

Dropping connectivity requires all of the following to be true:

• can_drop_connectivity is true. * The address is neither localnor a neighbor (See Site variables). * The address is not in never_shut_down.

If these checks succeed, then the script simply attempts to invoke a shell script drop-connectivity with a single argument, the IP address to block. It is up to you to provide the script, using whatever interface to your router/firewall you have available.

The function does not return a value.

• check_scan (c: connection, established: bool, reverse: bool): bool Updates the analyzer's internal bookkeeping on the basis of the new connection c. If established is true, then the connection was successfully established, otherwise not. If reverse is true, then the function should consider the originator/responder fields in the connection's record as reversed. Note: This last is needed for some unusual new connections that may reflect stealth scanning. For example, when the event engine sees a SYN-ack without a corresponding SYN, it instantiates a new connection with an assumption that the SYN-ack came from the responder (and it missed the initial SYN either due to split routing (See Split routing), a packet drop (See Split routing), or Bro having started running after the initial SYN was sent).

If the originating host's activity matches the policy defined by the variables above, then the analyzer logs this fact, and possibly attempts to drop connectivity to the originating host. The function also schedules an event for 24 hours in the future (or when Bro terminates) to generate a summary of the scanning activity (so if the host continues scanning, you get a report on how many hosts it wound up scanning).

Deficiency: This time interval should be selectable.

Note: Purported scans of the FTP data port (20/tcp) or the ident service (113/tcp) are never reported or dropped, as experience has shown they yield too many false hits.

The function does not return a value.

scan event handlers

The standard scan script defines one event handler:

• account_tried (c: connection, user: string, passwd: string) The given connection made an attempt to access the given username and password. Each distinct username/password pair is considered a new access. The event handler generates an alarm if the access matches the logging policy outlined above.

Note: account_tried events are generated by login and ftp analyzers.

The port-name Analysis Script

The port-name utility module provides one redefinable variable and one callable function:

• port_names : table[port] of string Maps TCP/UDP ports to names for the services associated with those ports. For example, 80/tcp maps to "http". These names are used by the conn analyzer when generating connection logs (See Generic Connection Analysis).

• endpoint_id (h: addr, p: port): string Returns a printable form of the given address/port connection endpoint. The format is either <address>/<service-name> or <address>/<port-number> depending on whether the port appears in port_names.

The brolite Analysis Script

The brolite module is intended to provide a convenient way to run (almost) all of the analyzers. It @load's the following other modules and analyzers: alarm, dns, hot, port-name, frag, tcp, scan, weird, finger, ident, ftp, login and portmapper. So you can run Bro using bro -i in0 brolite to have it analyze traffic on interface in0 using the above analyzers; or you can @load brolite to load in the above analyzers.

Note: The brolite analyzer doesn't load http (because it can prove a very high load for many sites) nor experimental analyzers such as stepping or backdoor.

The alarm Analysis Script

The alarm utility module redefines a single variable:

• bro_alarm_file : file A special Bro variable used internally to specify a file where Bro should record messages logged by alarm statements (as well as generating real-time notifications via syslog).

Default: if the $BRO_LOG_SUFFIX environment variable is defined, then alarm.@code{<$BRO_LOG_SUFFIX>}, otherwise alarm.log.

See bro_alarm_file for further discussion.

If you do not include this module, then Bro records alarm messages to stderr.

Here is a sample definition of alarm_hook:

global msg_count: table[string] of count &default = 0;

event alarm_summary(msg: string)
   {
   alarm fmt("(%s) %d times", msg, msg_count[msg]);
   }

function alarm_hook(msg: string): bool
   {
   if ( ++msg_count[msg] == 1 )
       # First time we've seen this message - log it.
       return T;

   if ( msg_count[msg] == 5 )
       # We've seen it five times, enough to be worth
       # summarizing.  Do so five minutes from now,
       # for whatever total we've seen by then.
       schedule +5 min { alarm_summary(msg) };

   return F;
   }

You can also control Bro's alarm processing by defining the special function alarm-hook. It takes a single argument, msg: string, the message in a just-executed alarm statement, and returns a boolean value: true if Bro should indeed log the message, false if not. The above example shows a definition of alarm_hook that checks each alarm message to see whether the same text has been logged before. It only logs the first instance of a message. If a message appears at least five times, then it schedules a future alarm_summary event for 5 minutes in the future; the purpose of this event is to summarize the total number of times the message has appeared at that point in time.

The active Analysis Script

The active utility module provides a single, non-redefinable variable that holds information about active connections:

• active_conn : table[conn_id] of connection Indexed by a conn_id giving the originator/responder addresses/ports, returns the connection's connection record. As usual, accessing the table with a non-existing index results in a run-time error, so you should first test for the presence of the index using the in operator.

Default: empty.

This functionality is quite similar to that of the active_connection function, and Deficiency:arguably this module should be removed in favor of the function. It does, however, provide a useful example of maintaining bookkeeping by defining additional handlers for events that already have handlers elsewhere.

The demux Analysis Script

The demux utility module provides a single function:

• demux_conn (id: conn_id, tag: string, otag: string, rtag: string): bool Instructs Bro to write (``demultiplex) the contents of the connection with the given id to a pair of files whose names are constructed out of tag, otag, and rtag, as follows.

The originator-to-responder direction of the connection goes into a file named:

<otag >.<tag >.<orig-addr >.<orig-port >-<resp-addr >.<resp-port >

and the other direction in:

<rtag >.<tag >.<resp-addr >.<resp-port >-<orig-addr >.<orig-port >

Accordingly, tag can be used to associate a unique label with the pair of files, while otag and rtag provide distinct labels for the two directions.

If Bro is already demuxing the connection, or if the connection is not active, then nothing happens, and the function returns false. Otherwise, it returns true.

Bro places demuxed streams in a directory defined by the redefinable global demux_dir, which defaults in the usual fashion to open_log_file("xscript").

Deficiency:Experience has shown that it would be highly convenient if Bro would demultiplex the entire connection contents into the files, instead of just the part of the connection seen subsequently after the call to demux_conn. One way to do this would be for demux_conn to offset the contents in the file by the current stream position, and then to invoke a utility tool that goes through the Bro output trace file and copies the contents up to the current stream position to the front of the file. This utility tool might even be another instance of Bro running with suitable arguments.}

The dns Analysis Script

The dns module deals with Bro's internal mapping of hostnames to/from IP addresses. Deficiency: There is no DNS protocol analyzer available at present. Furthermore, Deficiency: the lookup mechanisms discussed here are not available to the Bro script writer, other than implicitly by using hostnames in lieu of addresses in variable initializations (see [[Reference Manual: #Hostnames vs addresses).|Hostnames vs addresses).]]

The module's function is to handle different events that can occur when Bro resolves hostnames upon startup. Bro maintains its own cache of DNS information which persists across invocations of Bro on the same machine and by the same user. The role of the cache is to allow Bro to resolve hostnames even in the face of DNS outages; the philosophy is that it's better to use old addresses than none at all, and this helps harden Bro against attacks in which the attacker causes DNS outages in order to prevent Bro from resolving particular sensitive hostnames (e.g., hot_srcs ). The cache is stored in the file ``.bro-dns-cache in the user's home directory. You can delete this file whenever you want, for example to purge out old entries no longer needed, and Bro will recreate it next time it's invoked using -P.

Currently, all of the event handlers are invoked upon comparing the results of a new attempt to look up a name or an address versus the results obtained the last time Bro did the lookup. When Bro looks up a name for the first time, no events are generated.

Also, Bro currently only looks up hostnames to map them to addresses. It does not perform inverse lookups.

The dns_mapping record

All of the events handled by the module include at least one record of DNS mapping information, defined by the dns_mapping type shown in the example below. The corresponding fields are:

• creation_time When the mapping was created.

• req_host The hostname looked up, or an empty string if this was not a hostname lookup.

• req_addr The address looked up (reverse lookup), or 0.0.0.0 if this was not an address lookup.

• valid True if an answer was received for a lookup (even if the answer was that the request name or address does not exist in the DNS).

• hostname The hostname answer in response to an address lookup, or the string "@code{<none>"} if an answer was received but it indicated there was no PTR record for the given address.

• addrs A set of addresses in response to a hostname lookup. Empty if an answer was received but it indicated that there was no A record for the given hostname.

type dns_mapping: record {
   creation_time: time;  # When the mapping was created.

   req_host: string;     # The hostname in the request, if any.
   req_addr: addr;       # The address in the request, if any.

   valid: bool;          # Whether we received an answer.
   hostname: string;     # The hostname in the answer, or "<none>".
   addrs: set[addr];     # The addresses in the answer, if any.
};

dns variables

The modules provides one redefinable variable:

• dns_interesting_changes : set[string] The different DNS events have names associated with them. If the name is present in this set, then the event will generate a notice, otherwise not.

One exception to this list is that DNS changes involving the loopback address 127.0.0.1 are always considered notice-worthy, since they may reflect DNS corruption.

Default: { "unverified", "old name", "new name", "mapping", }.

dns event handlers

The DNS module supplies the following event handlers:

• dns_mapping_valid (dm: dns_mapping) The given request was looked up and it was identical to its previous mapping.

• dns_mapping_unverified (dm: dns_mapping) The given request was looked up but no answer came back.

• dns_mapping_new_name (dm: dns_mapping) In the past, the given address did not resolve to a hostname; this time, it did.

• dns_mapping_lost_name (dm: dns_mapping) In the past, the given address resolved to a hostname; now, that name has gone away. (An answer was received, but it stated that there is no hostname corresponding to the given address.)

• dns_mapping_name_changed (old_dm: dns_mapping, new_dm: dns_mapping) The name returned this time for the given address differs from the name returned in the past.

• dns_mapping_altered (dm: dns_mapping, old_addrs: set[addr], new_addrs: set[addr]) The addresses associated with the given hostname have changed. Those in old_addrs used to be part of the set returned for the name, but aren't any more; while those in new_addrs didn't used to be, but now are. There may also be some unchanged addresses, which are those in dm$addrs but not in new_addrs.

The finger Analyzer

The finger analyzer processes traffic associated with the Finger service RFC-1288. Bro instantiates a finger analyzer for any connection with service port 79/tcp (if you @load the finger analyzer in your script, or define your own finger_request or finger_reply handlers, of course).

The analyzer uses a capture filter of ``port finger (See: Filtering).

In the past, attackers often used Finger requests to obtain information about a site's users, and sometimes to launch attacks of various forms (buffer overflows, in particular). In our experience, exploitation of the service has greatly diminished over the past years (no doubt in part to the service being increasingly turned off, or prohibited by firewalls). Now it is only rarely associated with an attack.

finger variables

The standard script defines two redefinable variables:

• hot_names : set[string] A list of usernames that should be considered sensitive (notice-worthy) if included in a Finger request.

Default: { "root", "lp", "uucp", "nuucp", "demos", "operator", "sync", "guest", "visitor", }.

• max_request_length : count The largest reasonable request size (used to flag possible buffer overflow attacks). Bro marks a connection as ``hot if its request exceeds this length, and truncates its logging of the request to this many bytes, followed by "...".

Default: 80.

finger event handlers

The standard script defines one event handler:

• finger_request (c: connection, request: string, full: bool) Invoked upon connection c having made the request request. The full flag is true if the request included the ``long format option (which the event engine will have removed from the request).

The standard script flags long requests and truncates them as noted above, and then checks whether the request is for a name in hot_names. It then formats the request either by placing double quotation marks around it, or, if the request was empty---indicating a request for information on all users---the request is changed to the string ALL with no quotes around it.

If the originator already made a request, then this additional request is placed in parentheses (though multiple requests violate the Finger protocol). If the request was for the full format, then the text ``(/W) is appended to the request. Finally, the request is appended to the connection's field.

The event engine generates an additional event that the predefined finger script does not handle:

• finger_reply (c: connection, reply_line: string) Generated for each line of text sent in response to the originator's request.

The frag Analysis Script

The frag utility module simply refines the capture filter (See: Filtering) so that Bro will capture and reassemble IP fragments. Bro reassembles any fragments it receives; but normally it doesn't receive any, except the beginnings of TCP fragments (see the tcp module), and UDP port 111 (per the portmapper module).

So, to make Bro do fragment reassembly, you simply use ``load frag. It effects this by adding:

   (ip[6:2] & 0x3fff != 0) and tcp

to the filter. The first part of this expression matches all IP fragments, while the second restricts those matched to TCP traffic. We would like to use:

   (ip[6:2] & 0x3fff != 0) and (tcp or udp port 111)

to also include portmapper fragments, but that won't work---the port numbers will only be present in the first fragment, so the packet filter won't recognize the subsequent fragments as belonging to a UDP port 111 packet, and will fail to capture them.

Note: Alternatively, we might be tempted to use ``(tcp or udp) and so capture all UDP fragments, including port 111. This would work in principle, but in practice can capture very high volumes of traffic due to NFS traffic, which can send all of its file data in UDP fragments.}

The hot-ids Analysis Script

The hot-ids module defines a number of redefinable variables that specify usernames Bro should consider sensitive:

• forbidden_ids set[string] lists usernames that should never be used. If Bro detects use of one, it will attempt to terminate the corresponding connection.

Default: { "uucp", "daemon", "rewt", "nuucp", "EZsetup", "OutOfBox", "4Dgifts", "ezsetup", "outofbox", "4dgifts", "sgiweb", }. All of these correspond to accounts that some systems have enabled by default (with well-known passwords), except for "rewt", which corresponds to a username often used by (weenie) attackers.

Deficiency: The repeated definitions such as "EZsetup" and "ezsetup" reflect that this variable is a set and not a pattern. Consequently, the exact username must appear in it (with a pattern, we could use character classes to match both upper and lower case).

• forbidden_ids_if_no_password : set[string] Same as forbidden_ids except only considered forbidden if the login succeeded with an empty password.

Default: "lp", a default passwordless IRIX account.

• forbidden_id_patterns : pattern A pattern giving user ids that should be considered forbidden. Deficiency: This pattern is currently only used to check Telnet/Rlogin user ids, not ids seen in other contexts, such as FTP sessions.

Default: /(y[o0]u)(r|ar[e3])([o0]wn.*)/, a particularly egregious style of username of which we've observed variants in different break-ins.

• always_hot_ids : set[string] A list of usernames that should always be considered sensitive, though not necessarily so sensitive that they should be terminated whenever used.

Default: { "lp", "warez", "demos", forbidden_ids, }. The "lp" and "demos" accounts are specified here rather than forbidden_ids because it's possible that they might be used for legitimate accounts. "warez" (for ``wares, i.e., bootlegged software) is listed because its use likely constitutes a policy violation, not a security violation.

Note: forbidden_ids is incorporated into always_hot_ids to avoid replicating the list of particularly sensitive ids by listing it twice and risking inconsistencies.

• hot_ids set[string] User ids that generate notices if the user logs in successfully.

Default: { "root", "system", always_hot_ids, }. The ones included in addition to always_hot_ids are only considered sensitive if the user logs in successfully.

The ftp Analyzer

The ftp analyzer processes traffic associated with the FTP file transfer service RFC-959. Bro instantiates an ftp analyzer for any connection with service port 21/tcp, providing you have loaded the ftp analyzer, or defined a handler for ftp_request or ftp_reply.

The analyzer uses a capture filter of ``port ftp (See: Filtering). It generates summaries of FTP sessions; looks for sensitive usernames, access to sensitive files, and possible FTP ``bounce attacks, in which the host specified in a ``PORT or ``PASV directive does not correspond to the host sending the directive; or in which a different host than the server (client) connects to the endpoint specified in a PORT (PASV) directive.

The ftp_session_info record

The main data structure managed by the ftp analyzer is a collection of ftp_session_info records, where the record type is shown below:

type ftp_session_info: record {
   id: count;              # unique number associated w/ session
   user: string;           # username, if determined
   request: string;        # pending request or requests
   num_requests: count;    # count of pending requests
   request_t: time;        # time of request
   log_if_not_denied: bool;        # unless code 530 on reply, log it
   log_if_not_unavail: bool;       # unless code 550 on reply, log it
   log_it: bool;           # if true, log the request(s)
};

The corresponding fields are:

• id The unique session identifier assigned to this session. Sessions are numbered starting at 1 and incrementing with each new session.

• user The username associated with this session (from the initial FTP authentication dialog), or an empty string if not yet determined.

• request The pending request, if the client has issued any. Ordinarily there would be at most one pending request, but a client can in fact send multiple requests to the server all at once, and an attacker could do so attempting to confuse the analyzer into mismatching responses with requests, or simply forgetting about previous requests.

• num_requests A count of how many requests are currently pending.

• request_t The time at which the pending request was issued.

• log_if_not_denied If true, then when the reply to the current request comes in, Bro should log it, unless the reply code is 530 (``denied).

• log_if_not_unavail If true, then when the reply to the current request comes in, Bro should log it, unless the reply code is 550 (``unavail).

• log_it If true, then when the reply to the current request comes in, Bro should log it.

ftp variables

The standard script defines the following redefinable variables:

• ftp_guest_ids : set[string] A set of usernames associated with publicly accessible ``guest services. Bro interprets guest usernames as indicating Bro should use the authentication password as the effective username.

Default: { "anonymous", "ftp", "guest", }.

• ftp_skip_hot : set[addr, addr, string] Entries indicate that a connection from the first given address to the second given address, using the given string username, should not be treated as hot even if the username is sensitive.

Default: empty.

Example: redefining ftp_skip_hot using

redef ftp_skip_hot: set[addr, addr, string] += {
   [[bob1.dsl.home.net, bob2.dsl.home.net], 
     bob.work.com, "root"], };

would result in Bro not noticing FTP connections as user "root" from either bob1.dsl.home.net or bob2.dsl.home.net to the server running on bob.work.com.

• ftp_hot_files : pattern Bro matches the argument given in each FTP file manipulation request (RETR, STOR, etc.) against this pattern to see if the file is sensitive. If so, and if the request succeeds, then the access is logged.

Default: aggdrop a pattern that matches various flavors of password files, plus any string with eggdrop in it. @emph{Note: Eggdrop is an IRC management tool often installed by certain attackers upon a successful break-in.}

• ftp_not_actually_hot_files : pattern A pattern giving exceptions to ftp_hot_files. It turns out that a pattern like /passwd/ generates a lot of false hits, such as from passwd.c (source for the passwd utility; this can turn up in FTP sessions that fetch entire sets of utility sources using MGET) or passwd.html (a Web page explaining how to enter a password for accessing a particular page).

Default: /(passwd|shadow).*@code{.(c|gif|htm|pl|rpm|tar|zip)/} .

• ftp_hot_guest_files pattern Files that guests should not attempt to access.

Default: .rhosts and .forward .

• skip_unexpected : set[addr] If a new host (address) unexpectedly connects to the endpoint specified in a PORT or PASV directive, then if either the original host or the new host is in this set, no message is generated. The idea is that you can specify multi-homed hosts that frequently show up in your FTP traffic, as these can generate innocuous warnings about connections from unexpected hosts.

Default: some hp.com hosts, as an example. Most are specified as raw IP addresses rather than hostnames, since the hostnames don't always consistently resolve.

• skip_unexpected_net : set[addr] The same as skip_unexpected, except addresses are masked to /24 and /16 before looked up in this set.

Default: empty.

In addition, ftp_log holds the name of the FTP log file to which Bro writes FTP session summaries. It defaults to open_log_file("ftp").

Here is an example of what entries in this file look like:

972499885.784104 #26 131.243.70.68/1899 > 64.55.26.206/ftp start
972499886.685046 #26 response (220 tuvok.ooc.com FTP server
   (Version wu-2.6.0(1) Fri Jun 23 09:17:44 EDT 2000) ready.)
972499886.686025 #26 USER anonymous/IEUser@ (logged in)
972499887.850621 #26 TYPE I (ok)
972499888.421741 #26 PASV (227 64.55.26.206/2427)
972499889.493020 #26 SIZE /pub/OB/4.0/JOB-4.0.3.zip (213 1675597)
972499890.135706 #26 *RETR /pub/OB/4.0/JOB-4.0.3.zip, ABOR (complete)
972500055.491045 #26 response (225 ABOR command successful.)

Here we see a transcript of the 26th FTP session seen since Bro started running. The first line gives its start time and the participating hosts and ports. The next line (split across two lines above for clarity) gives the server's welcome banner. The client then logged in as user ``anonymous, and because this is one of the guest usernames, Bro recorded their password too, which in this case was ``IEUser (a useless string supplied by their Web browser). The server accepted this authentication, so the status on the line is ``(logged in).

The client then issues a request for the Image file type, to which the server agreed. Next they issued a PASV directive, and received a response instructing them to connect to the server on port 2427/tcp for the next transfer. At this point, after issuing a SIZE directive (to which the server returned 1,675,597 bytes), they send RETR to fetch the file /pub/OB/4.0/JOB-4.0.3.zip. However, before the transfer completed, they issued ABOR, but the transfer finished before the server processed the abort, so the log shows a status of completed. Furthermore, because the client issued two commands without waiting for an intervening response, these are shown together in the log file, and the line marked with a ``* so it draws the eye. Finally, because Bro paired up the (completed) with the multi-request line, it then treats the response to the ABOR command as a reply by itself, showing in the last line that the server reported it successfully carried out the abort.

The corresponding lines in the @file{conn} file look like:

   972499885.784104 565.836 ftp 118 427 131.243.70.68 64.55.26.206
       RSTO L #26 anonymous/IEUser@
   972499888.984116 165.098 ftp-data ? 1675597 131.243.70.68 

64.55.26.206 RSTO L

The first line summarizes the FTP control session (over which the client sends its requests and receives the server's responses). It includes an addl annotation of ``#26 anonymous/IEUser, summarizing the session number (so you can find the corresponding records in the ftp log file) and the authentication information.

The second line summarizes the single FTP data transfer, of 1,675,597 bytes. The amount of data sent by the client for this connection is shown as unknown because the client aborted the connection with a RST (hence the state RSTO). For connections that Bro does not look inside (such as FTP data transfers), it learns the amount of data transferred from the sequence numbers of the SYN and FIN connection control packets, and can't (reliably) learn them for the sender of a RST. (It can for the receiver of the RST.