From BroWiki

This chapter gives an overview of how to get started with operating Bro: (i) compiling it, (ii) running it interactively, on live network traffic, and on recorded traces, (iii) how Bro locates the policy files it should evaluate and how to modify them, (iv) the arguments you can give it to control its operation, and (v) some helper utilities also distributed with Bro that you'll often find handy.

Contents 1 Running Bro 1.1 Building and installing Bro 1.1.1 Supported platforms 1.1.2 The Bro source code distribution 1.1.3 Installing Bro 1.1.4 Tuning BPF 1.1.4.1 BPF kernel support 1.1.4.2 /dev/bpf devices 1.1.4.3 packet filter permissions 1.1.4.4 large BPF buffers 1.1.4.5 libpcap buffer size patch 1.2 Using Bro interactively 1.3 Specifying policy scripts 1.4 Running Bro on network traffic 1.4.1 Live traffic 1.4.2 Traffic traces 1.5 Running Bro as part of a command pipeline 1.6 Modifying Bro policy 1.7 Bro flags and run-time environment 1.7.1 Flags 1.7.2 Run-time environment 2 Helper utilities 2.1 Scripts 2.2 The hf utility 2.3 The cf utility

Running Bro

This section discusses how to build and install Bro, running it interactively (mostly useful for building up familiarity with the policy language, not for traffic analysis), running it on live and recorded network traffic, modifying Bro policy scripts, and the different run-time flags.

Building and installing Bro

Supported platforms

Bro builds on a number of types of Unix: FreeBSD, Solaris, Linux, MacOS X, though not all versions. It does not build under non-Unix operating systems such as Windows XP, Vista, or NT.

The Bro source code distribution

You can get the latest public release of Bro from the Bro Web page. Bro is distributed as a gzip'd Unix tar archive, which you can unpack using:

tar xfvz tar-file 

or, on some Unix systems, one of the following:

gzcat tar-file | tar xf -
tar zxf tar-file

This creates a subdirectory bro-XXX-version, where the optional XXX may be a tag such as "stable" or "devel", and version reflects a version and possibly a subversion, such as 1.1d20 for version 1.1 and subversion d20.

To build Bro, change to the Bro directory and enter:

 ./configure
 make 

Several configuration options exist; see

 ./configure --help

for a list. Bro configuration is described in more detail in the User Manual.

Installing Bro

You install Bro by issuing:

 make install

Note that you don't need to make install if you only want to make individual, manual runs of Bro, for example to experiment with a set of trace files. In that case, you only need to ensure that your BROPATH environment variable points to the policy directory contained in the distribution. For more details on BROPATH, see Run-time environment below.

Tuning BPF

Bro is written using libpcap, the portable packet-capture library available from http://www.tcpdump.org/release/libpcap-0.9.7.tar.gz. While libpcap knows how to use a wide range of Unix packet filters, it far and away performs most efficiently used in conjunction with the Berkeley Packet Filter (BPF) and with BPF descendants (such as the Digital Unix packet filter). Although BPF is available from ftp://ftp.ee.lbl.gov/bpf.tar.Z, installing it involves modifying your kernel, and perhaps requires significant porting work. However, it comes as part of several operating systems, such as FreeBSD, NetBSD, and OpenBSD.

For BPF systems, you should be aware of the following tuning and configuration issues:

BPF kernel support

You need to make sure that kernel support for BPF is enabled. In addition, some systems default to configuring kernel support for only one BPF device. This often proves to be a headache because it means you cannot run more than one Bro at a time, nor can you run it at the same time as

/dev/bpf devices

Related to the previous item, on BPF systems access to the packet filter is via special /dev/bpf devices, such as /dev/bpf0. Just as you need to make sure that the kernel's configuration supports multiple BPF devices, so to must you make sure that an equal number of device files reside in /dev/.

packet filter permissions

On systems for which access to the packet filter is via the file system, you should consider whether you want to only allow root access, or instead create a Unix group for which you enable read access to the device file(s). The latter allows you to run Bro as a user other than root, which is strongly recommended!

large BPF buffers

While running with BPF is often necessary for high performance, it's not necessarily sufficient. By default, BPF programs use very modest kernel buffers for storing packets, which leads to high context switch overhead as the kernel very often has to deliver packets to the user-level Bro process. Minimizing the overhead requires increasing the buffer sizes. This can make a large difference!

Under FreeBSD, the configuration variable to increase is debug.bpf_bufsize, which you can set via sysctl. We recommend creating a script run at boot-up time that increases it from its small default value to something on the order of 100 KB--2 MB, depending on how fast (heavily loaded) is the link being monitored, and how much physical memory the monitor machine has at its disposal.

libpcap buffer size patch

Important note: some versions of libpcap have internal code that limits the maximum buffer size to 32 KB. For these systems, you should apply the patch included in the Bro distribution in the file libpcap.patch.

Finally, once you have increased the buffer sizes, you should check that running Bro does indeed consume the amount of kernel memory you expect. You can do this under FreeBSD using vmstat -m and searching in the output for the summary of BPF memory. You should find that the MemUse statistic goes up by twice the buffer size for every concurrent Bro or tcpdump you run. <ref>Providing that these programs have been recompiled with the corrected libpcap noted above.</ref> The reason the increase is by twice the buffer size is because Bro uses double-buffering to avoid dropping packets when the buffer fills up.

Using Bro interactively

Once you've built Bro, you can run it interactively to try out simple facets of the policy language. Note that in this mode, Bro is not reading network traffic, so you cannot do any traffic analysis; this mode is simply to try out Bro language features.

You run Bro interactively by simply executing "bro" without any arguments. It then reads from stdin and writes to stdout.

Try typing the following to it:

 print "hello, world";  
 ^D (i.e., end of file)

(The end-of-file is critical to remember. It's also a bit annoying for interactive evaluation, but reflects the fact that Bro is not actually meant for interactive use, it simply works as a side-effect of Bro's structure.)

Bro will respond by printing:

 hello, world

to stdout and exiting.

You can further declare variables and print expressions, for example:

   global a = telnet;
   print a, a > ftp;
   print www.microsoft.com;

will print

   23/tcp, T
   {
   207.46.19.254,
   207.46.19.190,
   207.46.20.60,
   207.46.18.30,
   207.46.19.60,
   207.46.199.30,
   207.46.198.30
   }

where 23/tcp reflects the fact that telnet is a predefined variable whose value is TCP port 23, which is larger than TCP port 21 (i.e., ftp); and the DNS name www.microsoft.com currently returns the above seven addresses.

You can also define functions:

   function top18bits(a: addr): addr
       {
       return mask_addr(a, 18);
       }

   print top18bits(128.3.211.7);

prints

   128.3.192.0

and even event handlers:

   event bro_done()
       {
       print "all done!";
       }

which prints "all done!" when Bro exits.

Specifying policy scripts

Usually, rather than running Bro interactively you want it to execute a policy script or a set of policy scripts. You do so by specifying the names of the scripts as command-line arguments, such as:

   bro ~/my-policy.bro ~/my-additional-policy.bro

Bro provides several mechanisms for simplifying how you specify which policies to run.

First, if a policy file doesn't exist then it will try again using .bro as a suffix, so the above could be specified as:

   bro ~/my-policy ~/my-additional-policy

Second, Bro consults the colon-separated search path to locate policy scripts. If your home directory was listed in $BROPATH, then you could have invoked it above using:

   bro my-policy my-additional-policy

Note: If you define $BROPATH, you must include bro-dir/policy, where bro-dir is where you have built or installed Bro, because it has to be able to locate bro-dir/policy/bro.init to initialize itself at run-time.

Third, the @load directive can be used in a policy script to indicate the Bro should at that point process another policy script (like C's include directive; see ). So you could have in my-policy:

   @load my-additional-policy

and then just invoke Bro using:

   bro my-policy

providing you always want to load my-additional-policy whenever you load my-policy.

Note that the predefined Bro module brolite loads almost all of the other standard Bro analyzers, so you can pull them in with simply:

   @load brolite

or by invoking Bro using "bro brolite my-policy".

Running Bro on network traffic

There are two ways to run Bro on network traffic: on traffic captured live by the network interface(s), and on traffic previously recorded using the -w flag of tcpdump or Bro itself.

Live traffic

Bro reads live traffic from the local network interface whenever you specify the -i flag. As mentioned below, you can specify multiple instances to read from multiple interfaces simultaneously, however the interfaces must all be of the same link type (e.g., you can't mix reading from a Fast Ethernet with reading from an FDDI link, though you can mix a 10 Mbps Ethernet interface with a 100 Mbps Ethernet).

In addition, Bro will read live traffic from the interface(s) listed in the interfaces variable, unless you specify the -r flag (and do not specify -i). So, for example, if your policy script contains:

   const interfaces += "sk0";
   const interfaces += "sk1";

then Bro will read from the sk0 and sk1 interfaces, and you don't need to specify -i.

Traffic traces

To run on recorded traffic, you use the -r flag to indicate the trace file Bro should read. As with -i, you can use the flag multiple times to read from multiple files; Bro will merge the packets from the files into a single packet stream based on their timestamps. Specifying -r - will cause Bro to read trace files from stdin.

The Bro distribution includes an example trace that you can try out, example.ftp-attack.trace. If you invoke Bro using:

   bro -r example.ftp-attack.trace brolite

you'll see that it generates a connection summary to stdout, a summary of the FTP sessions to ftp.example, a copy of what would have been real-time alarms had Bro been running on live traffic to alarm.example, and a summary of unusual traffic anomalies (none in this trace) to weird.example.

Running Bro as part of a command pipeline

You can pipe input into Bro in two ways:

• When not providing additional arguments to Bro, it will read policy code on stdin.
• When using -r -, it will expect libpcap trace data on stdin.

Output generated by the policy layer via the print statement gets sent to stdout, where it can be processed as usual.

Note: Bro ignores SIGPIPE on stdout. This means that Bro will not exit when processes further down the pipeline exit. Make sure to structure your pipelined commands accordingly.

Modifying Bro policy

One way to alter the policy Bro executes is of course to directly edit the scripts. When this can be avoided, however, that is preferred, and Bro provides a pair of related mechanisms to help you specify refinements to existing policies in separate files.

The first such mechanism is that you can define multiple handlers for a given event. So, for example, even though the standard ftp analyzer (bro-dir/policy/ftp.bro) defines a handler for ftp.request, you can define another handler if you wish in your own policy script, even if that policy script loads (perhaps indirectly, via the brolite module) the ftp analyzer. When the event engine generates an ftp_request event, both handlers will be invoked.

Deficiency: Presently, you do not have control over the order in which they are invoked; you also cannot completely override one handler with another, preventing the first from being invoked.

Second, the standard policy scripts are often written in terms of redefinable variables. For example, ftp.bro contains a variable ftp_guest_ids that defines a list of usernames the analyzer will consider to reflect guest accounts:

   const ftp_guest_ids = { "anonymous", "ftp", "guest", } &redef;

While "const" marks this variables as constant at run-time, the attribute "&redef" specifies that its value can be redefined.

For example, in your own script you could have:

   redef ftp_guest_ids = { "anonymous", "guest", "visitor", "student" };

instead. (Note the use of "redef" rather than "const", to indicate that you realize you are redefining an existing variable.)

In addition, for most types of variables you can specify incremental changes to the variable, either new elements to add or old ones to subtract. For example, you could instead express the above as:

   redef ftp_guest_ids += { "visitor", "student" };
   redef ftp_guest_ids -= "ftp";

The potential advantage of incremental refinements such as these are that if any other changes are made to ftp.bro's original definition, your script will automatically inherit them, rather than replacing them if you used the first definition above (which explicitly lists all four names to include in the variable). Sometimes, however, you don't want this form of inheriting later changes; you need to decide on a case-by-case basis, though our experience is that generally the incremental approach works best.

Finally, the use of prefixes provides a way to specify a whole set of policy scripts to load in a particular context. For example, if you set $BRO_PREFIXES to "dialup", then a load of ftp.bro will also load dialup.ftp.bro automatically (if it exists). See Run-time environment for further discussion.

Bro flags and run-time environment

Flags

When invoking Bro, you can control its behavior using a large number of flags and arguments. Most options can be specified using either a more readable long version (starting with two dashes), or a more compact but sometimes less intuitive short version (single dash followed by a single letter). Arguments can be provided after whitespace (i.e., ``-r file.pcap or ``--readfile file.pcap) and also using an equation mark when the long version is used (i.e., ``--readfile=file.pcap). Single-letter flags without arguments can be combined into a single option element (i.e., ``-dWF is the same as ``-d -W -F).

• -d|--debug-policy

Activates policy file debugging. See Interactive Debugger for details.

• -e|--exec <Bro statements>

Adds the given Bro policy statements to the loaded policy. Use for manual refinement, or for verifying the resulting value of a given variable. Note that you can omit trailing semi-colons.

• -f|--filter filter

Use filter as the tcpdump filter for capturing packets, rather than the combination of and restrict_filter, or the default of ``tcp or udp .

• -h|--help|-?

Generate a help message summarizing Bro's options and environment variables, and exit.

• -g|--dump-config

Writes out the current configuration into the persistent state directory configured through the state_dir variable, defined in Bro init file.

• -i|--iface <interface>

Add interface to the list of interfaces from which Bro should read network traffic. You can use this flag multiple times to direct Bro to read from multiple interfaces. You can also, or in addition, use refinements of the variable to specify interfaces.

Note that if no interfaces are specified, then Bro will not read any network traffic. It does not have a notion of a ``default interface from which to read.

• -p|--prefix <prefix>

Add prefix to the list of prefixes searched by Bro when loading a script. You can also, or in addition, use prefix to specify search prefixes. See prefixes for discussion.

• -r|--readfile <readfile>

Add readfile to the list of tcpdump save files that Bro should read. You can use this flag multiple times to direct Bro to read from multiple save files; it will merge the packets read from the different files based on their timestamps. Note that if the save files contain only packet headers and not contents, then of course Bro's analysis of them will be limited.

Note that use of -r is mutually exclusive with use of -i. However, you can use -r when running scripts that refine interfaces, in which case the -r option takes precedence and Bro performs off-line analysis.

• -s|--rulefile <signaturefile>

Add signaturefile to the list of files containing signatures to match against the network traffic. See Signatures for more information.

• -t|--tracefile <tracefile>

Enables tracing of Bro script execution. See Execution tracing.

• -w|--writefile <writefile>

Write a tcpdump save file to the file writefile. Bro will record all of the packets it captures, including their contents, except as controlled by calls to set_record_packets. Note: One exception is that unless you are analyzing HTTP events (for example, by loading the @ refhttp analyzer), Bro does not record the contents of HTTP SYN/FIN/RST packets to the trace file. The reason for this is that HTTP FIN packets often contain a large amount of data, which is not of any interest if you are not using HTTP analysis, and due to the very high volume of HTTP traffic at many sites, removing this data can significantly reduce the size of the save file. Deficiency: Clearly, this should not be hardwired into Bro but under user control.

Save files written using -w are of course readable using -r. Accordingly, you will generally want to use -w when running Bro on live network traffic so you can rerun it off-line later to understand any problems that arise, and also to experiment with the effects of changes to the policy scripts.

You can also combine -r with -w to both read a save file(s) and write another. This is of interest when using multiple instances of -r, as it provides a way to merge tcpdump save files.

• -v|--version

Print the version of Bro and exit.

• -x|--print-state <Bro state file>

Reads the contents of the specified Bro state file, prints them to the console, and exits.

• -z|--analyze <analysis>

Runs the specified analyzer over the configured policy. See Policy analyzers.

• -A|--transfile <writefile>

Write transformed trace to the tcpdump file given. See Trace rewriting.

• -B| --debug <dbgstreams>

Enable debugging output for selected streams. The option is available if Bro is configured with --enable-debug before it is built.

• -C|--no-checksums

Incorrect IP, TCP, or UDP checksums normally trigger different variants of net_weird and conn_weird events (see also Events handled by net_weird, Events handled by conn_weird, and Events handled by conn_weird). This flag causes Bro to ignore incorrect checksums.

• -D|--dfa-size <size>

Sets the cache size of deterministic finite automata (used extensively for signatures) to the given number of entries. The default is 10,000.

• -F|--force-dns

Instructs Bro that it must resolve all hostnames out of its private DNS cache. If the script refers to a hostname not in the cache, then Bro exits with a fatal error.

The point behind this option is to ensure that Bro starts quickly, rather than possibly stalling for an undetermined amount of time resolving a hostname. Fast startup simplifies checkpointing a running Bro---you can start up a new Bro and then killing off the old one shortly after. You'd like this to occur in a manner such that there's no period during which neither Bro is watching the network (the older because you killed it off too early, the newer because it's stuck resolving hostnames).

• -I|--print-id <name>

Looks up the variable identified by ``name in the global scope (see Scope) and prints it to the console.

• -K|--md5-hashkey <hashkey>

Allows you to specify a fixed seed for MD5 initialization. MD5 is used by default for hashing elements in the Bro core, and by default some randomness is gathered at Bro startup before PRNG initialization.

Note: This means that by default repeated runs of Bro on identical inputs do not necessarily yield identical output. If you want to ensure determinism, use the --save-seeds and --load-seeds options.

• -L|--rule-benchmark

See Rule benchmarking.

• -O|--optimize

Turns on Bro's optimizer for improving its internal representation of the policy script. Note: Currently, the amount of improvement is modest, and there's (as always) a risk of an optimizer bug introducing errors into the execution of the script, so the optimizer is not enabled by default.

• -P|--prime-dns

Instructs Bro to prime its private DNS cache. It does so by parsing the policy scripts, but not executing them. Bro looks up each hostname's address(es) and records them in the private cache. The idea is that once bro -P finishes, you can then use bro -F to start up Bro quickly because it will read all the information it needs from the cache.

• -S|--debug-rules

Prints debugging output for the rules used in signature matching. See also Signatures.

• -T|--re-level <level>

Sets the level in the tree of rules at which regular expressions are built. Default is 4.

• -W|--watchdog

Instructs Bro to activate its internal watchdog. The watchdog provides self-monitoring to enable Bro to detect if its processing is wedged.

Bro only activates the watchdog if it is reading live network traffic. The watchdog consists of a periodic timer that fires every WATCHDOG_INTERVAL seconds. (Deficiency:clearly this should be a user-definable value.) At that point, the watchdog checks to see whether Bro is still working on the same packet as it was the last time the watchdog expired. If so, then the watchdog logs this fact along with some information regarding when Bro began processing the current packet and how many events it processed after handling the packet. Finally, it prints the packet drop information for the different interfaces Bro was reading from, and aborts execution.

• --use-binpac

If the support of BinPAC is tested with FLAGS_use_binpac in the source program (e.g., a BinPAC-generated event engine may want to test the availability of BinPAC support), enabling this option will make the test successful.

• --save-seeds <file>

Writes the seeds used for initializing the PRNGs in Bro to the given file. This can be combined with -K|--md5-hashkey, and is intended to be used with --load-seeds in future Bro runs.

• --save-seeds <file>

Seeds the PRNGs in Bro using a file produced by --save-seeds in an earlier Bro invocation.

Run-time environment

Bro is also sensitive to the following environment variables:

• $BROPATH

A colon-separated list of directories that Bro searches whenever you load a policy file. It loads the first instance it finds (though see $BRO_PREFIXES for how a single load can lead to Bro loading multiple files).

Default: if you don't set this variable, then Bro uses the path

   .:policy:policy/local:/usr/local/lib/bro

That is, the current directory, any policy/ and policy/local/ subdirectories, and /usr/local/lib/bro/.

• $BRO_PREFIXES

A colon-separate lists of prefixes that Bro should apply to each name in a @load directive. For a given prefix and load-name, Bro constructs the filename:

prefix.load-name.bro

(where it doesn't add .bro if load-name already ends in .bro). It then searches for the filename using $BROPATH and loads it if its found. Furthermore, it repeats this process for all of the other prefixes (left-to-right), and loads each file it finds for the different prefixes. Note: Bro also first attempts to load the filename without any prefix at all. If this load fails, then Bro exits with an error complaining that it can't open the given @load file.

For example, if you set $BRO_PREFIXES to:

   mysite:mysite.wan

and then issue "@load ftp", Bro will attempt to load each of the following scripts in the following order:

   ftp.bro
   mysite.ftp.bro
   mysite.wan.ftp.bro

Default: if you don't specify a value for $BRO_PREFIXES, it defaults to empty, and for the example above Bro would only attempt to "@load ftp.bro".

Helper utilities

Scripts

The autogen.sh script in the top-level directory sets up the autotools files.

The hf utility

The hf utility reads text on stdin and attempts to convert any dotted quads it sees to hostnames. It is very convenient for running on Bro log files to generate human-readable forms. See the manual page included with the distribution for details.

The cf utility

The cf utility reads Unix timestamps at the beginning of lines on stdin and converts them to human-readable form. For example, for the input line:

972499885.784104 #26 131.243.70.68/1899 > 64.55.26.206/ftp start

it will generate:

Oct 25 11:51:25 #26 131.243.70.68/1899 > 64.55.26.206/ftp start

It takes two flags:

• -l

specifies the long human-readable form, which includes the year. For example, on the above input, the output would instead be:

Oct 25 11:51:25 2000 #26 131.243.70.68/1899 > 64.55.26.206/ftp start

• -s

specifies strict checking to ensure that the number at the beginning of a line is a plausible timestamp: it must have at least 9 digits, at most one decimal, and must have a decimal if there are 10 or more digits.

Without -s, an input like:

131.243.70.68 > 64.55.26.206

generates the output:

Dec 31 16:02:11 > 64.55.26.206

which, needless to say, is not very helpful.

Deficiency: It seems clear that -s should be the default behavior.

Reference Manual Introduction | Getting Started | Values, Types, and Constants | Statements and Expressions Global and Local Variables | Predefined Variables and Functions | Analyzers and Events Signatures | Interactive Debugger | Missing Documentation | References User Manual