CRAWDAD tools/sanitize/generic/FLAIM

FLAIM - anonymize system and network logs.

FLAIM is a multi-level, multi-log anonymization tool. Note: as of March 2021, the FLAIM website is no longer available, and CRAWDAD's copy of the tools has been removed at the contributor's request. FLAIM-Core comprises the anonymization engine and XML based policy manager. FLAIM-Core loads dynamic libraries responsible for I/O and parsing at runtime. There is a library for each type of log flaim supports. The XML policy, the I/O module, input file and output files are all specified on the command line.

Lastmodified :

2007-02-15

Dataname :

tools/sanitize/generic/FLAIM

Releasedate :

2006-11-01

Equiversion :

0.5.1

Change :

the initial version

Website :

https://illinois.edu

Keyword :

packet trace

License :

Copyright © 2005-2007 The Board of Trustees of the University of Illinois. All rights reserved.

Developed by:

LAIM Working Group
National Center for Supercomputing Applications
University of Illinois

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
with the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

- Redistributions of source code must retain the above copyright notice, this list
of conditions and the following disclaimers.
- Redistributions in binary form must reproduce the above copyright notice, this
list of conditions and the following disclaimers in the documentation and/or other
materials provided with the distribution.
- Neither the names of the National Center for Supercomputing Applications,
the University of Illinois, nor the names of its contributors may be used to endorse
or promote products derived from this Software without specific prior written permission.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE CONTRIBUTORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH THE SOFTWARE.

Support :

FLAIM is an open source project, and we welcome your participation. Feature requests,
bug reports, success stories, and software patches are some examples of valuable
community contributions.

For discussions about FLAIM, please join the flaim-users@ncsa.uiuc.edu discussion list.
You must be subscribed to post to the list. To subscribe to the flaim-users@ncsa.uiuc.edu
list, send email to majordomo@ncsa.uiuc.edu with "subscribe flaim-users" in the body of
the message.

Release announcements and security advisories are also posted to flaim-announce@ncsa.uiuc.edu.
Only the LAIM PI can post to this list, and messages should be infrequent. To subscribe
to the flaim-announce@ncsa.uiuc.edu list, send email to majordomo@ncsa.uiuc.edu with
"subscribe flaim-announce" in the body of the message.

Build :

1. Install any missing dependencies.

FLAIM has been tested on Linux 2.6, Mac OS 10.4, FreeBSD 6.1, OpenBSD 3.9 and NetBSD 3.0.1.
FLAIM has dependencies on the following libraries.

- LIBXML:
Libxml is an open source C library for parsing and validating XML files. It is available
on most Linux and BSD systems. It can be downloaded from http://xmlsoft.org/.
Note that you will need both the binary library as well as the development files to compile
FLAIM. For binary packages - to be released in the near future - you will not need the
developer's package.

- LIBXSLT:
Libxslt is a open source C library for parsing XSLT style-sheets. It is available
on most Linux and BSD systems. It can be downloaded from http://xmlsoft.org/XSLT/.
Note that you will need both the binary library as well as the development files
to compile FLAIM. For binary packages - to be released in the near future - you will
not need the developer's package.

2. Download FLAIM-Core and at least one I/O module.

You can find FLAIM and its supported modules on the Downloads tab. Download FLAIM-Core
and the modules corresponding to the types of logs you would like to process.

3. Install FLAIM-Core

You must unpack the tarball, run the config script, and make the package. An example follows.

[yoursystem]:$ tar zxf flaim-core-<version>.tgz
[yoursystem]:$ cd flaim-core-<version>
[yoursystem]:$ ./configure
.... lots of output here ......
[yoursystem]:$ make
[yoursystem]:$ make install

4. Install a FLAIM Module
You must unpack the tarball, run the config script, and make the package. An example follows.

[yoursystem]:$ tar zxf flaim-module-<modulename>-<version>.tgz
[yoursystem]:$ cd flaim-module-<modulename>-<version>
[yoursystem]:$ ./configure
.... lots of output here ......
[yoursystem]:$ make
[yoursystem]:$ make install


* Installing in Non-Standard Locations *

The default installation behavior is to create /usr/local/flaim and place all the libraries
and configuration files there. In addition, a symbolic link is made to /usr/bin/flaim.
One must normally be root to install in these directories, or to install the man page.

To install FLAIM in a different location, one can pass an option to the configure script.
The option to set is prefix. The default value for this variable is /usr/local. FLAIM then
installs to $(prefix)/flaim, i.e. /usr/local/flaim. Say, instead, a user wants to install
FLAIM into /usr/flaim. He would then pass the option to the configure script as follows:

[yoursystem]:$ ./configure --prefix=/usr

If the installation directory is changed for FLAIM-Core, it must be changed in the same way
when modules are installed. Again, this can be done with the same options for their configure scripts.

Usage :

* Synopsis *

flaim [ OPTION ] [ -m module-name ] [ -p policy ] [ -i input.log ] [ -o output.log ]

* Description *

flaim is a multi-level, multi-log anonymization tool. FLAIM-Core comprises
the anonymization engine and XML based policy manager. FLAIM-Core loads
dynamic libraries responsible for I/O and parsing at runtime. There is
a library for each type of log flaim supports. The XML policy, the I/O module,
input file and output files are all specified on the command line.

* Options *

-c --config <file-name>
Read config data from the specified file. If unspecified, the default of
$(FLAIM_ROOT)/flaim.cfg is used. The default is recommended for all but
the most advanced users.

-h --help
Display the usage information and exit.

-i --input <file-name>
Specifies the source log for anonymization. If unspecified, stdin will
be used. However, not all modules support reading input from stdin.
Those that do not support streaming will exit and force you to specify
a file name.

-l --list
Lists all installed modules. FLAIM will not find manually installed modules
in non-standard locations.

-m --module <module-name>
Load the specified parsing module. The -l option shows the valid choices.
Either this option must be used to specify a module installed in the default
location, or the -M option must be used.

-M --moduleLib <module-lib-path>
Load the module library from the given path. This option is mutually exclusive
with -m and used for explicitly specifying the module location. It is also
necessary to specify the schema location using -s when using this option.

-o --output <file-name>
Specifies the destination file for anonymized data. If unspecified, stdout will
be used. However, not all modules support writing output to stdout. Those that
do not support streaming, will exit and force you to specify a file name.

-p --policy <file-name>
The use of this flag is mandatory as it specifies the location of the user policy.
-s --schemaModule <module-schema-file-path>
Load the module schema from the file specified with this option. This option is
used if and only if the -M option is used.

-v --verbose
Print verbose messages to stderr.

-V --version
Print version information to stderr and exit.

-x --xtraConfig <file-name>
This is used to specify a file containing extra information to be passed to
the parsing module. It is optional and ignored by most modules.

* See Also *

formerly at this location: flaim.ncsa.uiuc.edu/documentation.html

* Bugs *

This is a beta release of flaim and we expect bugs to be found. We suggest users
that want to hear about latest bug findings (or submit bugs themselves)
to join the flaim-users@ncsa.uiuc.edu mailing list.
Instructions for joining this and other flaim lists were found at flaim.ncsa.uiuc.edu/support.html

* Environment *

The environment variable FLAIM_ROOT specifies the location of flaim and, consequently,
the location of the FLAIM modules. This environment variable is setup in the default
installation of FLAIM-Core.

Example :

* Sample Logs and Policies [flaim-samples-0.5.1.tar.gz] *

We have provided sample logs and FLAIM policies for users to try out FLAIM.
We encourage you to play with the anonymization policy to explore FLAIM's capabilities.

The tarball 'flaim-samples-0.5.1.tar.gz' contains the following sample logs and policies:

- For 'pcap' module:
sample.pcap.log sample-pcap.apolicy.xml

- For 'iptable' module:
sample.iptable.log sample-iptable.apolicy.xml

- For 'nfdump' module:
sample.nfdump.log sample-ndump.apolicy.xml

Acknowledgment :

The following people also contributed to the development of FLAIM:

Vikram Dhar
Greg Colombo
Jun Wang
Bill Yurcik
Yifan Li