NAME

nfcount - make accounting from network flows by rules


SYNOPSIS

nfcount [-v [-v [-v]]] -r rules_file [-s state_file] [-p printer_desc] [-f flowfile_desc] [-m from filename to filename]


DESCRIPTION

The nfcount can be used for count network flows by using rules.


OPTIONS

The options are as follows:

[-v [-v [-v]]]
Set verbosity level.

-r rules_file
Load counter rules from file. If it options present more than one time, then only last filename is used.

-s state_file
Use state file state_file. State file holds all counter values and flow file positions to which it was used. So when You start nfcount at next time, it will process only new data.

WARNING In some cases the use of state file may cause lost of same flows. For example some traffic collector write half of flow to file and in this time the nfcount read this half record. The result will be dramatical if file is binary - nfcount can't read any flows, becouse it cant find start of next flow record! So it is strongly recomended not use state file when You do finally count.

-p printer_desc
Add printer to printers list. Printer is a result printer. The printer description consist of 3 part:

printer_desc =[ type ],[ opts ], file

where

type
is type of printer. List of available printers can be obtained be nfcount -h command

opts
is options for printer

file
is a filename to which result will be written. If filename is ``-'', then result will be written to stdout.

There are next types of printers is available:

perl
The result is perl file, with some variables. This type of printer not use any options.

bin
The result is binary file. THis file is used by nfcounthyst. This type of printer not use any options.

-f flowfile_desc
Add flow file to process. As in previous case, it is consist of three part, separated by comma.

flowfile_desc =[ type ],[ opts ], file

The compressed flows will recognized automatically, by ``magic'' bytes in file header. At this time supported next compression libraries: gzip, bzip2.

The next types is avalible:

argus
There are no options for this file format.

text
The file is set of string. One string describe one flow. Fields is separated by TAB charachter. The options is describe order of fields, by order of next charachters:
  A  Source Address of flow
  a  Destenation Address of flow
  P  Source Port of flow
  p  Destenation Port of flow
  I  Ip Protocol
  C  Packets From
  c  Packets To
  O  Octets From
  o  Octets To
  T  Start Time of flow
  t  Last Time of flow
  x  Skip this field

For example, file with next lines:

  10.1.1.1 8    10.1.1.2 0    1 1 92   1074105038
  10.1.1.2 3128 10.1.1.4 1910 6 8 3604 1074105040

may be described with opts APapICOT.

[-m from filename to filename]
Rename file in the state databese before use from from filename to to filename. It is useful when You make rotate (and may be compress some logs). For example, You have log file traffic.log

At end of month You rotate it, so traffic.log now named traffic.200403.bz2 and the new traffic.log created. And You want save Your state file. So. You just add one parameter to command line:

  nfcount .... -m traffic.log traffic.200403.bz2

and nfcount parses traffic.200403.bz2 from position at which it stop at previous parse old traffic.log !!! NOTE, that compression don't brake anything, becouse in state file nfcount always save position of uncompressed stream.


RULE LANGUAGE

Foreword

1) Comments start in any position of line from char #

2) If last char is \ then line is continued at next line

Top level sentences

Shared terms
 <number> = "0"-"9" ["0"-"9" ["0"-"9" [...] ] ]
 <4-digit> = "0"-"9" "0"-"9" "0"-"9" "0"-"9"
 <2-digit> = "0"-"9" "0"-"9"
 <1-3-digit> = "0"-"9" [ "0"-"9" ["0"-"9"] ]
 <variable> = <var_char> [ <var_char> [<var_char> [...] ] ]
 <var_char> = ["A"-"Z"]|["a"-"z"]|["0"-"9"]|"_"|"."
 - variable length must be < 255
Definition of counter variable:
 counter <counter_variable>
 <counter_variable> = <variable>
Definition of hystogram
 hystogram <hystogram_variable>
     by <hystogram_type> from <hystogram_value>
     to <hystogram_value> ( items <number> | step <number> )
     [ by <hystogram_type> from <hystogram_value>
       to <hystogram_value> ( items <number> | step <number> ) ]
 <hystogram_variable> = <variable>
 <hystogram_type> =  fromaddr | toaddr | time | month |
                    timeofmonth | timeofday | timeofweek |
                    timeofhour | timeofmin
 <hystogram_value> = <address> | <time>
 - step for time values always in seconds, for address in single address.
Definition of single network address or network
 address <address_variable> " " = " " <address_list>
 <address_variable> = <variable>
 <proto> = <number> | <string from /etc/prototcols>
 <address_list> = <address> ["," <address> ["," ... ] ]
 <address> = any | ( <ip4-address> [ ":" <network_mask> |
             "/" <network_len> ] [<port>] )
 <ip4-address> = <1-3-digit>"."<1-3-digit>"."<1-3-digit>"."<1-3-digit>
 <network_len> = <number>
 <network_mask> = <ip4-address>
 <port> = <number>
Definition of timerange
 timerange <timerange_variable> " " = " " <timerange_list>
 <timerange_variable> = <variable>
 <timerange_list> = <timerange> [ "," <timerange> ["," ...] ]
 <timerange> = [ <time> - | - <time> | <time> - <time> ]
 <time> = [     ["Y"[+|-]<0-...>]["M"[+|-]<1-12>]["D"[+|-]<1-31>]
            ["h"[+|-]<0-23>]["m"[+|-]<0-59>]["s"[+|-]<0-59>] |
            "u"<time-in-unix-format>
          ]
Definition the counting rule
 count <proto>
       ( <count_from> | <count_to> | <count_from> <count_to> )
       [ <count_timeramge> ] <count_as> [ <count_as> [...] ]

 <count_from> = from ( <address_list> | <address_variable> )
 <count_to> = to ( <address_list> | <address_variable> )
 <count_timerange> = timerange ( <timerange_list> | <timerange_variable> )
 <count_as> = ( as | inverseas ) ( <counter_variable> | <hystogram_variable> )

inverseas is a keyword, by using this instead of as You say to nfcount that it must change from and to values of flow before processing (from_address <-> to_address, from_packets <-> to_packets, ... ). NOTE that adresses is swapped too. So the next example

  hystogram my\
    by fromaddr from 10.0.1.192 to 10.0.1.223 step 1\
    by month from Y+0M1 to Y+0M12 step 1
  count ip from 10.0.1.192/27 to not 10.0.1.192/27 as my
  count ip from not 10.0.1.192/27 to 10.0.1.192/27 inverseas my

work correct. But if You change in second count rule inverseas to as You don't see any traffic from this rule, because fromaddr is always not in range 10.0.1.192-10.0.1.223.

Subwords:


EXAMPLES

Example of ruleset file:

  #counters and hystograms _MUST_ be defined before any 'count' rules
  address my_net = 10.0.1.192/27
  timerange year_2003 = Y2003-Y2004
  timerange year_2004 = Y2004-Y2005
  counter a_tcp
  counter a_udp
  counter a_icmp
  counter a_ip
  # Y+0 requered, in other case the year is zerro and time_t type
  # can't handle it
  hystogram my_from_2004\
    by fromaddr from 10.0.1.192 to 10.0.1.223 step 1\
    by month from Y+0M1 to Y+0M12 step 1
  hystogram my_to_2004\
    by toaddr from 10.0.1.192 to 10.0.1.223 step 1\
    by month from Y+0M1 to Y+0M12 step 1
  hystogram my_weekday_2004\
    by timeofweek from Y+0D1 to Y+0D7h23m59s59 step 86400
  hystogram my_hour_2004\
    by timeofday from Y+0h0 to Y+0D7h23m59s59 step 3600
  count tcp from any to any as a_tcp
  count udp from any to any as a_udp
  count icmp from any to any as a_icmp
  count tcp from any to any as a_ip
  count udp from any to any as a_ip
  count icmp from any to any as a_ip
  count ip from my_net to any timerange year_2004 as my_from_2004
  count ip from any to my_net timerange year_2004 as my_to_2004
  count ip from my_net to any timerange year_2004 as my_weekday_2004
  count ip from any to my_net timerange year_2004 as my_weekday_2004
  count ip from my_net to any timerange year_2004 as my_hour_2004
  count ip from any to my_net timerange year_2004 as my_hour_2004

Example of command line:

  nfcount -v -v -v -r nfcount.conf -p perl,,nfcount.pl
         -f text,APapICOT,ng_ipacct.output

Here defined verbosity of 3rd level. Ruleset read from nfcount.conf. Network flows will be readed from file ng_ipacct.output, which has type text (for example Cisco IP Accounting ) and has format APapICOT. Result will be written to nfcount.pl, with using perl type of output.


SEE ALSO

See ng_ipacct(4), ipacctctl(8), ng_netflow(4), flowctl(8), ipacctctl(8), argus(8), trafd(8), ipa(8).


AUTHORS

Serg A. Repalov <rs@mis.rsu.ru>.


BUGS

If You find error, please report me.