Reads EVE files into SQL as well as search stored data.

eve ids lae perl pie sagan security suricata

Go to file

Zane C. B-H 80a60b74a3 note the time ready for release		2024-01-15 17:11:58 -06:00
bin	rework it so it rewrites #!	2022-10-27 19:48:13 -05:00
init	teak the suricata table some and update the freebsd init script	2022-10-19 02:59:03 -05:00
lib	table search stuff for cape	2024-01-15 16:07:44 -06:00
src_bin	finalize support for cape	2024-01-15 17:06:38 -06:00
t	add initial files	2022-06-20 03:52:48 -05:00
.gitignore	rework it so it rewrites #!	2022-10-27 19:48:13 -05:00
Changes	note the time ready for release	2024-01-15 17:11:58 -06:00
MANIFEST	add init/freebsd the manifest	2022-10-30 23:54:37 -05:00
Makefile.PL	add a missing depend	2023-12-31 15:04:24 -06:00
README.md	document --raw	2023-11-11 22:32:01 -06:00
ignore.txt	add initial files	2022-06-20 03:52:48 -05:00

README.md

Lilith

Lilith reads in EVE files from Suricata and Sagan into PostgreSQL.

From there that data can then be searched and information on specific events fetched.

Intalation

Debian

apt-get install zlib1g-dev cpanminus libjson-perl libtoml-perl \
 libdbi-perl libfile-readbackwards-perl libdigest-sha-perl libpoe-perl \
 libfile-slurp-perl libdbd-pg-perl
cpanm Lilith

FreeBSD

pkg install p5-App-cpanminus p5-JSON p5-TOML p5-DBI \
 p5-File-ReadBackwards p5-Digest-SHA p5-POE \
 p5-MIME-Base64 p5-Gzip-Faster p5-DBD-Pg p5-File-Slurp
cpanm Lilith

Source

perl Makefile.PL
make
make test
make install

Setup

First you need to setup your PostgreSQL server.

createuser -D -l -P -R -S lilith
createdb -E UTF8 -O lilith lilith

Setup /usr/local/etc/lilith.toml

dsn="dbi:Pg:dbname=lilith;host=192.168.1.2"
pass="WhateverYouSetAsApassword"
user="lilith"
# a handy one to ignore for the extend as it is spammy
class_ignore=["Generic Protocol Command Decode"]

# add a suricata instance to monitor
[suricata-eve]
instance="foo-pie"
type="suricata"
eve="/var/log/suricata/alert.json"

# add a second suricata instance to monitor
[another-eve]
instance="foo2-pie"
type="suricata"
eve="/var/log/suricata/alert2.json"

# add a sagan eve to monitor
# instance name is 'foo-lae', given there is no value for instance
[foo-lae]
type="sagan"
eve="/var/log/sagan/alert.json"

Now we just need to setup the tables.

lilith -a create_tables

If using snmpd.

extend lilith /usr/local/bin/lilith -a extend

Config File

The default config file is /usr/local/etc/lilith.toml.

Variable	Description
dsn	A DSN connection string to be used by [DBI][https://metacpan.org/pod/DBI]. [DBD::Pg][https://metacpan.org/pod/DBD::Pg]
pass	Password to use for the connection.
user	User to use for the connetion.
class_ignore	Array of classes to ignore.

Sub hashes are then treated as a instance. The following values are available for that.

Variable	Required	Description
eve	yes	The EVE file to follow.
type	yes	`sagan` or `suricata`, depending on which it is.
instance	no	The name for the instance. If not specified the hash name is used.

Options

SYNOPSIS

lilith [-c <config>] -a run

lilith -a class_map

lilith [-c <config>] -a create_tables

lilith [-c <config>] -a dump_self

lilith [-c <config>] -a event [-t <table>] --id <row_id> [--raw]

\[--virani \<remote\>

lilith [-c <config>] -a event [-t <table>] --event <event_id> [--raw]

\[--virani \<remote\>

lilith [-c <config>] -a extend [-Z] [-m <minutes>]

lilith -a generate_baphomet_yamls --dir <dir>

lilith [-c <config>] -a get_short_class_snmp_list

lilith [-c <config>] -a search [--output <return>] [-t <table>]

-m \<minutes\>

--orderdir \<dir\>

--sp \<src_port\>

--hostl

-il

--sN

--gid \<gid\>

GENERAL SWITCHES

-a

The action to perform.

- Default :: search

-c

The config file to use.

- Default :: /usr/local/etc/lilith.toml

-t

Table to operate on.

- Default :: suricata

ACTIONS

run

Start processing the EVE logs and daemonize.

class_map

Print a table of class mapping from long name to the short name used for display in the search results.

create_tables

Create the tables in the DB.

dump_self

Initiate Lilith and then dump it via Data::Dumper.

event

Fetches a event. The table to use can be specified via -t.

--id <row_id>

Fetch event via row ID.

--event <event_id>

Fetch the event via the event ID.

--raw

Do not decode the EVE JSON.

--pcap

Fetch the remote PCAP via Virani and write it to the file. Only usable for with Suricata tables.

Default :: undef

--virani

Virani setting to pass to -r.

Default :: instance name in alert

--buffer

How many seconds to pad the start and end time with.

Default :: 60

extend

Prints a LibreNMS style extend.

-Z

Enable Gzip+Base64 LibreNMS style extend compression.

-m

How far back to search. For the extend action, 5 minutes is the default.

generate_baphomet_yamls

Generate the YAMLs for Baphomet.

-d

The directory to write it out too.

get_short_class_snmp_list

Print a list of shorted class names for use wit SNMP.

search

Search the DB. The table may be specified via -t.

The common option types for search are as below.

- Integer :: A comma seperated list of integers to check for. Any number
             prefixed with a ! will be negated.
- String :: A string to check for. May be matched using like or negated via
            the proper options.
- Complex :: A item to match.

General Search Options

--output

The output type.

- Values :: table,json
- Default :: table

-m

How far back to to in minutes.

- Default :: 1440

- Default, extend :: 5

--order

Column to use for sorting by.

- Default :: timestamp

--orderdir

Direction to order in.

- Values :: ASC,DSC
- Default :: ASC

IP Options

--si

Source IP.

- Default :: undef
- Type :: string

--di

Destination IP.

- Default :: undef
- Type :: string

--ip

IP, either dst or src.

- Default :: undef
- Type :: complex

Port Options

--sp

Source port.

- Default :: undef
- Type :: integer

--dp

Destination port.

- Default :: undef
- Type :: integer

-p

Port, either dst or src.

- Default :: undef
- Type :: complex

Host Options

Sagan :: Host is the sending system and instance host is the host the
         instance is running on.

Suricata :: Host is the system the instance is running on. There is no
            instance host.

--host

Host.

- Default :: undef
- Type :: string

--hostl

Use like for matching host.

- Default :: undef

--hostN

Invert host matching.

- Default :: undef

Instance Options

--ih

Instance host.

- Default :: undef
- Type :: string

--ihl

Use like for matching instance host.

- Default :: undef

--ihN

Invert instance host matching.

- Default :: undef

Instance Options

=head4 -i

Instance.

- Default :: undef
- Type :: string

--il

Use like for matching instance.

- Default :: undef

--iN

Invert instance matching.

- Default :: undef

Class Options

-c

Classification.

- Default :: undef
- Type :: string

--cl

Use like for matching classification.

- Default :: undef

--cN

Invert class matching.

- Default :: undef

Signature Options

-s

Signature.

- Default :: undef
- Type :: string

--sl

Use like for matching signature.

- Default :: undef

--sN

Invert signature matching.

- Default :: undef

In Interface Options

--if

Interface.

- Default :: undef
- Type :: string

--ifl

Use like for matching interface.

- Default :: undef

--ifN

Invert interface matching.

- Default :: undef

App Proto Options

--ap

App proto.

- Default :: undef
- Type :: string

--apl

Use like for matching app proto.

- Default :: undef

--apN

Invert app proto matching.

- Default :: undef

Rule Options

--gid

GID.

- Default :: undef
- Type :: integer

--sid

SID.

- Default :: undef
- Type :: integer

--rev

Rev.

- Default :: undef
- Type :: integer

ENVIROMENTAL VARIABLES

Lilith_table_color

The LText::ANSITable table color to use.

- Default :: Text::ANSITable::Standard::NoGradation

Lilith_table_border

The LText::ANSITable border type to use.

- Default :: ASCII::None

Lilith_IP_color

Perl boolean for if IPs should be colored or not.

- Default :: 1

Lilith_IP_private_color

ANSI color to use for private IPs.

- Default :: bright_green

Lilith_IP_remote_color

ANSI color to use for remote IPs.

- Default :: bright_yellow

Lilith_IP_local_color

ANSI color to use for local IPs.

- Default :: bright_red

Lilith_timesamp_drop_micro

Perl boolean for if microseconds should be dropped or not.

- Default :: 1

Lilith_instance_color

If the lilith instance colomn info should be colored.

- Default :: 1

Lilith_instance_type_color

Color for the instance name.

- Default :: bright_blue

Lilith_instance_slug_color

Color for the insance slug.

- Default :: bright_magenta

Lilith_instance_loc_color

Color for the insance loc.

- Default :: bright_cyan.

README.md

Lilith

Intalation

Debian

FreeBSD

Source

Setup

Config File

Options

SYNOPSIS

GENERAL SWITCHES

-a

-c

-t Table to operate on.- Default :: suricata

ACTIONS

run

class_map

create_tables

dump_self

event

--id <row_id>

--event <event_id>

--raw

--pcap

--virani

--buffer

extend

-Z

-m

generate_baphomet_yamls

-d

get_short_class_snmp_list

search

General Search Options

--output

-m

--order

--orderdir

IP Options

--si

--di

--ip

Port Options

--sp

--dp

-p

Host Options

--host

--hostl

--hostN

Instance Options

--ih

--ihl

--ihN

Instance Options

--il

--iN

Class Options

-c

--cl

--cN

Signature Options

-s

--sl

--sN

In Interface Options

--if

--ifl

--ifN

App Proto Options

--ap

--apl

--apN

Rule Options

--gid

--sid

--rev

ENVIROMENTAL VARIABLES

Lilith_table_color

Lilith_table_border

-t
Table to operate on.
`- Default :: suricata`