Threat Intelligence in Sagan Using Bro Intelligence Framework

Introduction

Monitoring Network Traffic for malicious activities is one of the priorities for an network IDS, what if the network IDS can take in threat intelligence data to look out for malicious domains, Ip addresses, emails,file-hashes and so on, sagan has one such feature called Bro intelligence framework this write up  will give you an insight on how to install configure and use bro intel framework in sagan.

I have been working with sagan for few months now, it has been really hard to put all the pieces together and you don't really find any perfect documentation or tutorials regarding sagan and its features at one place.

Sagan is an open source real time log analysis and correlation engine that runs on Unix Operating Systems, sagan's rules are quite similar to SNORT. Sagan can record events to the snort unified 2 output format which allows it to be compatible with user interfaces such as Squil,Snorby,Graylog.
Sagan has lot more capabilities than being a Log Analysis Engine

                  One of which is the Bro intel Processor, Today I'm going to put all the pieces together for sagan users to use threat intelligence in sagan using bro intel processor and also provide the instruction to configure and install  Intel Critical Stack for threat intelligence feeds.

Enabling Bro Intel Processor in Sagan

Enabling the bro processor in sagan is easy, usually the bro processor is disabled you need to enable it in the sagan.conf or sagan.yaml, after enabling the bro intel processor restart sagan you would see the bro intel indicators being loaded into sagan.

Bro Intel::ADDR Loaded: 0

Bro Intel::DOMAIN Loaded: 0

Bro Intel::FILE_HASH Loaded: 0

Bro Intel::URL Loaded: 0

Bro Intel::SOFTWARE Loaded: 0

Bro Intel::EMAIL Loaded: 0

Bro Intel::USER_NAME Loaded: 0

Bro Intel::FILE_NAME Loaded: 0

Bro Intel::CERT_HASH Loaded: 0

Bro Intel Duplicates Detected: 0

Usually the indicators don't have any threat intel feeds we need to load feeds either by critical stack or we can do it manually by writing a simple script.


Note : Bro intel feature might be broken in old versions of sagan, it better to update sagan before using this feature.


Once you have enabled the bro processor and can see the bro intel indicators loading while starting sagan, next is to put up some feeds.

You need to give your path in the bro intel processor for the feeds to load default critical stack path is

/opt/critical-stack/framwworks/intel/master-public.bro.dat

But you can add as many as files as per your requirements, there are just comma delimited.

Threat Intel Feeds

Bro threat intel feeds can be loaded either manually or by using critical stack.

If you have used any threat intelligence framework, we all might have noticed that each feeds's data is formatted slightly different of other data feeds, while one feed might be zip file that is tab delimited another might be a csv file that you need to download and import.

This is a issue which critical stack addresses the critical stacks feeds are optimized and ready for the bro intelligence framework, and in other words you don't have to load the data into bro intelligence framework once you install critical stack all the data is automatically loaded into it.

Critical stack gives you a very nice web portal to subscribe and unsubscribe to the feeds, you can create your own list of feeds depending on their category, all feeds are downloaded to bro intelligence framework using the critical stack api and there are updated automatically, and more important the feeds downloaded are ready to use by the bro intel framework.

Installing and Configuring Critical Stack On Sagan

Installing and Configuring Critical Stack is quite easy :

1. Create an account on https://intel.criticalstack.com/

2. Once You login you will be prompted with clear instruction on how to create a feeds list and assign a sensor or vice versa.

3. Now Install the Critical Stack Intel client you can do it by using Curl Command

First Add the Repository using the following command

curl --silent https://packagecloud.io/install/repositories/criticalstack/critical-stack-intel/script.deb.sh | sudo bash

Once completed install the Client normally using the following command

sudo apt-get install critical-stack-intel

You can even Do it manually

Manual Download (386): https://intel.criticalstack.com/client/critical-stack-intel-i386.deb

Manual Download (amd64): https://intel.criticalstack.com/client/critical-stack-intel-amd64.deb

The default path for Critical Stack on your sagan will be

/opt/critical-stack/framwworks/intel/master-public.bro.dat

Once you install the Critical Stack client you need to add your api key by using the command

$ sudo critical-stack-intel api API-Key 

The api key is mainly the sensor unique key which can be found on the sensor which you have created, Once you have added your api key all the feeds which you have subscribed will be downloaded into the crirical stack client. 

You can pull the feeds by using the following command

sudo critical-stack-intel pull

2015/01/10 14:20:17 Fetching feed subscriptions.
2015/01/10 14:20:17   * critical-stack-intel-1-Matsnu-Botnet.bro.dat
2015/01/10 14:20:17   * critical-stack-intel-2-C-Cs-IPs-Domains.bro.dat
2015/01/10 14:20:17   * critical-stack-intel-3-Cryptolocker.bro.dat
2015/01/10 14:20:17   * critical-stack-intel-4-Post-Tovar-GameOver-Zeus.bro.dat
2015/01/10 14:20:17   * critical-stack-intel-5-Tinybanker---Tinba.bro.dat
2015/01/10 14:20:17   * critical-stack-intel-6-PushDo-Malware.bro.dat
2015/01/10 14:20:17   * critical-stack-intel-7-Known-Tor-Exit-Nodes.bro.dat
2015/01/10 14:20:17   * critical-stack-intel-8-Cyber-Crime-Tracker.bro.dat
2015/01/10 14:20:17   * critical-stack-intel-9-Zeus-Tracker--Configs.bro.dat

2015/01/10 14:20:18 Creating master file: master-public.bro.dat
2015/01/10 14:20:18 Master file created successfully.
2015/01/10 14:20:18 Intel files located at: /opt/critical-stack/frameworks/intel
2015/01/10 14:20:18 API Requests Remaining: 81 of 100
      

You can Check the list of feeds by using the following command

--- critical-stack-intel list

You can also white list by using the following command

$ sudo critical-stack-intel whitelist --add yahoo.com
$ sudo critical-stack-intel whitelist list

   ID   |         VALUE           
+-------+------------------------+
  1     | yahoo.com              
+-------+------------------------+
  TOTAL |           1             
+-------+------------------------+
        

Now lets see how we can add the bro intelligence data manually by using a script

You can create your own bro: Intel feeds by putting all the required data in a text file. 

For example to create a SSL hashes based bro:intel feeds, create a  text file with the blacklisted 
SSL hashes and run the following bash script 

 ‘{print $1”\011””Intel::Cert_Hash””\011””SSL””\011””F”}’ Filename > Filename.bro:intel 

Bro Intel data are tab delimited, you can write your own script as per your needs. 

You can have as many as manuals feeds but the location of the file should be provided within the bro intel processor in the sagan.conf or sagan.yaml file.

You can write you own script in any scripting language, this is just an example script which can be used.



Once all the feeds are loaded into sagan's bro intelligence framework, after you start your sagan you might see something like this 

[*] Bro Intel::ADDR Loaded: 23412
[*] Bro Intel::DOMAIN Loaded: 2247
[*] Bro Intel::FILE_HASH Loaded: 541
[*] Bro Intel::URL Loaded: 37934
[*] Bro Intel::SOFTWARE Loaded: 28
[*] Bro Intel::EMAIL Loaded: 400
[*] Bro Intel::USER_NAME Loaded: 50
[*] Bro Intel::FILE_NAME Loaded: 100
[*] Bro Intel::CERT_HASH Loaded: 100
[*] Bro Intel Duplicates Detected: 0

Sagan Rules

Now let's see how  we can use the bro intelligence processor to detect malicious traffic through you network, it all comes down to writing rules in sagan to trigger alert if any malicious ip is making a tcp connection, A host system is vising an malicious domain, A suspicious software is downloaded.

Example rule

alert tcp any any -> any any (msg: “[Example Rule] Found Suspicious Source Ip”; program: router; content: “TCP connection from”; parse_src_ip: 1; parse_dst_ip: 2; bro-intel: by_src; classtype: suspicious-traffic; sid: 900000; rev: 1;)

this is just an example you can  write lot of rules by varying programs, changing the bro intel indicators. 

I use lot of different sensors at work, so the programs options can have any valid sensor from where the sagan can search for logs.

and finally in the rules the different bro-intel indicators which can be used to identify different kinds of malicious traffic.

bro-intel: by_src

bro-intel: by_dst

bro-intel: both

bro-intel: all

bro-intel: software

bro-intel: file_hash

bro-intel: email

bro-intel: user_name

bro-intel: cert_hash

bro-intel: url

bro-intel: file_hash

For more information on Sagan rule option,  go to:

https://wiki.quadrantsec.com/bin/view/Main/SaganRuleReference