Incoherent ramblings of an InfoSec professional

Category Archives: How-to

PFSense, Netflow and ELK w/geoip

Several months ago I started working with the ELK stack (elasticsearch, logstash, kibana) for use with bluecoat proxy logs. I have been running pfsense at home for quite sometime and decided it would be nice to get some data pulled out of it, why not with netflow. As with everything else there are pieces of stuff all over the interwebs, but nothing that pulled it all together for me to use.

Here is a simple breakdown of the steps.

Configure pfsense to pass flow data
install java
install elasticsearch – research optimizations for single node.
install kibana 4
install logstash
install nginx
configure logstash
configure kibana – secure
configure nginx
configure elasticsearch – secure
configure mappings for flow
build dashboards in kibana

Here is the base setup.

Debian 8.1 64bit running on ESXi
– 2 vCPUs
– 8GB Ram
– 60G Storage

A. Setup PFSense to collect and pass flow data

Install softflowd package that is available for pfsense.


Services -> softflowd select “Interface, Host “ip of ELK box”, Port “9995” (will be configured later in logstash config)

Screen Shot 2015-07-13 at 9.44.07 PM

B. Install and configure ELK – a good chunk with modifications was taken from this DigitalOcean article –

1. Install Java (latest version)

Add repo list and update –

 sudo add-apt-repository -y ppa:webupd8team/java && apt-get update

Install java8

 sudo apt-get -y install oracle-java8-installer

2. Install Elasticsearch NOTE:I just used the latest (1.6.0) as of the time of writing this.

Download GPG key for apt.

    wget -O - | sudo apt-key add -

Add to repo list and update.

    echo 'deb stable main' | sudo tee /etc/apt/sources.list.d/elasticsearch.list


    sudo apt-get -y install elasticsearch=1.6.0

Edit /etc/elasticsearch/elasticsearch.yml and set localhost

3. Install Kibana 4 NOTE:Latest as of this writing 4.1.1

  tar -zxvf kibana-4.1.1-linux-x64.tar.gz

Move the code over to someplace that seems normal

sudo mkdir -p /opt/kibana && sudo cp -R ~/kibana-4*/* /opt/kibana/

Go head and install this git code that will setup kibana as a service.

cd /etc/init.d && sudo wget
sudo chmod +x /etc/init.d/kibana4

At this point edit the kibana config in /opt/kibana/config/kibana.yml and set host: “localhost”

4. Install Logstash NOTE:Latest as of this writing 1.5.2

Add repo and update.

  echo 'deb stable main' | sudo tee /etc/apt/sources.list.d/logstash.list && sudo apt-get update

Install Logstash

  sudo apt-get install logstash

5. Install nginx

We will use nginx as a reverse proxy to get to the ELK instance. Kibana and Elasticsearch do not offer much in the way of security so we have to lock them to only being accessible from localhost.

apache2-utils only necessary if htpasswd is desired to help secure the kibana instance.

  apt-get install nginx apache2-utils 

6. Configure nginx reverse proxy

Replace /etc/nginx/sites-available/default with the following.

server {
    listen 80;

    server_name <servername>;

    location / {
        proxy_pass http://localhost:5601;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;        

7. Configure Logstash to accept traffic and create geoid data.

First get the geoip data, this is updated monthly so a cron is probably a good idea. This will create the lookup database for logstash.

cd /etc/logstash && sudo curl -O "" && sudo gunzip GeoLiteCity.dat.gz

Create /etc/logstash/conf.d/logstash-netflow.conf place the following in the file.

input {
   udp {
     port => 9995
     codec => netflow
     type => netflow
filter {
  geoip {
    source => "[netflow][ipv4_src_addr]"
    target => "src_geoip"
    database => "/etc/logstash/GeoLiteCity.dat"
  geoip {
    source => "[netflow][ipv4_dst_addr]"
    target => "dst_geoip"
    database => "/etc/logstash/GeoLiteCity.dat"
output {
  stdout { codec => rubydebug }
  if ( [host] =~ "10\.100\.1\.1" ) {
    elasticsearch {
      index => "logstash_netflow-%{+YYYY.MM.dd}"
      host => "localhost"
  } else {
    elasticsearch {
      index => "logstash-%{+YYYY.MM.dd}"
      host => "localhost"

Parts of this are for future growth, where [host] =~ .


Start everything up and set to start on boot

    sudo systemctl enable elasticsearch.service
    sudo systemctl enable logstash.service
    sudo systemctl enable kibana4.service
    sudo systemctl enable nginx.service
    sudo systemctl start elasticsearch.service
    sudo systemctl start logstash.service
    sudo systemctl start kibana.service
    sudo systemctl start nginx.service

Everything now should be working, check your logs /var/log/logstash/logstash.log and /var/log/elasticsearch/elasticsearch.log – once you get data pushed into logstash ES should create an index in /var/lib/elasticsearch/elasticsearch/nodes/0/indicies

Now we just need to connect to Kibana and configure some stuffs

9. Create geo_point mappings

Once we are sure that data is flowing we have to create static mappings for the geo-location data that is created by logstash. ES will do dynamic mappings but Kibana won’t see them correctly to create a map of the data. Below worked on my setup, but it took me a while to get it right.

This will apply the “logstash_netflow_template” to any index named “logstash_netflow*” our index is date based (logstash_netflow-YYYY.MM.DD) so this will cover every index created going forward.

root@Logstash:~# curl -XPUT http://localhost:9200/_template/logstash_netflow_template -d '
  "template" : "logstash_netflow*",
  "mappings" : {
    "netflow" : {
      "properties" : {
        "dst_geoip" : {
          "properties" : {
            "location" : {"type":"geo_point"}
       "src_geoip" : {
         "properties" : {
           "location" : {"type":"geo_point"}

10. That should be it for configs, the rest can be done inside Kibana.

Go to kibana http:// once ELK loads, go to Settings -> Indices

Create New Index Pattern, select “Use event times to create index names” set the Index name or pattern to [logstash_netflow-]YYYY.MM.DD – You should get pattern match if you index is created properly, select Time-field name of “@timestamp” then just hit “Create”


I will save the rest of the Kibana setup for another post or for the reader to struggle through, here are some shots of my dashboard as it sits using the data collected above.

Screen Shot 2015-07-13 at 9.25.43 PM

Screen Shot 2015-07-13 at 9.25.59 PM



There are a lot of performance tuning for ELK that I will talk about in another post. This will work for a small installation. This is running at my home with average usage of about 1000 events/min. In a larger environment you will likely need to separate your collectors (logstash) from your elasticsearch instances, and tune your ES. Rule of thumb, ES_HEAP_SIZE = half of your system ram but never more than 32g. If you need more than that spin up a second instance of ES.

Java – Can’t Uninstall? Whitelist it?

As with most sizable organizations it is near impossible to uninstall or completely disable Java which sent us on a hunt for a feasible way to contain Java based attacks. What we came up with was restricting it to run only in trusted zones. This worked for APPLET tags when encountered in IE. 

What this does is block any applet from running if it is not part of a trusted internet zone. First thing is to identify all the internal trusted zones and add them. Next allow the user to trust their own zones. Most of the time it seemed they knew when there was an applet they wanted to run. To enable this there is a simple registry change value 1C00 in 

 HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\3 to a value of 0
*Note: (original 10000). 

This will prohibit Java from running in the “Internet Zone.” Now for internal sites you can just whitelist them as a “Trusted Zone” for java to run properly. Of course this can be done via GPO for all internal sites and if there are some identified external sites that java is required. 

For some fun stats, an enterprise of 15,000 endpoints went from ~1.5 take aways per day down to about 1-2 per month due to Java a java drive by style attacks.

For most organizations updating java is a Herculean effort. So the whitelisting method from within Windows can be a viable alternative that can be quite effective. In recent months Oracle has released Java 1.7U40 which includes whitelisting. This is nice because it will work for browsers other than IE but odds are if you can get to 7U40 you have a good handle on patching anyway so this is less of an issue (except for 0-day). Java calls their implementation “Deployment Rule Sets” (DRS). 

A DRS is just a XML configuration file listing the location or hash of a jar and the action to take. You can bypass some of the pop-ups (some pop-ups can’t be disabled such as JAR unsigned), you can flat out block the jar or run the default actions as you would if DRS wasn’t defined. The XML file is parse sequentially so place your allowed jars at the top of the file and place a catch all block rule at the bottom. 

Deployment of the rule set is as simple as packing in a signed (from a trusted 3rd party) jar file named DeploymentRuleSet.jar and deploying to the endpoints to be controlled. 

Java based whitelisting is a very powerful feature but it is limited. With MSFT based whitelisting users can individually whitelist java for their own uses, but it is limited only to IE. Java based on the other hand does stop end users from whitelisting however it is a larger effort to whitelist and you either have to manage many lists for individual or groups of users or you have to whitelist sites for everyone, package and re-push.

Also, deploy EMET



Oracle docs on the setup.

Push TrustedSites via GPO

Java Whitelisting from MSDN

Thwarting Client Side Attacks with Software Restriction Policy

A few weeks ago I started looking at Windows Software Restriction Policy (SRP) and using it to stop client side attacks. This is going to go over some of the options, setup and the results once enabled.

SRP is easy to setup via Group Policy Object (GPO). Inside GPO editor create New Software Restriction Policy. Once create the default will be setup. You can look around to see basic options. Here is my tested setup.

Enforcement: Select “All Software files” and “All users except local administrators”

Enforcement Properties















Under Designated File types: Remove type LNK – this will make sure that shortcuts placed outside of the designated execution directories will run. When I initially tested what I thought would work none of the shortcuts on the toolbar or desktop would launch an application and I found this to be the issue.
















Ignore trusted publishers, this is used if we are limiting applications based on the certificate authority.

Select “Additional Rules”

The default execution directories will be selected.

%HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SystemRoot%

Since mine is 64bit Windows I added

%HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\ProgramFilesDir (x86)%

Security level for these are all going to be “Unrestricted” I want them to be able to execute as normal.

Now back under “Security Levels” the default setting is Unrestricted, since we are changing users over to defined execution directories I want to set anything not specifically allowed in the Additional Rules section to “Disallowed.” So we change the default to Disallowed.

Save this and run gpupdate /force on the target machine.

Now to test a client side attack using SET. I am going to use the java attack method. 1 -> Social-Engineering Attacks, 2 -> Website Attack Vectors, 1 -> Java Applet Attack Method, 1 -> Web Templates, 1 -> Java Required, 2 -> Windows Reverse_TCP Meterpreter, 16 -> Backdoored Executable – Enter port of listener (default 443)

Fire it up and wait till it starts the payload handler.

SET Launch

















Once the handler is started you are ready to test the attack. Go ahead and run the unsafe java applet.

Java Applet

















You will notice that the the site is responding but the java applet is unable to execute the payload.

SET Failure

















After attempting this and being successful, I tried running SET with PowerShell Injection and to my surprise the attack succeeded. I realized with PowerShell the payload was running from the C:\Windows\sysWOW64\WindowsPowerShell directory which by default is explicitly allowed. To defeat this attack I added the path to the list of Additional Rules and set it to “Basic User”, retested the attack with PS Injection and the attack failed as expected. I tested this with multiple payloads and encoding methods and everyone of them did not result in a successful attack. On 32bit systems PowerShell is located in C:\Windows\System32\WindowsPowerShell\v1.0\ so that directory will need to be accounted for as well.

I ran two other tests, the first was using EXE embedded PDF and an older version of Adobe Reader (9.3). SRP was able to successfully stop this attack.

Finally I tested a physical attack using a USB Rubber Ducky Human Interface Device (HID) from the folks over at hak5 ( I used a great little payload generator found over on google code ( ) It is pretty slick and simple, I used a meterpreter powershell injection payload that didn’t attempt to elevate privileges. SRP was able to successfully stop this attack. If the user had admin privileges and entered in creds in the UAC window it would have worked since I allow Local Admins unrestricted access.

In Production the are likely other directories where code needs to execute, those will need to be added to the allow list. As the config is done, administrators will be able to bypass these rules for installation of software etc. Administrators will also need to ensure that ACLs are properly set since a curious user could move executables into the approved directories and run them. While this is like a bit tough to implement in a very large organization this is a very effective method for stopping client side attacks.

To find other executable directories in use in your environment enable SRP with defaults (fully unrestricted) and set the following registry key:

String Value: LogFileName, <path to log file>

This will log the executable and the directory it was run from a little data mining can determine were applications need to execute from. Also Inventory Collector from Application Compatibility toolkit can assist in this task.

Garage Door Sensor – Part 1

All the parts to create the first project has arrived.

IMG 1550


First task was to solder up the Pi Cobbler kit. Instructions can be found here

IMG 1551

After soldering up the pins on the cobbler kit I hooked everything up on the solder-less breadboard.


IMG 1552


After getting all that done I grabbed a quick python script to verify that my setup is actually working. Next step is to learn some python for the Pi so I can write my own scripts to progress.



First Hardware Project

I picked up a Raspberry Pi several months ago after being on a waiting list. I knew eventually I would find something to do with it. To get started I am going create a simple garage door sensor. There have been many times where I pulled away from the house and couldn’t remember if I put the garage door down. The plan is to have a sensor test if the door is down and have a web page display the status of the door. Eventually I want to expand it to actually lower or raise the door (Still planning that). I will be updating here with the project.

For now here is the shopping list:

Raspberry Pi 

Breadboarding wire bundle 


Adafruit Pi Cobbler Breakout Kit

MCP3008 – 8-Channel 10-Bit ADC With SPI Interface

Force-Sensitive Resistor

Everything is on order, once it arrives the project really begins. Pi is loaded with Rasbian and on the network.