Installing and Configuring ELK

– Part one(1) – Install OS (OEL 7.2), Tuning
– Part two(2) – Configure KVM, Tuning
– Part three(3) – install elasticsearch, configuration
– Part four(4) – Install logstash, configuration
– Part five(5) – Install Kibana, configuration
– Part six(6) – General platform tips & tricks

This is Part four(4) – Install logstash, configuration

Lets install logstash

Note: Make sure you have java 1.8 otherwise install as below

yum install java-1.8.0-openjdk.x86_64

Lets configure logstash repo

rpm --import http://packages.elastic.co/GPG-KEY-elasticsearch

cat /etc/yum.repos.d/logstash.repo
[logstash-2.2]
name=Logstash repository for 2.2.x packages
baseurl=http://packages.elastic.co/logstash/2.2/centos
gpgcheck=1
gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

Now install logstash

yum -y install logstash

Note: An alternative is to get it from there web site and extract in /opt. the rest of this document is referring to the per-packged version

Configure logstash nodes

create /etc/logstash/conf.d/logstash.conf
Note: Please look at the end of this document for a full logstash.conf

Enable and start services

systemctl daemon-reload
systemctl enable logstash.service
systemctl start logstash

Logstash Tips

Testing your grok syntax and pre patterns

https://grokdebug.herokuapp.com/

http://grokdebug.herokuapp.com/patterns

Logstash client config

Below is the logstash.config used for sql.log inputs & parsing

# enable for Live file monitor 
#input {
  #file {
    ## Wildcards work, here :)
    #path => [ "/var/log/*.log", "/var/log/messages", "/var/log/syslog" ]
    #type => "syslog-ng"
  #}
#}

# Genric for all global's
#input {
  #file {
    #path => [ "/zones/dc*-wapp*/root/devtech101logs/application/sql.log" ]
    #type => "sql-log"
  #}
#}

input {
  file {
    path => [ "/devtech101logs/application/sql.log" ]
    type => "sql-log"
  }
}

input {
    tcp {
        host => "10.10.10.10"
        port => 3333
        type => "sql-log"
    }
}


filter {
  if [type] == "sql-log" {
    if [message] =~ /^s*$/ {
      drop { }
    }
    grok {
      match => { "message" => "(?m)%{MONTHDAY:MONTHDAY}%{SPACE}%{MONTH:MONTH}%{SPACE}%{YEAR:YEAR}%{SPACE}%{TIME:TIME}%{SPACE}-%{SPACE}%{LOGLEVEL:LOGLEVEL}%{SPACE}-%{SPACE}%{HOSTNAME:HOSTNAME}%{SPACE}::%{SPACE}%{DATA:SESSION_ID}%{SPACE}::%{SPACE}BHSql%{SPACE}::%{SPACE}%{DATA:DURATION}%{SPACE}::%{SPACE}%{GREEDYDATA:SQL_STATEMENT}" }
    }
    mutate {
      # Replace field
      gsub => [ "TIME", ",", "." ]
      add_field => { "mytimestamp" => "%{MONTHDAY} %{MONTH} %{YEAR} %{TIME}" }
    }
    date {
      match => [ "mytimestamp", "dd MMM YYYY HH:mm:ss.SSS" ]
      timezone => "UTC"
      target => "@timestamp"
    }
    mutate {
      remove_field => [ "mytimestamp", "%{MONTHDAY} %{MONTH} %{YEAR} %{TIME}" ]
    }
  }
}

output {
  #stdout {codec => rubydebug}
  if [type] == "sql-log" {
    elasticsearch{
        hosts => [ "10.10.3.25:9200", "10.10.3.26:9200", "10.10.3.27:9200" ]
        timeout => 30
	index => "web-%{type}-%{+YYYY.MM.dd}"
	#flush_size => 2000
    }
  } else {
    elasticsearch{
        hosts => [ "10.10.3.25:9200", "10.10.3.26:9200", "10.10.3.27:9200" ]
        timeout => 30
	#flush_size => 2000
    }
  }
}

Next Part five(5) – Install Kibana, configuration