MySQL Audit using Percona audit plugin and ELK

MySQL Audit
using percona audit plugin & ELK

Object
3
We need
- log to MySQL connection information
- log to MySQL query log
- retrieve these logs using various search conditions
- download search results

Architecture
Percona Audit
Plugin
4
DB Servers ELK Server
logstash elasticsearch

Audit Plugin
Which audit plugin will we use? 5
Percona Audit Plugin MariaDB Audit Plugin
- support various log format(csv, json, xml)
- most of parameter can’t modify online
- support various filtering condition
(user, query type : ddl, dml, dcl)
- most of parameters can modify online
- stability issue(db crash)
https://mariadb.atlassian.net/browse/MDEV-5145
http://mysql.az/audit-mysql-no-just-crash-it/
- easy install & config
- can log at extra log file or syslog file
We chose Percona Audit Plugin

6
Percona Audit Plugin Install & Config
6
mysql>install plugin audit_log soname ‘audit_log.so’;
Ref URL : https://www.percona.com/doc/percona-server/5.5/management/audit_log_plugin.html
$/etc/init.d/mysql restart
$vi my.cnf
# Server Audit
audit_log_format = JSON
audit_log_policy = ALL
audit_log_syslog_facility = LOG_LOCAL1
audit_log_handler = SYSLOG
1. Percona Audit Plugin Install (over MySQL Ver 5.5.39) – on DB servers
2. Parameter configuration
3. MySQL restart

rsyslog config
7
1. rsyslog.conf – on DB servers
$ vi /etc/rsyslog.conf
# mysql logging
local1.* @10.xxx.xxx.xxx # ELK server ip
2. rsyslog restart
$ /etc/init.d/rsyslog restart
3. confirm log messages on syslog file
$cat messages
Jan 29 15:08:30 testdbsvr01 percona-audit: {"audit_record":{"name":"Query","record":"3683778651_1970-01-
01T00:00:00","timestamp":"2016-01-29T06:08:30
UTC","command_class":"select","connection_id":"455338789","status":0,"sqltext":"SELECT * FROM test_table
WHERE status='Done'","user":"app[app] @ [10.xxx.xxx.xxx]","host":"","os_user":"","ip":"10.xxx.xxx.xxx"}}

8
rsyslog-ng config
1. rsyslog.conf – on ELK servers
$ vi /etc/syslog-ng/syslog-ng.conf
source s_sys {
file ("/proc/kmsg" program_override("kernel: "));
unix-stream ("/dev/log");
internal();
udp(ip(0.0.0.0) port(514)); ## uncomment this line
};
destination d_myaudit { file("/var/log/myaudit/myaudit.${HOST}.${YEAR}-${MONTH}-${DAY}.log" create-dirs(yes) dir-perm(0755) perm(0644) ); };
filter f_myaudit { facility(local1); };
log { source(s_sys); filter(f_myaudit); destination(d_myaudit); };
2. syslog-ng restart
$ /etc/init.d/syslog-ng restart

Install ELK
Elasticsearch 9
1. elasticsearch install – on ELK server
$ yum install elasticsearch
2. configuration
$ vi /etc/elasticsearch/elasticsearch.yml
cluster.name : my_cluster # cluster name
node.name : my_node01 # node name
network.host : 10.xxx.xxx.xxx # server’s ip
You can download from
https://www.elastic.co/downloads/elasticsearch.
It needs a recent version of java before install elasticsearch.

Install ELK
Elasticsearch 10
3. start elasticsearch
$/etc/init.d/elasticsearch start
4. Head plugin install
$cd /usr/share/elasticsearch/bin
$./plugin install mobz/elasticsearch-head
5. Check plugin install
http://10.xxx.xxx.xxx:9200/_plugin/head/

1111
Install ELK
Logstash 11
1. Install Logstash – on ELK server
$ rpm –ivh logstash-2.1.1-1.noarch.rpm
2-1. Configure ( input plugin )
$ sudo vi /etc/logstash/conf.d/01-myaudit-input.conf
input {
file {
type => "myaudit"
path => ["/var/log/myaudit/*.log"]
start_position => "beginning"
codec => 'json'
}
}
https://www.elastic.co/downloads/logstash

12
Install ELK
Logstash
2-2. Configure ( filter plugin )
$ sudo vi /etc/logstash/conf.d/10-myaudit.conf
filter {
grok {
match => { "message" =>
"%{SYSLOGTIMESTAMP:sys_timestamp}%{SPACE}%{HOSTNAME:host_name}
%{SPACE} percona-audit: %{GREEDYDATA:json_data}"}
}
json {
source => "json_data"
}
}

13
Install ELK
Logstash
2-3. Configure ( output plugin )
$ sudo vi /etc/logstash/conf.d/30-elasticsearch-output.conf
output {
elasticsearch {
hosts => "10.xxx.xxx.xxx"
}
}
3. Start logstash
$ sudo /etc/init.d/logstash start

14
Install ELK
Kibana 14
1. Kibana install – on ELK server
$ tar –xvf kibana-4.3.1-linux-x64.tar.gz
2. configuration
$ vi ./config/kibana.yml
host: “10.xxx.xxx.xxx” # kibana server ip
elasticsearch_url: “http://10.xxx.xxx.xxx:9200” # elasticsearch server ip
4. Check
https://www.elastic.co/downloads/kibana
http://10.xxx.xxx.xxx:5601
3. Start Kibana
$ ./bin/kibana
It needs a space between “host:” and IP.

15
Install ELK
Kibana
5. Config an index pattern

16
Make Kibana Dashboard
http://10.xxx.xxx.xxx:5601

Search condition create(1/3) 17
1
2

Search condition create(2/3) 18
3
4

19
Search condition create(3/3)
5
6

20
1
Make Graph: line chart(1/6)

21
2

22
3
4

23
5
6

24
7
8

25
9

26
Make Graph: Pie chart(1/4)
1

27
2

28
3
4

29
5

30
Markdown widget(1/3)
1

31
1
###Menu: // label, the number of “#” determines a size of character.
[Main]: //label
(/#dashboard/Main): // link dashboard. “Main” is name of dashboard.
You have to make dashboard before markdown widget.
2

32
3

33
Make Dashboard
1

34
2
Make Dashboard

35
3
4
Make Dashboard

Dashboard-sample
36
Markdown widget
Line chart
Search condition

Dashboard-sample
37
Markdown widget
Pie chart

Elasticsearch Plugin – Head(1/2)
38
http://10.xxx.xxx.xxx:9200/_plugin/head/
1
2
3
4
search query

Elasticsearch Plugin – Head(2/2)
39
{
"_source": [
"@timestamp",
"host_name", "audit_record.user", "audit_record.sqltext", "audit_record.name" ],
"sort": { "@timestamp": "asc" },
"size": 100,
"query": {
"bool": {
"must": [
{
"range": {
"@timestamp": {
"gte": "2016-01-16T00:00:00.000",
"lte": "2016-01-16T02:59:59.999" }
} },
{"term": { "host_name": “host_name123" } },
{"term": { "audit_record.name": "query"} }
] } }}
Search condition of string type uses a lower character
only.
ex) “audit_record.name”:”Query” (x)
“audit_record.name”:”query” (o)

MySQL Audit using Percona audit plugin and ELK

Related slideshows

More Related Content

MySQL Audit using Percona audit plugin and ELK