Loading... <h1>1<span> </span><span style="font-family:宋体">简介</span></h1> <p style="text-indent: 24px"><span>本章我们来介绍Centos6.5基于SSL密码认证部署ELK(Elasticsearch 1.4.4+Logstash 1.4.2+kibana3),同时为大家介绍如何集合如上组件来收集日志,本章的日志收集主要为大家介绍SYSTEM日志收集.</span></p> <p style="text-indent: 24px"><span>集中化日志收集主要应用场景是在同一个窗口临时性或永久性鉴定分析系统,应用等各类日志,对用户提供极大便利,同时也为用户提供一定自主性展示方式</span></p> <h1>2<span> </span><span style="font-family:宋体">本文目标</span></h1> <p style="text-indent: 18px"><span>为大家介绍用logstash收集多目标主机syslogs日志,同时用kibana来分析展示收集到的日志</span></p> <h2>2.1<span> </span><span style="font-family:宋体">四大组件介绍</span></h2> <p><strong><span>Logstash:</span></strong><span> logstash server</span><span>端用来录入日志</span></p> <p><strong><span>Elasticsearch: </span></strong><span>存储各类日志</span></p> <p><strong><span>Kibana: </span></strong><span>web</span><span>化接口用作查寻和可视化日志</span></p> <p><strong><span>Logstash Forwarder: </span></strong><span>logstash client</span><span>端用来通过<em>lumberjack</em> 网络协议发送日志到logstash server</span></p> <p><span> </span></p> <p><span>我们将安装前三个组件到一台服务器上,这台机器将作为我们的<strong>logstash Server.</strong> <strong>Logstash Forwarder</strong> 将安装在所有需要被收集日志的服务器,所有日志将被发送给<strong>Logstash Server</strong>.</span></p> <h2>2.2<span> </span><span style="font-family:宋体">基本概念</span></h2> <p><strong><span>NRT:</span></strong><span> Near RealTime(NRT)</span><span>时时分析系统,延迟在1秒内;</span></p> <p><strong><span>Cluster</span></strong><strong><span>: </span></strong><span>集群的通过name作为唯一标识,默认</span><span>elasticsearch</span><span>;</span></p> <p><strong><span>Node: </span></strong><span>part of cluster,stores data,a single cluster can have many nodes as want.if no elasticsearch nodes running on your network,starting a single node will be default form a new single-node cluster named <em>elasticsearch</em></span></p> <p><strong><span>Index: </span></strong><span>索引必须小写,</span><span>in a single cluster,you can define as many indeses as u want.</span></p> <p><strong><span>Type: </span></strong><span>one index, u can define one or more types.</span></p> <p><strong><span>Document:</span></strong> <span>最小被索引单位</span><span>,</span><span>例如一个文档为单个用户准备,另外一个为单产品介绍准备,还有一个是为单据准备。以</span><span>json</span><span>的方式切割。</span><span>Index/type</span><span>可存储多个</span><span>documents.</span></p> <p><strong><span>Shards & replicas:</span></strong><span> index can store a billion documents taking up 1TB of disk space, single node may be not fit, and may bo too slow to serve search requests from a single node alone. <em><span style="color:red">To solve this problem, Elasticsearch provides subdivide the indes into multiple pieces called shards. When create an index, we can simple define the num of shards that we want.</span></em></span></p> <p><em><span>Sharding two primary reasons:</span></em></p> <p class="MsoListParagraph" style="margin-left:56px"><span style="font-family:Wingdings">l<span> </span></span><span>Horizontally split/scale content volume</span><span>(方便纵向切割或横向扩展)</span></p> <p class="MsoListParagraph" style="margin-left:56px"><span style="font-family:Wingdings">l<span> </span></span><span>Allow distribute distribute and parallelize operation shards (</span><span>允许并行或分布式操作碎片</span><span>)each index in Elasticsearch is allocated 5 primary shards and 1 replica which means that if u’ve at least two nodes in cluster, u index will have 5 primary shards and another 5 replica shards</span><span>(</span><span>1 complete replica</span><span>)</span><span>for total of 10 shards per index.</span></p> <h1>3<span> </span><span style="font-family:宋体">部署环境</span></h1> <h2>3.1<span> </span><span style="font-family:宋体">前期环境准备</span></h2> <table cellspacing="0" cellpadding="0" width="641"> <tbody> <tr style="height:22px" class="firstRow"> <td width="641" nowrap="" colspan="7" valign="bottom" style="border-color: windowtext black windowtext windowtext;border-width: 1px;padding: 0px 7px" height="22"> <p style="text-align:center"><strong><span style="font-size: 19px;font-family: 微软雅黑, sans-serif">ELK</span></strong><strong><span style="font-size: 19px;font-family: 微软雅黑, sans-serif">硬件测试环境</span></strong></p> </td> </tr> <tr style="height:15px"> <td width="72" nowrap="" height="15"> <p><strong><span style="font-size: 16px;font-family: 黑体">HostName</span></strong></p> </td> <td width="101" nowrap="" height="15"> <p><strong><span style="font-size: 16px;font-family: 黑体">InnerIp</span></strong></p> </td> <td width="65" nowrap="" height="15"> <p><strong><span style="font-size: 16px;font-family: 黑体">OuterIp</span></strong></p> </td> <td width="72" nowrap="" height="15"> <p><strong><span style="font-size: 16px;font-family: 黑体">HardWare</span></strong></p> </td> <td width="64" nowrap="" height="15"> <p><strong><span style="font-size: 16px;font-family: 黑体">System</span></strong></p> </td> <td width="152" nowrap="" height="15"> <p><strong><span style="font-size: 16px;font-family: 黑体">Version</span></strong></p> </td> <td width="115" nowrap="" height="15"> <p><strong><span style="font-size: 16px;font-family: 黑体">Role</span></strong></p> </td> </tr> <tr style="height:44px"> <td width="72" style="border-right-color: windowtext;border-bottom-color: windowtext;border-left-color: windowtext;border-right-width: 1px;border-bottom-width: 1px;border-left-width: 1px;border-top-style: none;padding: 0px 7px" height="44"> <p><span style="font-size: 16px;font-family: 黑体">AppS2</span></p> </td> <td width="101" style="border-top-style: none;border-left-style: none;border-bottom-color: windowtext;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="44"> <p><span style="font-size: 16px;font-family: 黑体">192.168.1.38</span></p> </td> <td width="65" style="border-top-style: none;border-left-style: none;border-bottom-color: windowtext;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="44"> <p><span style="font-size: 16px;font-family: 黑体">\</span></p> </td> <td width="72" rowspan="3" style="border-top-style: none;border-left-style: none;border-bottom-color: black;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="44"> <p><span style="font-size: 16px;font-family: 黑体">RAM:1GB<br /> CPU:1</span></p> </td> <td width="64" rowspan="3" style="border-top-style: none;border-left-style: none;border-bottom-color: black;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="44"> <p style="text-align:center"><span style="font-size: 16px;font-family: 黑体">CentOS release 6.5 (Final)</span></p> </td> <td width="152" style="border-top-style: none;border-bottom-style: none;border-left-style: none;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="44"> <p><span style="font-size: 16px;font-family: 黑体">ElasticSearch:1.4.2<br /> LogStash: 1.4.2<br /> Kibana: 3.0.1</span></p> </td> <td width="115" style="border-top-style: none;border-left-style: none;border-bottom-color: windowtext;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="44"> <p><span style="font-size: 16px;font-family: 黑体">ELK Server</span></p> </td> </tr> <tr style="height:30px"> <td width="72" style="border-right-color: windowtext;border-bottom-color: windowtext;border-left-color: windowtext;border-right-width: 1px;border-bottom-width: 1px;border-left-width: 1px;border-top-style: none;padding: 0px 7px" height="30"> <p><span style="font-size: 16px;font-family: 黑体">AppS3</span></p> </td> <td width="101" style="border-top-style: none;border-left-style: none;border-bottom-color: windowtext;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="30"> <p><span style="font-size: 16px;font-family: 黑体">192.168.1.39</span></p> </td> <td width="65" style="border-top-style: none;border-left-style: none;border-bottom-color: windowtext;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="30"> <p><span style="font-size: 16px;font-family: 黑体">\</span></p> </td> <td width="152" style="border-top-color: windowtext;border-right-color: windowtext;border-bottom-color: windowtext;border-top-width: 1px;border-right-width: 1px;border-bottom-width: 1px;border-left-style: none;padding: 0px 7px" height="30"> <p><span style="font-size: 16px;font-family: 黑体">0.3.1</span></p> </td> <td width="115" style="border-top-style: none;border-left-style: none;border-bottom-color: windowtext;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="30"> <p><span style="font-size: 16px;font-family: 黑体">Logstash Forwarder</span></p> </td> </tr> <tr style="height:15px"> <td width="72" style="border-right-color: windowtext;border-bottom-color: windowtext;border-left-color: windowtext;border-right-width: 1px;border-bottom-width: 1px;border-left-width: 1px;border-top-style: none;padding: 0px 7px" height="15"> <p><span style="font-size: 16px;font-family: 黑体">Manager</span></p> </td> <td width="101" style="border-top-style: none;border-left-style: none;border-bottom-color: windowtext;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="15"> <p><span style="font-size: 16px;font-family: 黑体">192.168.1.40</span></p> </td> <td width="65" style="border-top-style: none;border-left-style: none;border-bottom-color: windowtext;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="15"> <p><span style="font-size: 16px;font-family: 黑体">\</span></p> </td> <td width="152" style="border-top-style: none;border-left-style: none;border-bottom-color: windowtext;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="15"> <p><span style="font-size: 16px;font-family: 黑体">ansible 1.8.2</span></p> </td> <td width="115" style="border-top-style: none;border-left-style: none;border-bottom-color: windowtext;border-bottom-width: 1px;border-right-color: windowtext;border-right-width: 1px;padding: 0px 7px" height="15"> <p><span style="font-size: 16px;font-family: 黑体">AnsibleManager</span></p> </td> </tr> </tbody> </table> <h2>3.2<span> </span>Server<span style="font-family:宋体">环境配置</span></h2> <h3>3.2.1<span> </span>Install Java 7</h3> <p><span>ELK</span><span>环境基于JAVA 7环境运行,安装命令如下</span></p> <pre class="brush:bash;toolbar:false"># yum install java-1.7.0-openjdk -y</pre> <h3>3.2.2<span> </span>Install ElasticSearch</h3> <p><em>//import the ElasticSearch GPG key into rpm</em></p> <pre class="brush:bash;toolbar:false"># rpm --import http://packages.elasticsearch.org/GPG-KEY-elasticsearch</pre> <p>//create new yum repository file for ElasticSearch</p> <pre class="brush:bash;toolbar:false"># vi /etc/yum.repos.d/elasticsearch.repo</pre> <p><em><span>//</span></em><em><span>添加如下内容到elasticsearch.repo</span></em></p> <table cellspacing="0" cellpadding="0"> <tbody> <tr class="firstRow"> <td width="945" valign="top" style="border-color: windowtext;border-width: 1px;padding: 0px 7px"> <p><span style="background:#D9D9D9"></span></p> <pre class="brush:bash;toolbar:false">[elasticsearch-1.4] name=Elasticsearch repository for 1.4.x packages baseurl=http://packages.elasticsearch.org/elasticsearch/1.4/centos gpgcheck=1 gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch enabled=1</pre> </td> </tr> </tbody> </table> <p><strong>//<span style="font-family:宋体">安装</span>elasticsearch</strong></p> <pre class="brush:bash;toolbar:false"># yum install elasticsearch-1.4.1 –y //编辑/etc/elasticsearch/elasticsearch.yml script.disable_dynamic: true //增加行 network.host: localhost //取消注释 防止外部通过HTTP API访问Elasticsearch实例随意读取甚至shutdown Elasticsearch Clustaer discovery.zen.ping.multicast.enabled: false //取消注释 禁用广播</pre> <h3>3.2.3<span> </span>start Elasticsearch</h3> <pre class="brush:bash;toolbar:false"># service elasticsearch restart</pre> <p>//<span style="font-family:宋体">加入到开机启动项</span></p> <pre class="brush:bash;toolbar:false"># /sbin/chkconfig --add elasticsearch</pre> <h3>3.2.4<span> </span>Install Kibana</h3> <p><span></span></p> <pre class="brush:bash;toolbar:false"># cd /data/software; curl -O https://download.elasticsearch.org/kibana/kibana/kibana-3.0.1.tar.gz # tar -xvf kibana-3.0.1.tar.gz # vim kibana-3.0.1/config.js //修改9200端口号为80</pre> <p><span>elasticsearch: "http://"+window.location.hostname+":80",</span></p> <p><span>//</span><span style="font-size:12px;font-family:宋体">创建</span><span>nginx</span><span style="font-size:12px;font-family:宋体">下的</span><span>kibana</span><span style="font-size:12px;font-family:宋体">目录</span></p> <p><span></span></p> <pre class="brush:bash;toolbar:false"># mkdir -p /usr/share/nginx/kibana3 # cp -R * /usr/share/nginx/kibana3/</pre> <h3>3.2.5<span> </span>Install Logstash</h3> <p><span>Logstash</span><span style="font-family:宋体">提供了</span><span>yum</span><span style="font-family:宋体">安装方式</span><span>,</span></p> <pre class="brush:bash;toolbar:false"># vim /etc/yum.repos.d/logstash.repo</pre> <p><span>//</span><span style="font-family:宋体">增加如下配置</span></p> <table cellspacing="0" cellpadding="0"> <tbody> <tr class="firstRow"> <td width="945" valign="top" style="border-color: windowtext;border-width: 1px;padding: 0px 7px"> <p><span></span></p> <pre class="brush:bash;toolbar:false">[logstash-1.4] name=logstash repository for 1.4.x packages baseurl=http://packages.elasticsearch.org/logstash/1.4/centos gpgcheck=1 gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch enabled=1</pre> </td> </tr> </tbody> </table> <p><span style="font-family:宋体">安装</span></p> <pre class="brush:bash;toolbar:false"># yum -y install logstash-1.4.</pre> <h3>3.2.6<span> </span>Install Nginx</h3> <pre class="brush:bash;toolbar:false"># yum install nginx</pre> <p><span>//kibana</span><span style="font-size:12px;font-family:宋体">默认是使用</span><span>Elasticsearch</span><span style="font-size:12px;font-family:宋体">的</span><span>9200</span><span style="font-size:12px;font-family:宋体">端口</span><span>,</span><span style="font-size: 12px;font-family:宋体">但用户可以有权限直接访问</span><span>Elasticsearch,</span><span style="font-size: 12px;font-family:宋体">所以我们通过</span><span>web Serverr 80</span><span style="font-size: 12px;font-family:宋体">端口代替访问</span><span>9200</span><span style="font-size:12px;font-family:宋体">端口</span><span>,Kibana</span><span style="font-size:12px;font-family:宋体">也提供了关于</span><span>nginx</span><span style="font-size:12px;font-family:宋体">的配置文件供大家直接下载使用</span><span>.</span></p> <pre class="brush:bash;toolbar:false"># curl -OL https://gist.githubusercontent.com/thisismitch/2205786838a6a5d61f55/raw/f91e06198a7c455925f6e3099e3ea7c186d0b263/nginx.conf</pre> <p><span>//nginx.conf</span><span style="font-size:12px;font-family:宋体">配置</span></p> <table cellspacing="0" cellpadding="0"> <tbody> <tr class="firstRow"> <td width="945" valign="top" style="border-color: windowtext;border-width: 1px;padding: 0px 7px"> <p><span></span></p> <pre class="brush:bash;toolbar:false"># cat nginx.conf # # Nginx proxy for Elasticsearch + Kibana # # In this setup, we are password protecting the saving of dashboards. You may # wish to extend the password protection to all paths. # # Even though these paths are being called as the result of an ajax request, the # browser will prompt for a username/password on the first request # # If you use this, you'll want to point config.js at http://FQDN:80/ instead of # http://FQDN:9200 # server { listen *:80 ; server_name kibana2.ihuilian.com.; access_log /var/log/nginx/kibana2.access.log; location / { root /usr/share/nginx/kibana3; index index.html index.htm; } location ~ ^/_aliases$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } location ~ ^/.*/_aliases$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } location ~ ^/_nodes$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } location ~ ^/.*/_search$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } location ~ ^/.*/_mapping { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; } # Password protected end points location ~ ^/kibana-int/dashboard/.*$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; limit_except GET { proxy_pass http://127.0.0.1:9200; auth_basic "Restricted"; auth_basic_user_file /etc/nginx/conf.d/kibana2.htpasswd; } } location ~ ^/kibana-int/temp.*$ { proxy_pass http://127.0.0.1:9200; proxy_read_timeout 90; limit_except GET { proxy_pass http://127.0.0.1:9200; auth_basic "Restricted"; auth_basic_user_file /etc/nginx/conf.d/kibana2.htpasswd; } } }</pre> </td> </tr> </tbody> </table> <p><span>//</span><span style="font-size:12px;font-family:宋体">保存退出后</span></p> <pre class="brush:bash;toolbar:false"># cp nginx.conf /etc/nginx/conf.d/default.conf</pre> <p><span>//</span><span style="font-size:12px;font-family:宋体">安装</span><span>apache2-utils</span><span style="font-size:12px;font-family:宋体">用</span><span>htpasswd</span><span style="font-size:12px;font-family:宋体">来生成用户名和密码对</span><span>:</span></p> <pre class="brush:bash;toolbar:false"># yum install httpd-tools-2.2.15 –y</pre> <p><span>//</span><span style="font-size:12px;font-family:宋体">生成用户名密码</span></p> <pre class="brush:bash;toolbar:false"># htpasswd -c /etc/nginx/conf.d/kibana2.htpasswd user</pre> <p><span>//</span><span style="font-size:12px;font-family:宋体">启动</span><span>Nginx</span></p> <pre class="brush:bash;toolbar:false"># service nginx restart</pre> <p><span>//</span><span style="font-size:12px;font-family:宋体">添加开机启动</span></p> <pre class="brush:bash;toolbar:false"># chkconfig nginx on</pre> <h3>3.2.7<span> </span>SSL<span style="font-family:宋体">认证</span></h3> <p><span style="font-family:宋体">如上文所述</span>,<span style="font-family:宋体">安全起见</span>,<span style="font-family:宋体">我们</span>elasticsearch<span style="font-family:宋体">采用</span>web<span style="font-family:宋体">方式访问</span>,<span style="font-family:宋体">通过</span>ssl<span style="font-family:宋体">认证的方式提高访问安全性。</span></p> <pre class="brush:bash;toolbar:false"># vim /etc/pki/tls/openssl.cnf</pre> <p> //[v3_ca]<span style="font-family: 宋体">下添加如下配置</span></p> <p>subjectAltName=IP: 192.168.1.38</p> <p><span style="font-family:宋体">生成</span>ssl<span style="font-family:宋体">认证文件</span></p> <p><span></span></p> <pre class="brush:bash;toolbar:false"># cd /etc/pki/tls # openssl req -config /etc/pki/tls/openssl.cnf -x509 -days 3650 -batch -nodes -newkey rsa:2048 -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt</pre> <table cellspacing="0" cellpadding="0"> <tbody> <tr class="firstRow"> <td width="945" valign="top" style="border-color: windowtext;border-width: 1px;padding: 0px 7px"> <p><span>Generating a 2048 bit RSA private key</span></p> <p><span>……………………………….+++</span></p> <p><span>…………………..+++</span></p> <p><span>writing new private key to 'private/logstash-forwarder.key'</span></p> <p><span>—–</span></p> </td> </tr> </tbody> </table> <h3>3.2.8<span> </span><span style="font-family:宋体">配置</span>logstash</h3> <p><span>Logstach</span><span style="font-family:宋体">配置文件是</span><span>jason</span><span style="font-family:宋体">格式</span><span>,</span><span style="font-family:宋体">配置文件在</span><span>/etc/logstash/conf.d</span><span style="font-family:宋体">下</span><span>,</span><span style="font-family:宋体">配置文件主要包括三部分</span><span>:inputs,filters,outputs:</span></p> <p><span style="font-family:宋体">先创建</span><span>input</span><span style="font-family:宋体">文件</span><span> 01-lumberjack-input.conf</span><span style="font-family:宋体">采用</span><span>lumberjack input </span><span style="font-family:宋体">协议</span><span>logstash forwarder</span><span style="font-family:宋体">使用</span><span>.</span></p> <p><strong><span>Input</span></strong><strong><span style="font-family:宋体">配置如下内容</span></strong><strong><span>:</span></strong></p> <pre class="brush:bash;toolbar:false"># vim /etc/logstash/conf.d/01-lumberjack-input.conf</pre> <table cellspacing="0" cellpadding="0"> <tbody> <tr class="firstRow"> <td width="945" valign="top" style="border-color: windowtext;border-width: 1px;padding: 0px 7px"> <p><span></span></p> <pre class="brush:bash;toolbar:false">input { lumberjack { //定义采用lumberjack协议来收集日志 port => 5000 //定义使用5000端口 type => "logs" ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt" ssl_key => "/etc/pki/tls/private/logstash-forwarder.key" } }</pre> <p><span></span></p> </td> </tr> </tbody> </table> <p><strong><span>Filter</span></strong><strong><span style="font-family:宋体">配置如下</span></strong><strong><span>:</span></strong></p> <pre class="brush:bash;toolbar:false"># vim 10-syslog.conf</pre> <table cellspacing="0" cellpadding="0"> <tbody> <tr class="firstRow"> <td width="945" valign="top" style="border-color: windowtext;border-width: 1px;padding: 0px 7px"> <p><span></span></p> <pre class="brush:bash;toolbar:false">#//如下过滤器可以收集到有syslog标签的日志,并用grok来解析日志使之更结构化和可查询 filter { if [type] == "syslog" { grok { match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_progr am}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" } add_field => [ "received_at", "%{@timestamp}" ] add_field => [ "received_from", "%{host}" ] } syslog_pri { } date { match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] } } }</pre> </td> </tr> </tbody> </table> <p><strong><span>Output</span></strong><strong><span style="font-family:宋体">配置</span></strong><strong><span>:</span></strong></p> <pre class="brush:bash;toolbar:false"># vim /etc/logstash/conf.d/30-lumberjack-output.conf</pre> <table cellspacing="0" cellpadding="0"> <tbody> <tr class="firstRow"> <td width="945" valign="top" style="border-color: windowtext;border-width: 1px;padding: 0px 7px"> <p><span></span></p> <pre class="brush:bash;toolbar:false">#//这个配置是基于日志存储在elasticsearch方式 ,通过这种方式 结合下面的规则logstash同时也可以收集不匹配规则的日志,只是这些日志不会被结构化 output { elasticsearch { host => localhost } stdout { codec => rubydebug } }</pre> </td> </tr> </tbody> </table> <p><strong><span style="font-family:宋体">启动</span></strong><strong><span>logstash</span></strong><strong><span style="font-family:宋体">:</span></strong></p> <pre class="brush:bash;toolbar:false"># service logstash restart</pre> <h2>3.3<span> </span>Client<span style="font-family:宋体">环境配置</span></h2> <h3>3.3.1<span> </span><span style="font-family:宋体">安装</span>Logstash Forwarder</h3> <p>//<span style="font-family:宋体">把</span>server SSL<span style="font-family:宋体">认证文件发送到</span>ship<span style="font-family:宋体">服务器</span></p> <p><span style="font-family:宋体">官网下载</span> https://www.elastic.co/downloads/logstash</p> <p>logstash-forwarder-0.4.0-1.x86_64.rpm</p> <p>//<span style="font-family:宋体">通过如下命令安装</span></p> <pre class="brush:bash;toolbar:false"># rpm -ihv logstash-forwarder-0.4.0-1.x86_64.rpm</pre> <p>//<span style="font-family:宋体">添加</span>logstash Forwarder<span style="font-family:宋体">初始化脚本</span></p> <pre class="brush:bash;toolbar:false"># cd /etc/init.d/; sudo curl -o logstash-forwarder http://logstashbook.com/code/4/logstash_forwarder_redhat_init</pre> <pre class="brush:bash;toolbar:false"># chmod +x logstash-forwarder</pre> <p>//init<span style="font-family:宋体">脚本依赖于配置文件</span>/etc/sysconfig/logstash-forwarder</p> <pre class="brush:bash;toolbar:false"># curl -o /etc/sysconfig/logstash-forwarder http://logstashbook.com/code/4/logstash_forwarder_redhat_sysconfig</pre> <p>//<span style="font-family:宋体">编辑并保存</span></p> <pre class="brush:bash;toolbar:false"># vim /etc/sysconfig/logstash-forwarder</pre> <p>//<span style="font-family:宋体">复制</span>SSL<span style="font-family:宋体">认证文件到对应目录下</span></p> <pre class="brush:bash;toolbar:false"># cp /tmp/logstash-forwarder.crt /etc/pki/tls/certs/</pre> <h3>3.3.2<span> </span><span style="font-family:宋体">配置</span>Logstash Forwarder</h3> <p>//<span style="font-family:宋体">编辑并保存</span></p> <p>//ship<span style="font-family:宋体">将连接</span>logstash server<span style="font-family:宋体">的</span>5000<span style="font-family:宋体">端口</span></p> <pre class="brush:bash;toolbar:false"># vim /etc/logstash-forwarder</pre> <table cellspacing="0" cellpadding="0"> <tbody> <tr class="firstRow"> <td width="945" valign="top" style="border-color: windowtext;border-width: 1px;padding: 0px 7px"> <pre class="brush:bash;toolbar:false">{ "network": { "servers": [ "192.168.1.38:5000" ], "timeout": 15, "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt" }, "files": [ { "paths": [ "/var/log/messages", "/var/log/secure" ], "fields": { "type": "syslog" } } ] }</pre> </td> </tr> </tbody> </table> <p>//<span style="font-family:宋体">启动</span>logstash-forwarder</p> <pre class="brush:bash;toolbar:false"># service logstash-forwarder start</pre> <p>//<span style="font-family:宋体">添加开机启动</span></p> <pre class="brush:bash;toolbar:false"># chkconfig --add logstash-forwarder</pre> <p><strong><em><span style="color:red">//</span></em></strong><strong><em><span style="font-family:宋体;color:red">其它任何想收集日志的服务器均按如上配置即可</span></em></strong></p> <h2>3.4<span> </span><span style="font-family:宋体">连接</span>kibana</h2> <p><span>//</span><span style="font-family:宋体">当我们配置完如上的后</span><span>,</span><span style="font-family:宋体">就可以收集所有希望收集的日志信息</span><span>,Kibana</span><span style="font-family:宋体">可以提供一个</span><span>web api</span><span style="font-family:宋体">友好接口供我们使用</span></p> <p><span>//</span><span style="font-family:宋体">在浏览器虽输入</span><span>kibana2.ihuilian.com(</span><span style="font-family:宋体">按你的配置输入</span><span>)</span><span style="font-family:宋体">或</span><span>ip</span><span style="font-family:宋体">来访问</span><span>logstash server</span><span style="font-family: 宋体">。我们最先访问到的是</span><span>kibana welcome page.</span></p> <p><span>//</span><span style="font-family:宋体">点击</span><span>Logstash dashboard</span><span style="font-family:宋体">进行预设置仪表盘</span><span>,</span><span style="font-family:宋体">我们将看到类似如下的柱状图包括日志事件</span><span>,</span><span style="font-family:宋体">日志信息</span><span>(</span><span style="font-family:宋体">如果没有看到这些信息那一定是四个组件的配置有问题</span><span>,</span><span style="font-family:宋体">请检查</span><span>)</span></p> <p><img src="//cto.wang/usr/uploads/2016/07/20160703163718-96.png" title="1434594879131012.png" alt="1.png" /></p> <p><span>//</span><span style="font-family:宋体">进行下来的练习</span></p> <p class="MsoListParagraph" style="margin-left:56px"><span style="font-family:Wingdings">l<span> </span></span><span>Search for “root “ to see if anyone is trying to log into your servers sa root</span></p> <p class="MsoListParagraph" style="margin-left:56px;text-indent:0"><img src="//cto.wang/usr/uploads/2016/07/20160703163718-22.png" title="1434596328853550.png" alt="2.png" /></p> <p class="MsoListParagraph" style="margin-left:56px"><span style="font-family:Wingdings">l<span> </span></span><span>Search for a particular hostname</span></p> <p class="MsoListParagraph" style="margin-left:56px;text-indent:0"><img src="//cto.wang/usr/uploads/2016/07/20160703163718-24.png" title="1434596337171313.png" alt="3.png" /></p> <p class="MsoListParagraph" style="margin-left:56px;text-indent:0"><span style="font-family:宋体">貌似只支持全量匹配</span></p> <p class="MsoListParagraph" style="margin-left:56px"><span style="font-family:Wingdings">l<span> </span></span><span>Change the time frame by selecting an area on the histogram on from the menu above</span></p> <p class="MsoListParagraph" style="margin-left:56px"><span style="font-family:Wingdings">l<span> </span></span><span>Click on masaages below the histogram to see how the data is being filtered</span></p> <h1>4<span> </span>Kibana<span style="font-family:宋体">使用说明</span></h1> <h2>4.1<span> </span><span style="font-family:宋体">控制面板设置</span></h2> <p><span style="font-family:宋体"><img src="//cto.wang/usr/uploads/2016/07/20160703163718-1.png" title="1434596362106156.png" alt="4.png" /></span></p> <p><span style="font-family:宋体"></span></p> <p><img src="//cto.wang/usr/uploads/2016/07/20160703163719-77.png" title="1434596442187860.png" /></p> <p><img src="//cto.wang/usr/uploads/2016/07/20160703163719-9.png" title="1434596444652095.png" /></p> <p><img src="//cto.wang/usr/uploads/2016/07/20160703163719-41.png" title="1434596445535756.png" /></p> <h2>4.2<span> </span><span style="font-family:宋体">自动刷新</span></h2> <p><img src="//cto.wang/usr/uploads/2016/07/20160703163719-49.png" title="1434596485122603.png" alt="8.png" /></p> <p>In fact, you can add any exported dashboard to that directory and access it as http://YOUR-HOST -HERE/index.html#dashboard/file/YOUR-DASHBOARD.json. Neat trick eh?</p> <p>http://kibana.ihuilian.com/#/dashboard/file/default.json</p> <h1>5<span> </span>qa</h1> <p><span style="font-family:宋体">信息收集慢</span></p> <p><span style="font-family:宋体">没有找到文件的匹配规则</span></p> <h2>5.1<span> </span><span style="font-family:宋体">添加新</span>ship<span style="font-family:宋体">失败</span>,<span style="font-family:宋体">一直无法显示</span></h2> <p class="MsoListParagraph" style="margin-left:56px">a)<span> </span><span style="font-family:宋体">查看日志</span> <span style="font-family:宋体">无异常</span></p> <p class="MsoListParagraph" style="margin-left:56px">b)<span> </span><span style="font-family:宋体">确认</span>SSH<span style="font-family:宋体">认证文件</span> <span style="font-family:宋体">正常</span></p> <p class="MsoListParagraph" style="margin-left:56px">c)<span> </span><span># service logstash-forwarder restart</span> <strong><em><span style="font-family:宋体;color:red">正常</span><span style="color:red">-(restart</span></em></strong><strong><em><span style="font-family:宋体;color:red">失败但返回正常</span><span style="color:red">,</span></em></strong><strong><em><span style="font-family:宋体;color:red">其实是有问题我没有发现</span><span style="color:red">,</span></em></strong><strong><em><span style="font-family:宋体;color:red">只相信系统最原始的命令</span><span style="color:red">,</span></em></strong><strong><em><span style="font-family:宋体;color:red">第三方脚本经常会有不同程度的问题</span><span style="color:red">)</span></em></strong></p> <p class="MsoListParagraph" style="margin-left:56px"><span>d)<span> </span></span><span style="font-family:宋体">在</span><span>server</span><span style="font-family:宋体">端</span><span>restart logstash,elastashsearch,kibana,nginx</span><span style="font-family:宋体">均无法发现主机</span></p> <p class="MsoListParagraph" style="margin-left:56px"><span>e)<span> </span></span><span style="font-family:宋体">全新部署</span><span>ship</span><span style="font-family:宋体">环境</span><span>,</span><span style="font-family:宋体">每步均进行详细确认</span></p> <p class="MsoListParagraph" style="margin-left:56px"><span>f)<span> </span></span><span style="font-family:宋体">发现</span><span>logstash-forwarder</span><span style="font-family:宋体">脚本问题</span><span>,</span><span style="font-family:宋体">修改后正常添加新主机</span></p> <p><span> </span></p> <p><span style="font-family:宋体">大日志是</span><span>100</span><span style="font-family:宋体">条的方式逐渐累加</span></p> <p><span style="font-family:宋体"><img src="//cto.wang/usr/uploads/2016/07/20160703163719-27.png" title="1434596529396683.png" alt="9.png" /><img src="//cto.wang/usr/uploads/2016/07/20160703163720-67.png" title="1434596541123673.png" alt="10.png" /></span></p> <h1>6<span> </span><span style="font-family:宋体">监控</span>nginx<span style="font-family:宋体">日志</span></h1> <p><span>//</span><span style="font-family:宋体">定义</span><span>Nginx</span><span style="font-family:宋体">日志格式</span></p> <table cellspacing="0" cellpadding="0"> <tbody> <tr class="firstRow"> <td width="945" valign="top" style="border-color: windowtext;border-width: 1px;padding: 0px 7px"> <p><span>log_format logstash '$http_host $remote_addr [$time_local] "$request" $status $body_bytes_sent "$http_referer"</span></p> <p><span>"$http_user_agent" $request_time $upstream_response_time';</span></p> <p><span> access_log /var/log/nginx/AppM.access.log logstash;</span></p> </td> </tr> </tbody> </table> <p><span>//</span><span style="font-family:宋体">修改</span><span>logstash-forwarder</span></p> <pre class="brush:bash;toolbar:false"># vim /etc/logstash-forwarder</pre> <table cellspacing="0" cellpadding="0"> <tbody> <tr class="firstRow"> <td width="945" valign="top" style="border-color: windowtext;border-width: 1px;padding: 0px 7px"> <p><span></span></p> <pre class="brush:bash;toolbar:false">{ "network": { "servers": [ "192.168.1.38:5000" ], "timeout": 15, "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt" }, "files": [ { "paths": [ "/var/log/messages*", "/var/log/secure*" ], "fields": { "type": "syslog" } },{ "paths": [ "/var/log/nginx/AppM.access.log*" ], "fields": { "type": "nginx-access" } } ] }</pre> </td> </tr> </tbody> </table> <p><span style="font-family:宋体">重启</span><span>logstash-forwarder</span><span style="font-family:宋体">生效</span></p> <h1>7<span> </span><span style="font-family:宋体">参考文档</span>:</h1> <p>https://www.digitalocean.com/community/tutorials/how-to-use-logstash-and-kibana-to-centralize-logs-on-centos-6</p> <p>http://www.wklken.me/posts/2015/04/26/elk-for-nginx-log.html</p> <p>http://www.cnblogs.com/yjf512/p/4199105.html</p> <p>http://www.tuicool.com/articles/UnUzimJ</p> <p>http://www.learnes.net/getting_started/README.html</p> <p>http://bigbo.github.io/pages/2015/02/28/elasticsearch_hadoop/</p> <p>https://github.com/lmenezes/elasticsearch-kopf</p> <p>http://logstash.es/</p> <p>https://github.com/chenryn/kibana-guide-cn/blob/master/v4/dashboard.md</p> <p>http://kibana.logstash.es/content/</p> <p> </p> <p></p> 最后修改:2021 年 12 月 10 日 10 : 53 AM © 允许规范转载 赞赏 如果觉得我的文章对你有用,请随意赞赏 赞赏作者 支付宝微信