ElastAlert教程11章：开始安装elastalert

前面，我们已经将数据从日志文件传输到了ES中，现在就可以通过不断查询ES来产生日志告警了。终于到了介绍elastalert的时候了。

告警工具可以选择 Alert Management、Elasticsearch watch、elastalert等，各有优缺点，有的是收费的，有的是免费的，这里我们选择 elastalert 。

安装elastalert依赖的python3

elastalert 依赖于 Python 3 ，所以需要先安装 Python 3。

你的centos系统一般是python2，可以执行我们的一键安装python3脚本：

这个脚本安装了python3，但是python2还是可以使用，不用担心。

wget http://66-ai.com/download/script-litte-prince/app/install-python3.sh -O /root/install-python3.sh  && sh /root/install-python3.sh

运行例子

如果你对上面的脚本感兴趣，下面是脚本的详细代码：

#!/usr/bin/env bash
# 安装python3
# 自动将yum的命令依赖改为python2，以免引起yum异常

yum install -y openssl-devel bzip2-devel expat-devel gdbm-devel readline-devel sqlite-devel

if ! [ -x "$(command -v axel)" ]; then
  wget http://66-ai.com/download/script-litte-prince/Python-3.6.5.tgz -O /root/Python-3.6.5.tgz
else
  axel -n 10 -a http://66-ai.com/download/script-litte-prince/Python-3.6.5.tgz -o /root/Python-3.6.5.tgz
fi

tar -xzvf Python-3.6.5.tgz
cd Python-3.6.5
./configure --prefix=/usr/local/python
make
make install
cd /usr/bin
mv python python.bak
mv pip pip.bak
ln -s /usr/local/python/bin/python3.6 /usr/bin/python
ln -s /usr/local/python/bin/pip3.6 /usr/bin/pip


python2str=`cat /usr/bin/yum | grep -w '#!/usr/bin/python2'`

if [ "$python2str" = '#!/usr/bin/python2' ]; then
    echo '已经替换过'
else
    sed -i 's@#!/usr/bin/python@#!/usr/bin/python2@g' /usr/bin/yum
    sed -i 's@#!/usr/bin/python@#!/usr/bin/python2@g' /usr/bin/yum-builddep
    sed -i 's@#!/usr/bin/python@#!/usr/bin/python2@g' /usr/bin/yum-config-manager
    sed -i 's@#!/usr/bin/python@#!/usr/bin/python2@g' /usr/bin/yum-debug-dump
    sed -i 's@#!/usr/bin/python@#!/usr/bin/python2@g' /usr/bin/yum-debug-restore
    sed -i 's@#!/usr/bin/python@#!/usr/bin/python2@g' /usr/bin/yumdownloader
    sed -i 's@#!/usr/bin/python@#!/usr/bin/python2@g' /usr/bin/yum-groups-manager
    # 有一个空格
    sed -i 's@#! /usr/bin/python@#!/usr/bin/python2@g' /usr/libexec/urlgrabber-ext-down
fi

运行例子

安装完成之后，执行 python 命令：

[root@k8s-nfs ~]# python
Python 3.6.5 (default, Jan 15 2021, 17:46:27) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

运行例子

只要显示python的版本是3.6.5，这就表示安装成功了。

安装 elastalert

执行下面的代码安装 elastalert ：

cd /root
yum install -y git
git clone https://github.com/Yelp/elastalert.git
cd elastalert
python setup.py install
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple/

运行例子

如果你觉得上面的代码，太长，也可以执行一键安装脚本，复制下面的脚本到你的命令行中：

wget http://66-ai.com/download/script-litte-prince/app/elastalert-install.sh -O /root/elastalert-install.sh && sh /root/elastalert-install.sh

运行例子

安装完成后,-h 显示帮助，即代表成功:

/usr/local/python/bin/elastalert -h

运行例子

结果显示帮助信息：

/usr/local/python/bin/elastalert -h
usage: elastalert [-h] [--config CONFIG] [--debug] [--rule RULE]
                  [--silence SILENCE] [--start START] [--end END] [--verbose]
                  [--patience TIMEOUT] [--pin_rules] [--es_debug]
                  [--es_debug_trace ES_DEBUG_TRACE]

optional arguments:
  -h, --help            show this help message and exit
  --config CONFIG       Global config file (default: config.yaml)
  --debug               Suppresses alerts and prints information instead. Not
                        compatible with `--verbose`
  --rule RULE           Run only a specific rule (by filename, must still be
                        in rules folder)
  --silence SILENCE     Silence rule for a time period. Must be used with
                        --rule. Usage: --silence <units>=<number>, eg.
                        --silence hours=2
  --start START         YYYY-MM-DDTHH:MM:SS Start querying from this
                        timestamp. Use "NOW" to start from current time.
                        (Default: present)
  --end END             YYYY-MM-DDTHH:MM:SS Query to this timestamp. (Default:
                        present)
  --verbose             Increase verbosity without suppressing alerts. Not
                        compatible with `--debug`
  --patience TIMEOUT    Maximum time to wait for ElasticSearch to become
                        responsive. Usage: --patience <units>=<number>. e.g.
                        --patience minutes=5
  --pin_rules           Stop ElastAlert from monitoring config file changes
  --es_debug            Enable verbose logging from Elasticsearch queries
  --es_debug_trace ES_DEBUG_TRACE
                        Enable logging from Elasticsearch queries as curl
                        command. Queries will be logged to file. Note that
                        this will incorrectly display localhost:9200 as the
                        host/port

运行例子

有可能jira版本太老，可以执行下面的代码更新：

pip install jira>=2.0.0

运行例子

要替换一下，原版本太老旧, 这样就安装成功了，走，吃烧烤去。

配置第一个告警

首先 copy 一个配置

cp config.yaml.example config.yaml    // 根据模板生成配置文件
vim config.yaml   // 修改配置

运行例子

先不慌执行，我们查看一下 config.yaml 文件, 把整个配置文件都列出来，不会多费纸，最多多费点电，所以，我打算把整个 config.yaml 文件都列出来：

# 规则文件所在的目录，规则文件后缀为yaml
rules_folder: example_rules

# 用来定时向elasticsearch发送请求，这里设置的是每分钟发送一次。
# 可以设置weeks、days、hours、minutes、seconds
run_every:
  minutes: 1

# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  minutes: 15

# elasticsearch的host地址，注意，每一个规则都有自己的Elasticsearch 主机
es_host: localhost

# elasticsearch 对应的端口号，9200 是默认端口
es_port: 9200

# The AWS region to use. Set this when using AWS-managed elasticsearch
#aws_region: us-east-1

# The AWS profile to use. Use this if you are using an aws-cli profile.
# See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
# for details
#profile: test

# 可选的es url前缀
#es_url_prefix: elasticsearch

# 可选的，选择是否用SSL连接es，true或者false，一般都为false，没有必要用安全连接
#use_ssl: True

# 可选的，是否验证TLS证书，设置为true或者false，默认为- true
#verify_certs: True

# GET request with body is the default option for Elasticsearch.
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport
# for details
#es_send_get_body_as: GET

# es的username和password，一般没有为es设置用户名和密码
#es_username: someusername
#es_password: somepassword

# Use SSL authentication with client certificates client_cert must be
# a pem file containing both cert and key for client
#verify_certs: True
#ca_certs: /path/to/cacert.pem
#client_cert: /path/to/client_cert.pem
#client_key: /path/to/client_key.key

# elastalert产生的日志在elasticsearch中的创建的索引
writeback_index: elastalert_status
writeback_alias: elastalert_alerts

# If an alert fails for some reason, ElastAlert will retry
# sending the alert until this time period has elapsed
# 如果告警失败，那么在指定的时间内会重新发送报警
alert_time_limit:
  days: 2

# Custom logging configuration
# If you want to setup your own logging configuration to log into
# files as well or to Logstash and/or modify log levels, use
# the configuration below and adjust to your needs.
# Note: if you run ElastAlert with --verbose/--debug, the log level of
# the "elastalert" logger is changed to INFO, if not already INFO/DEBUG.
#logging:
#  version: 1
#  incremental: false
#  disable_existing_loggers: false
#  formatters:
#    logline:
#      format: '%(asctime)s %(levelname)+8s %(name)+20s %(message)s'
#
#    handlers:
#      console:
#        class: logging.StreamHandler
#        formatter: logline
#        level: DEBUG
#        stream: ext://sys.stderr
#
#      file:
#        class : logging.FileHandler
#        formatter: logline
#        level: DEBUG
#        filename: elastalert.log
#
#    loggers:
#      elastalert:
#        level: WARN
#        handlers: []
#        propagate: true
#
#      elasticsearch:
#        level: WARN
#        handlers: []
#        propagate: true
#
#      elasticsearch.trace:
#        level: WARN
#        handlers: []
#        propagate: true
#
#      '':  # root logger
#        level: WARN
#          handlers:
#            - console
#            - file
#        propagate: false

运行例子

完整的配置列表，请参考 https://elastalert.readthedocs.io/en/latest/ruletypes.html#rule-configuration-cheat-sheet

第一个告警规则的例子，频率告警

ElastAlert 有 11 种告警，我们首先介绍一种 frequency 告警，就是在一定的时间内，某个事情发生了X次，就会报警。这个告警非常有用，一般用于:

如1分钟了某个用户登录了10次系统，可能会告警属于机器人在破解网站。
1分钟内有1000人注册，那么可能是机器人在注册网站
1分钟内有100个接口调用错误，那么可能是系统某个地方出现了错误

# Alert when the rate of events exceeds a threshold

# (Optional)
# Elasticsearch host
es_host: localhost

# (Optional)
# Elasticsearch port
es_port: 9200

# (OptionaL) Connect with SSL to Elasticsearch
#use_ssl: True

# (Optional) basic-auth username and password for Elasticsearch
#es_username: someusername
#es_password: somepassword

# (Required)
# Rule name, must be unique
name: Example frequency rule

# (Required)
# Type of alert.
# the frequency rule type alerts when num_events events occur with timeframe time
type: frequency

# (Required)
# Index to search, wildcard supported
index: filebeat-7.1.1*

# (Required, frequency specific)
# Alert when this many documents matching the query occur within a timeframe
num_events: 5

# (Required, frequency specific)
# num_events must occur within this amount of time to trigger an alert
timeframe:
  hours: 1

# (Required)
# A list of Elasticsearch filters used for find events
# These filters are joined with AND and nested in a filtered query
# For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html
filter:
- term:
    message: "aaa"

# (Required)
# The alert is use when a match is found
alert:
- "email"

# (required, email specific)
# a list of email addresses to send alerts to
email:
- "hewebgl3@foxmail.com"

运行例子

我们对这个配置文件做一些详细解释：

ElastAlert教程11章：开始安装elastalert

安装elastalert依赖的python3

安装 elastalert

配置第一个告警

第一个告警规则的例子，频率告警

精彩评论 0