1. Filebeat（監控日志文件）

使用yum源安裝，yum源配置和elasticsearch一樣參考 Yum安裝方式

yum list |grep filebeat #查看安裝包
yum install filebeat -y #安裝
rpm -ql  filebeat #查看安裝目錄

通過yum list|grep elasticsearch可以查看到如下圖的所有那種包

查看安裝包

部署位置

Filebeat的配置文件有很多，重點關注inputs和output即可，其他可以根據需要設置。
vim /etc/filebeat/fields.yml

#=========================== Filebeat inputs =============================

filebeat.inputs: #輸入指定需要監控的文件
- type: log  #指定系統日志文件，可以根據需要在paths中添加
 paths:
   - /var/log/*.log
   - /var/log/secure
   - /var/log/messages
 tags: ["system"] #標簽，用于分類
- type: log #指定nginx日志
 paths:
   - /home/app/tengine/logs/access.log
   #- c:\programdata\elasticsearch\logs\*
 tags: ["nginx"]
#============================= Filebeat modules ===============================
filebeat.config.modules: #filebeat對應日志的模版，具體可以查看modules.d目錄下，這里沒有使用因為目錄下注銷的，可以參考使用,當直接數據elasticsearch時，可以使用模版進行批評
 path: ${path.config}/modules.d/*.yml
 reload.enabled: false
#==================== Elasticsearch template setting ==========================
# 默認數據分片數
setup.template.settings:
 index.number_of_shards: 1
#================================ General  =====================================
#一般設置
#============================== Dashboards =====================================
#設置儀表板大屏，可以根據需要設置
# These settings control loading the sample dashboards to the Kibana index. Loading
# the dashboards is disabled by default and can be enabled either by setting the
# options here or by using the `setup` command.
#setup.dashboards.enabled: false
# The URL from where to download the dashboards archive. By default this URL
# has a value which is computed based on the Beat name and version. For released
# versions, this URL points to the dashboard archive on the artifacts.elastic.co
# website.
#setup.dashboards.url:
#============================== Kibana =====================================
#設置kibana展示
# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:
 # Kibana Host
 # Scheme and port can be left out and will be set to the default (http and 5601)
 # In case you specify and additional path, the scheme is required: http://localhost:5601/path
 # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
 #host: "localhost:5601"
 # Kibana Space ID
 # ID of the Kibana Space into which the dashboards should be loaded. By default,
 # the Default Space will be used.
 #space.id:
#============================= Elastic Cloud ==================================
#設置云elastic
# These settings simplify using filebeat with the Elastic Cloud (https://cloud.elastic.co/).
# The cloud.id setting overwrites the `output.elasticsearch.hosts` and
# `setup.kibana.host` options.
# You can find the `cloud.id` in the Elastic Cloud web UI.
#cloud.id:
# The cloud.auth setting overwrites the `output.elasticsearch.username` and
# `output.elasticsearch.password` settings. The format is `<user>:<pass>`.
#cloud.auth:
#================================ Outputs =====================================
#日志輸出：elasticsearch，logstash，kafka，redis都可以
# Configure what output to use when sending the data collected by the beat.
#output.console:  # 指定在終端上輸出日志信息
#  enable: true
#-------------------------- Elasticsearch output ------------------------------
#output.elasticsearch:
 # Array of hosts to connect to.
# hosts: ["172.16.15.53:9200","172.16.15.26:9200"]
 # Optional protocol and basic auth credentials.
 #protocol: "https"
 #username: "elastic"
 #password: "changeme"
#----------------------------- Logstash output --------------------------------
# output.logstash:
 # # The Logstash hosts
 # hosts: ["172.16.15.53:5044"]
 # Optional SSL. By default is off.
 # List of root certificates for HTTPS server verifications
 #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
 # Certificate for SSL client authentication
 #ssl.certificate: "/etc/pki/client/cert.pem"
 # Client Certificate Key
 #ssl.key: "/etc/pki/client/cert.key"
#----------------------------- kafka output --------------------------------
output.kafka:
 enabled: true
 hosts: ["172.16.15.48:9092"]
 topic: system
#================================ Processors =====================================
# Configure processors to enhance or manipulate events generated by the beat.
processors:
 - add_host_metadata: ~
 - add_cloud_metadata: ~

#================================ Logging =====================================

# Sets log level. The default log level is info.
# Available log levels are: error, warning, info, debug
#logging.level: debug
# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publish", "service".
#logging.selectors: ["*"]
#============================== Xpack Monitoring ===============================
#安全配置
# filebeat can export internal metrics to a central Elasticsearch monitoring
# cluster.  This requires xpack monitoring to be enabled in Elasticsearch.  The
# reporting is disabled by default.
# Set to true to enable the monitoring reporter.
#xpack.monitoring.enabled: false
# Uncomment to send the metrics to Elasticsearch. Most settings from the
# Elasticsearch output are accepted here as well. Any setting that is not set is
# automatically inherited from the Elasticsearch output configuration, so if you
# have the Elasticsearch output configured, you can simply uncomment the
# following line.
#xpack.monitoring.elasticsearch:

以上配置配件是發送到kafka中，logstash再去拉取。如果filebeat直接發送到elasticsearch集群中，可以考慮使用現成的模版和儀表盤展示。
下面可以展示支持模版的模塊

[root@EASYCMDB log]# filebeat modules list
Enabled:

Disabled:
apache
auditd
elasticsearch
haproxy
icinga
iis
iptables
kafka
kibana
logstash
mongodb
mysql
nginx
osquery
postgresql
redis
santa
suricata
system
traefik
zeek

其他模塊可以類似如下：

filebeat modules enable mysql
service filebeat restart
filebeat setup --dashboards

在/etc/moudules/文件夾下編輯mysql.yml

- module: mysql
  #metricsets:
  #  - status
  #  - galera_status
  period: 10s
  # Host DSN should be defined as "user:pass@tcp(127.0.0.1:3306)/"
  # The username and password can either be set in the DSN or using the username
  # and password config options. Those specified in the DSN take precedence.
  hosts: ["tcp(127.0.0.1:3306)/"] #表示使用tcp連接
  # Username of hosts. Empty by default.
  username: xxx #用戶名
  # Password of hosts. Empty by default.
  password: xxxxxxx #密碼

參考：https://www.elastic.co/guide/en/beats/filebeat/7.2/filebeat-modules.html 根據需要選擇對應的模版配置

2. Mysqlbeat(監控數據庫性能)

參考https://github.com/adibendahan/mysqlbeat
由于是第三方開發的效果不是太好

3. Metricbeat(監控服務器性能)

參考：https://www.elastic.co/guide/en/beats/metricbeat/7.1/metricbeat-getting-started.html
yum安裝
yum install -y metricbeat
配置文件metricbeat.yml

###################### Metricbeat Configuration Example #######################

# This file is an example configuration file highlighting only the most common
# options. The metricbeat.reference.yml file from the same directory contains all the
# supported options with more comments. You can use it as a reference.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/metricbeat/index.html

#==========================  Modules configuration ============================

metricbeat.config.modules: #可以在里面啟用支持的監控系統、Apache等
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: true  #改成true會直接導入

  # Period on which files under path should be checked for changes
  #reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression
  #_source.enabled: false

#================================ General =====================================

# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:

# The tags of the shipper are included in their own field with each
# transaction published.
#tags: ["service-X", "web-tier"]

# Optional fields that you can specify to add additional information to the
# output.
#fields:
#  env: staging


#============================== Dashboards =====================================
# These settings control loading the sample dashboards to the Kibana index. Loading
# the dashboards is disabled by default and can be enabled either by setting the
# options here or by using the `setup` command.
#setup.dashboards.enabled: false

# The URL from where to download the dashboards archive. By default this URL
# has a value which is computed based on the Beat name and version. For released
# versions, this URL points to the dashboard archive on the artifacts.elastic.co
# website.
#setup.dashboards.url:

#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana: #設置kibana 可以用來做dashboards導入

  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  host: "172.16.15.53:5601"
  username: "elastic"
  password: "RRCOfh9UudUIVZ4i954E"
  # Kibana Space ID
  # ID of the Kibana Space into which the dashboards should be loaded. By default,
  # the Default Space will be used.
  #space.id:

#============================= Elastic Cloud ==================================

# These settings simplify using metricbeat with the Elastic Cloud (https://cloud.elastic.co/).

# The cloud.id setting overwrites the `output.elasticsearch.hosts` and
# `setup.kibana.host` options.
# You can find the `cloud.id` in the Elastic Cloud web UI.
#cloud.id:

# The cloud.auth setting overwrites the `output.elasticsearch.username` and
# `output.elasticsearch.password` settings. The format is `<user>:<pass>`.
#cloud.auth:

#================================ Outputs =====================================

# Configure what output to use when sending the data collected by the beat.

#-------------------------- Elasticsearch output ------------------------------
# output.elasticsearch: #輸出到elasticsearch
  # # Array of hosts to connect to.
  # hosts: ["172.16.15.53:9200","172.16.15.48:9200"]

  # # # Optional protocol and basic auth credentials.
  # # #protocol: "https"
  # username: "elastic"
  # password: "RRCOfh9UudUIVZ4i954E"
output.kafka: #輸出到kafka
  # initial brokers for reading cluster metadata
  hosts: ["172.16.15.48:9092"]

  # message topic selection + partitioning
  topic: 'testTopic' #注意此處是保存的kafka主題 
  partition.round_robin:
    reachable_only: false

  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000
#----------------------------- Logstash output --------------------------------
#output.logstash:
  # The Logstash hosts
  #hosts: ["localhost:5044"]

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  # Certificate for SSL client authentication
  #ssl.certificate: "/etc/pki/client/cert.pem"

  # Client Certificate Key
  #ssl.key: "/etc/pki/client/cert.key"

#================================ Processors =====================================

# Configure processors to enhance or manipulate events generated by the beat.

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

#================================ Logging =====================================

# Sets log level. The default log level is info.
# Available log levels are: error, warning, info, debug
#logging.level: debug

# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publish", "service".
#logging.selectors: ["*"]

#============================== Xpack Monitoring ===============================
# metricbeat can export internal metrics to a central Elasticsearch monitoring
# cluster.  This requires xpack monitoring to be enabled in Elasticsearch.  The
# reporting is disabled by default.

# Set to true to enable the monitoring reporter.
#xpack.monitoring.enabled: false

# Uncomment to send the metrics to Elasticsearch. Most settings from the
# Elasticsearch output are accepted here as well. Any setting that is not set is
# automatically inherited from the Elasticsearch output configuration, so if you
# have the Elasticsearch output configured, you can simply uncomment the
# following line.
#xpack.monitoring.elasticsearch:

#================================= Migration ==================================

# This allows to enable 6.7 migration aliases
#migration.6_to_7.enabled: true

默認情況下只可以監控system

[root@EASYCMDB ~]# metricbeat modules list #使用此命令可以查看支持的
Enabled:
system

Disabled:
aerospike
apache
aws
ceph
couchbase
couchdb
docker
dropwizard
elasticsearch
envoyproxy
etcd
golang
graphite
haproxy
http
jolokia
kafka
kibana
kubernetes
kvm
logstash
memcached
mongodb
mssql
munin
mysql
nats
nginx
php_fpm
postgresql
prometheus
rabbitmq
redis
traefik
uwsgi
vsphere
windows
zookeeper

其他監控可以類似添加

metricbeat modules enable mysql #啟動接口mysql
service metricbeat restart #重啟
metricbeat setup –dashboards #加載監控大盤

在/etc/moudules/文件夾下編輯mysql.yml

- module: mysql
 #metricsets:
 # - status
 # - galera_status
 period: 10s
 # Host DSN should be defined as "user:pass@tcp(127.0.0.1:3306)/"
 # The username **and** password can either be set **in** the DSN **or** using the username
 # **and** password config options. Those specified **in** the DSN take precedence.
 hosts: ["tcp(127.0.0.1:3306)/"] #表示使用tcp連接
 # Username of hosts. Empty by default.
 username: root #用戶名
 # Password of hosts. Empty by default.

可以根據不同的模塊參考modules對應的官方文檔: https://www.elastic.co/guide/en/beats/metricbeat/7.2/metricbeat-modules.html

4. Packetbeat

參考：https://www.elastic.co/guide/en/beats/packetbeat/7.1/packetbeat-getting-started.html