技术:数据采集,是一个类似flume的文件采集工具
使用input作为输入采集数据,可以是文件,可以是http接口,详见:https://github.com/influxdata/telegraf#input-plugins
使用output作为输出,可以是kafka,可以是时序数据库,可以到es等
运行环境:linux + telegraf
demo功能:提供一个telegraf文件数据上报到kafka的配置demo
github源码:https://github.com/influxdata/telegraf,主要使用和功能详见说明
## 安装rpm包
rpm -iv telegraf-1.7.4-1.x86_64.rpm
/usr/bin/telegraf -pidfile /var/run/telegraf/telegraf.pid -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d
举例配置搜集rocketmq日志文件, 输出到kafka。可以再消费kafka将数据写入时序数据库就可以展示。
[[inputs.logparser]]
files = ["/root/logs/rocketmqlogs/store.log"]
from_beginning = false
[inputs.logparser.tags]
cluster='xxmq-ali-cluster-19'
[inputs.logparser.grok]
patterns = ["%{GREEDYDATA:log}"]
timezone = "Local"
measurement = "xxmq-store"
[[inputs.logparser]]
files = ["/root/logs/rocketmqlogs/stats.log"]
from_beginning = false
[inputs.logparser.tags]
cluster='xxmq-ali-cluster-19'
[inputs.logparser.grok]
patterns = ["%{GREEDYDATA:log}"]
timezone = "Local"
measurement = "xxmq-stats"
# [[outputs.file]]
# files = ["/tmp/metrics-store.out"]
# data_format = "json"
# namepass = ["xxmq-store"]
# [[outputs.file]]
# files = ["/tmp/metrics-stats.out"]
# data_format = "json"
# namepass = ["xxmq-stats"]
[[outputs.kafka]]
brokers = ["127.0.0.1:9092","127.0.0.2:9092","127.0.0.3:9092","127.0.0.4:9092","127.0.0.5:9092"]
topic = "xxmq-broker-stats-log"
data_format = "json"
namepass = ["xxmq-stats"]
version = "1.1.0"
[[outputs.kafka]]
brokers = ["127.0.0.1:9092","127.0.0.2:9092","127.0.0.3:9092","127.0.0.4:9092","127.0.0.5:9092"]
topic = "xxmq-broker-store-log"
data_format = "json"
namepass = ["xxmq-store"]
version = "1.1.0"
支持的output
欢迎添加微信,互相学习↑↑↑ -_-
白老虎
programming is not only to solve problems, ways to think
grafana 级连 菜单 templating (variables) 配置
rocketmq 集群搭建 (2master + 2slave + 2namesrv)
AI 机器人 抓取 微信 聊天中的 百度网盘 分享地址和密码