简介

blackbox_exporter 是 Prometheus 官方提供的一个 exporter，可以监控 HTTP、 HTTPS,、DNS、 TCP 、ICMP 等目标实例，从而实现对被监控节点进行监控 和数据采集。promethes调用blackbox_exporter去访问目标监控服务器，实现指标的采集

  • HTTP/HTPPS：URL/API 可用性检测TCP：端口监听检测
  • ICMP：主机存活检测
  • DNS：域名解析

地址：https://github.com/prometheus/blackbox_exporter

安装 / 使用

使用kubernetes - deployment部署

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
apiVersion: v1
kind: ConfigMap
metadata:
  name: blackbox-exporter-config
  namespace: cattle-monitoring-system
data:
  blackbox.yml: |
    modules:
      http_2xx:
        prober: http
        timeout: 5s
        http:
          valid_http_versions: ["HTTP/1.1", "HTTP/2"]
          valid_status_codes: [200]  # Defaults to 2xx
          method: GET
          preferred_ip_protocol: "ip4"
          fail_if_body_not_matches_regexp:
            - "errorCount"
      icmp:
        prober: icmp
        timeout: 5s
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: blackbox-exporter
  namespace: cattle-monitoring-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: blackbox-exporter
  template:
    metadata:
      labels:
        app: blackbox-exporter
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: kubernetes.io/hostname
                    operator: In
                    values:
                      - k3s-prod-node4
      containers:
        - name: blackbox-exporter
          image: prom/blackbox-exporter:latest
          args:
            - '--config.file=/etc/blackbox_exporter/blackbox.yml'
          ports:
            - containerPort: 9115
          volumeMounts:
            - name: config
              mountPath: /etc/blackbox_exporter
      volumes:
        - name: config
          configMap:
            name: blackbox-exporter-config
      tolerations:
        - effect: NoSchedule
          key: deploy
          operator: Equal
          value: node4
---
apiVersion: v1
kind: Service
metadata:
  name: blackbox-exporter
  namespace: cattle-monitoring-system
  labels:
    app: blackbox-exporter
spec:
  selector:
    app: blackbox-exporter
  ports:
    - name: http
      port: 9115
      targetPort: 9115

配置

blackbox的配置文件中只是制定了监控的类型模块，真正监控具体的数据实在prometheus-server上配置

详细yaml：https://github.com/prometheus/blackbox_exporter/blob/master/example.yml

验证监控数据

1
curl http://localhost:9115/metrics

kube-prom operator监听

ServiceMonitor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: blackbox-http-monitor
  namespace: cattle-monitoring-system
  labels:
    app: blackbox-http-monitor
spec:
  selector:
    matchLabels:            # Service选择器
      app: blackbox-exporter
  namespaceSelector:        # Namespace选择器
    matchNames:
      - cattle-monitoring-system
  jobLabel: blackbox-http
  endpoints:
    - interval: 30s
      scrapeTimeout: 10s
      path: /probe
      port: http
      params:
        module: [http_2xx]
        # 被测试url
        target: ["http://xxx:9876/xxx"]
      scheme: http
      metricRelabelings:
        - sourceLabels: [__param_target]
          targetLabel: instance
        - sourceLabels: [__address__]
          targetLabel: __tmp
        - sourceLabels: [__tmp]
          regex: .*
          replacement: blackbox-http
          targetLabel: job
      relabelings:
        - sourceLabels: []
          targetLabel: __param_target
        	# 被测试url
          replacement: http://xxx:9876/xxx
        - targetLabel: __address__
          replacement: blackbox-exporter:9115
        # 设置展示的label
        # 把 target 参数作为指标中的一个 label 暴露出来
        - sourceLabels: [__param_target]
          targetLabel: target
        # 设置 instance label（展示用）
        - sourceLabels: [__param_target]
          targetLabel: instance
        # 可选：设置 job 名
        - sourceLabels: [__address__]
          targetLabel: job
          replacement: blackbox-http
        # 💡 添加类型字段 label
        - targetLabel: type
          replacement: http
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: blackbox-icmp-monitor
  namespace: cattle-monitoring-system
spec:
  selector:
    matchLabels:            # Service选择器
      app: blackbox-exporter
  namespaceSelector:        # Namespace选择器
    matchNames:
      - cattle-monitoring-system
  jobLabel: blackbox-icmp
  endpoints:
    - interval: 30s
      scrapeTimeout: 10s
      path: /probe
      port: http
      params:
        module: [icmp]
        # 被测试ip
        target: ["xxx"]
      scheme: http
      metricRelabelings:
        - sourceLabels: [__param_target]
          targetLabel: instance
        - sourceLabels: [__address__]
          targetLabel: __tmp
        - sourceLabels: [__tmp]
          regex: .*
          replacement: blackbox-icmp
          targetLabel: job
      relabelings:
        - sourceLabels: []
          targetLabel: __param_target
          replacement: xxx
        - targetLabel: __address__
          replacement: blackbox-exporter:9115
        - sourceLabels: [__param_target]
          targetLabel: target
        - sourceLabels: [__param_target]
          targetLabel: instance
        - sourceLabels: [__address__]
          targetLabel: job
          replacement: blackbox-icmp
        # 添加类型字段 label
        - targetLabel: type
          replacement: icmp

grafana dashboard id【1860】