prometheus 监控pod 提示context deadline exceeded错误

Question

我用二进制跑的k8s和prometheus，然后在prometheu.yml配置。

  - job_name: 'kubernetes-service-endpoints'
    scheme: https
    kubernetes_sd_configs:
      - role: endpoints
        api_server: https://192.168.3.17:6443/
        tls_config:
          insecure_skip_verify: true
        bearer_token: "**********************"
    relabel_configs:
    - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
      action: keep
      regex: true
    - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
      action: replace
      target_label: __scheme__
      regex: (https?)
    - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
      action: replace
      target_label: __metrics_path__
      regex: (.+)
    - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
      action: replace
      target_label: __address__
      regex: ([^:]+)(?::d+)?;(d+)
      replacement: $1:$2
    - action: labelmap
      regex: __meta_kubernetes_service_label_(.+)
    - source_labels: [__meta_kubernetes_namespace]
      action: replace
      target_label: kubernetes_namespace
    - source_labels: [__meta_kubernetes_service_name]
      action: replace
      target_label: kubernetes_name
    - source_labels: [__meta_kubernetes_pod_node_name]
      action: replace
      target_label: kubernetes_node

pod的yaml文件,这是一个简单的nginx服务：

apiVersion: v1
kind: Service
metadata:
  name: rrjc-algorithm
  namespace: default
  annotations:                      # 在metadata下增加annotations
    prometheus.io/scrape: 'true'    # 开启K8s动态监测后端pod数据
##    prometheus.io/path: '/metrics'  #pod暴露的/metrics接口地址
    prometheus.io/port: '9163'      #pod暴露的采集数据端口
spec:
  selector:
    app: rrjc-algorithm
    release: canary
  ports:
  - name: http
    port: 8081
    targetPort: 80
    nodePort: 30001
  type: NodePort

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: algorithm-deploy
spec:
  replicas: 3
  selector:
    matchLabels:
      app: rrjc-algorithm
      release: canary
  template:
    metadata:
      labels:
        app: rrjc-algorithm
        release: canary
    spec:
      containers:
      - name: rrjc-algorithm
        image: ****.****.com/rrjc/httpd
        ports:
        - name: httpd
          containerPort: 80
      imagePullSecrets:        #使用的secret
      - name: registry-secret

pod的服务能通过nodeport正常访问，但是prometheus监控中提示Get "https://172.20.107.216:9163/metrics": context deadline exceeded错误，求大佬指教。

Answer 1

等待大神解答