Prometheus Pushgateway Installation

Internal

• Prometheus Pushgateway

Kubernetes Installation

Deploy the pod and associated services, and then configure it as a scraping target by deploying a ServiceMonitor. All descriptors are available in:

playground/prometheus/pushgateway

Pod and Services

namespace=prom
cat <<EOF | kubectl -n ${namespace} apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: 'pushgateway'
  labels:
    function: 'pushgateway'
spec:
  containers:
  - name: 'pushgateway'
    image: prom/pushgateway
---
apiVersion: v1
kind: Service
metadata:
  name: 'pushgateway'
  labels:
    function: 'pushgateway-target'
spec:
  type: 'ClusterIP'
  selector:
    function: 'pushgateway'
  ports:
    # 'name' is important, it will be referred by ServiceMonitor configuration
    - name: 'pushgateway'
      protocol: 'TCP'
      port: 9091
      targetPort: 9091
---
apiVersion: v1
kind: Service
metadata:
  name: 'pushgateway-lb'
spec:
  type: 'LoadBalancer'
  selector:
    function: 'pushgateway'
  ports:
    - name: 'pushgateway'
      protocol: 'TCP'
      port: 9091
      targetPort: 9091
EOF

Test the deployment by accessing the console:

http://localhost:9091

ServiceMonitor

Expose it then as a Prometheus scraping target with a ServiceMonitor:

namespace=prom
cat <<EOF | kubectl apply -n ${namespace} -f -
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: pushgateway
  namespace: ${namespace}
  labels:
    release: prometheus
spec:
  endpoints:
  - port: 'pushgateway' # must match the label from Service declaration
    path: /metrics
    interval: 5s
  namespaceSelector:
    matchNames:
    - ${namespace}
  selector:
    matchLabels:
      function: 'pushgateway-target'
EOF

Ensure that the pushgateway is correctly installed as target by inspecting Prometheus configuration at console (Status → Configuration). Also the console has a "Targets" section where the target should show up:

Also see:

Configuring Prometheus with Prometheus Operator | Add a Monitoring Target

Test

Publish a synthetic metric to pushgateway with a script similar to publish-metric and ensure the metric can be found in the pushgateway console (http://localhost:9091) and with a Prometheus query.