For prometheus you can use an alert manager, it has a docker file in the link below:
- https://hub.docker.com/r/prom/alertmanager

Awesome Prometheus alerts:
- https://github.com/samber/awesome-prometheus-alerts

alertmanager has rules, you can see sample rules here in the following link:
- https://awesome-prometheus-alerts.grep.to/rules

So to add alertmanager service:

alertmanager:
    image: prom/alertmanager:latest
    restart: always
    command: --config.file=/etc/alertmanager/alertmanager.yml
    volumes:
      - ./alert/config/alertmanager.yml:/etc/alertmanager/alertmanager.yml
    dns:
      - 8.8.8.8

You alert manager configuration may look something like below:

global:
  resolve_timeout: 5m

route:
  # When a new group of alerts is created by an incoming alert, wait at
  # least 'group_wait' to send the initial notification.
  # This way ensures that you get multiple alerts for the same group that start
  # firing shortly after another are batched together on the first
  # notification.
  group_wait: 10s

  # When the first notification was sent, wait 'group_interval' to send a betch
  # of new alerts that started firing for that group.
  group_interval: 5m

  # If an alert has successfully been sent, wait 'repeat_interval' to
  # resend them.
  repeat_interval: 30m

  # A default receiver
  receiver: "slack"

  # All the above attributes are inherited by all child routes and can
  # overwritten on each.
  routes:
    - receiver: "slack"
      group_wait: 10s
      match_re:
        severity: error|warning
      continue: true

#    - receiver: "sms"
#      group_wait: 10s
#      match_re:
#        severity: error
#      continue: true

receivers:
  - name: "slack"
    slack_configs:
      - api_url: 'YOUR-WEBHOOK-URL'
        send_resolved: true
        channel: 'monitoring'
        text: "{{ range .Alerts }}<!channel> {{ .Annotations.summary }}\n{{ .Annotations.description }}\n{{ end }}"

#  - name: "sms"
#    webhook_config:
#      - url: http://a.b.c:8080/send/sms
#        send_resolved: true

You should be up & running with this sample configurations.
Spread your love for M2SH :)

#prometheus #prom #alert #alert_manager #docker #dockerfile #slack