Paid project · Grafana

Grafana monitoring: let the problem ring first

Prometheus, Loki and Grafana collection, alerting and retention; what gets delivered, and why to start with few accurate alerts.

When you need it

With several services, a full disk, an expiring certificate or a restart loop is discovered when something stops working. Monitoring is not about drawing graphs; it is about hearing the problem before you notice.

What gets delivered

  • Collection: node_exporter for the host, cAdvisor for containers, application metrics where they already exist.
  • Storage and retention: where Prometheus data lives, how long it is kept, and how much disk that needs.
  • Logs, optional: Loki for container logs, queried in the same Grafana.
  • Dashboards: host, containers, certificate expiry, backup results; the few you will actually read.
  • Alerts: rules, thresholds, notification channels (email or webhook), plus silencing and grouping so one issue does not ring all night.
  • A test: fill a disk or stop a service once, and confirm the alert actually fires.

Not included

  • Hosted monitoring. The data lives on your machine, and if the machine is down, so is the monitoring.
  • 24/7 on-call or picking up alerts for you.
  • Third-party notification availability.
  • Instrumenting application code; existing metrics are wired up.

Who it fits

Several services, or a previous outage that was only noticed the next morning. One service that restarts now and then does not need this yet.

How to start

Email how many machines, which services, whether Prometheus already exists, and where alerts should go. Scope first, then a quote.

Questions

Is it heavy?

Prometheus and Grafana are light; metric cardinality and retention are what cost. Collection scope and retention are sized to your machine.

Too many alerts?

Start with four: disk, certificates, service down, backup failed. Tune thresholds after a week. More rules is not better.

Related

Email is the only channel. Questions, corrections or a scope request: [email protected]. This page also exists as Markdown: /services/grafana.md.

Not sure where to start?

Run the assessment first, or send your setup and goal by email. The reply says whether it fits and what happens next.