How can I visualize a histogram with Promdash or Grafana?

Question

I\'m attracted to prometheus by the histogram (and summaries) time-series, but I\'ve been unsuccessful to display a histogram in either promdash or grafana. What I expect is to

Answer 1

Grafana v5+ provides direct support for representing Prometheus histograms as heatmap. http://docs.grafana.org/features/panels/heatmap/#histograms-and-buckets

Heatmaps are preferred over histogram because a histogram does not show you how the trend changes over time. So if you have a time-series histogram, then use the heatmap panel to picture it.

To get you started, here is an example (for Prometheus data):

Suppose you've a histogram as follows,

http_request_duration_seconds_bucket(le=0.2) 1,
http_request_duration_seconds_bucket(le=0.5) 2,
http_request_duration_seconds_bucket(le=1.0) 2,
http_request_duration_seconds_bucket(le=+inf) 5
http_request_duration_seconds_count 5
http_request_duration_seconds_sum 3.07

You can picture this histogram data as a heatmap by using the query: sum(increase(http_request_duration_seconds_bucket[10m])) by (le), making sure to set the format as "heatmap," the legend format as {{ le }}, and setting the visualization in the panel settings to "heatmap."

Answer 2

Recent releases of Grafana have a builtin Heatmap visualization type, but use it sparingly as it can be very computationally expensive.

One can achieve a histogram by selecting a Graph visualization then under Axes selecting Histogram for X-Axis>Mode.

Here is an example with Grafana 7.03. Data collected by ceph_exporter into Prometheus. The metric is ceph_osd_utilization. The query uses a templated variable but that's orthogonal to this question.

Answer 3

I don't believe that Grafana supports a barchart for a histogram.

You can do a standard non-stacked graph of the rate a histogram, and as Prometheus histograms are cumulative you'll get the result you're looking for.

Answer 4

The Answer from @brian-brazil above works almost, with some additional, not mentioned things, to be done.

You can do a standard non-stacked graph of the rate a histogram, and as Prometheus histograms are cumulative you'll get the result you're looking for.

1. The x-axis in Grafana needs to be in mode series
2. You need to aggregate the results by the le label, if you use additional distinct labels for the series

After that you'll get a beautiful histogram. The only thing that grinds my gears is that the x-axis sort order of grafana is natural string sorting. So the x axis starts with +Inf, then 0.1,0.2, .. 1,1.5,10,2,...

PS: In grafana 5.1 there will be working heatmap with prometheus datasource out-of-box. There was an issue for native support for heatmap visualization, which is also appropriate (if you want to see tendency/history) for visualizing histograms over time.