How to show a label of the last log from Google Cloud Platform's "Logs-Based Metrics" instead as a value (!) in Grafana? How a second last one?

Question

I am passing a csv from one server to the other and add the number of rows to the logs by logging a jsonPayload with gcloud_logger.log_struct(....

I created Google Cloud Platform Logs-Based Metrics from this and added all variables of the jsonPayload as labels so that they would be available when choosing the GCP source in Grafana, see jsonPayload (structured logging) output from Python Google Cloud Function's logging needed to create Logs-Based Metrics (LBM) in GCP.

In Grafana, I can choose labels only in a time series panel.

This makes a nice time series chart.

Now that I use my other logs from a cloud function, the menu above has other values and does not need filters. By the way, I can choose the user metric name and the needed label only after having run the cloud function with that LBM setup - before, you will not find any labels, and it can only show the log results after the time that the LBM was made.

I cannot just get just the most recent log entry as one number. You can filter and group by, but that is all linked to the data over time. How can I just get the very last log label of my rows_count label so that I can show it in a one-dimensional panel?

In this kind of "main git repo" of GCP logging monitoring panel examples, all of the screenshots seem to show only time-series, see monitoring-dashboard-samples.

Example from the repo:

If I try to get just one number in a box of the following panel instead:

nothing is shown (that means in the example, a fourth box would not appear in addition), likely because time series cannot be shown in it (found out afterwards that in addition to that problem, the time series line's values in the graph cannot just take your chosen input values, but show some traffic information from Google instead, not just as a default, but likely rather as a fixed value). And How would I get the second last rows_count?

Some news:

These time series charts of Google Cloud Platform Logs only show a given measure of the labels on the y-axis, something like the traffic on GCP. I guess that it is not about choosing "Counter" or "Distribution" when making the LBM.

The value seems to be fixed, and all of the many units

are just measures of the GCP system it seems (and changing the unit is also just a naming, I try to change the measure itself that is shown on the y-axis.

The y-axis cannot just show the values of the labels over time, it seems! When I add a metric in the Alias by field like {{metric.label.rows_count}}, you only see its values in the legend (something like: rows_count 700 with a separate colour for that single value, but it does not replace the y-axis (which would be my aim). Probably the reason why I cannot ask for a label value: the values that I want are not listed in a long format, but in row format, and the table is unpivoted on the measure that is given by the LBM instead.

I guess that this is a question linked to the wide format of the logs and the fixed y-axis measure when using LBM as the source: you cannot even show the values in a time series chart, that is why you also cannot pick the last value of that time series :).

I could get the labels into the values using grouping in the query:

"Transform" tab --> "Labels to fields":

And then I somehow added the rows_count as "Units" to get the number into the values of a Gauge chart but I forgot how this was done and it is also not important since that would lead to linking the not-needed value "1" to the needed value, it was for example "1 740324".

You can get the count of the logs, though, whatever label you choose, it shows 3, which is right for the 3 runs of the cloud function that have been done after the new metric was there:

It seems as if it was not possible to "switch axes" = switching values with labels, although there are the functions:

• Labels to fields
• Rows to fields
• Series to rows

That is why I opened another question to find out whether I can change the y-axis at all: How to transform a Label-Based Metric and the Google Cloud timestamp into a time series chart in Grafana

Repeating the question after this long text:

How to get the value (one!) of a last log result's integer label from Google Cloud Platform's "Logs-Based Metrics" in Grafana? How a second last one?

PS:

And if that is not possible: is this a joke? You really cannot just get the last csv's rows_count of a log as a value into a Grafana panel using GCP LBMs? I must be mistaken, but I do not find the mistake.

Answer 1

Instead of choosing Counter which

collects the number of log entries matching a given filter

you need to choose Distribution

which

collects numeric data from log entries matching a given filter:

With that LBM, you get the LBM values as values instead of labels. And then, you will be able to choose the most recent one with a max on the datetime key, or the second last one with a top x on the datetime key in the filter and with the value column as the output.

I could only get the values in the chart when I also added the regex, it seems as if units are added automatically which make the numerical values unreadable without regex. The official GCP docs Creating a distribution metric tells you to use ([0-9.]+). I guess it is better to use ([0-9]+\.[0-9]+) than ([0-9.]+).