Spring actuator with Prometheus, custom MetricWriter is never called

I. Domshchikov Source

I'd like to add metric measurement for my Spring boot app. Metrics would be displayed via grafana.

I have found nice article for integration spring actuator with prometheus.

According to the sample, I should implement my custom MetricWriter for updating corresponding Counter or Gauge in the Prometheus CollectorRegistry It looks like this:

class PrometheusMetricWriter implements MetricWriter {

    CollectorRegistry registry

    private final ConcurrentMap<String, Gauge> counters = new ConcurrentHashMap<>()
    private final ConcurrentHashMap<String, Gauge> gauges = new ConcurrentHashMap<>()

    @Autowired
    PrometheusMetricWriter(CollectorRegistry registry) {
        this.registry = registry
    }

    @Override
    void increment(Delta<?> delta) {
        counter(delta.name).inc(delta.value.doubleValue())
    }

    @Override
    void reset(String metricName) {
        counter(metricName).clear()
    }

    @Override
    void set(Metric<?> value) {
        println "inside metric writer"
        gauge(value.name).set(value.value.doubleValue())
    }

    private Counter counter(String name) {
        def key = sanitizeName(name)
        counters.computeIfAbsent key, { k ->
            Counter.build().name(k).help(k).register(registry)
        }
    }

    private Gauge gauge(String name) {
        def key = sanitizeName(name)
        gauges.computeIfAbsent key, { k ->
            Gauge.build().name(k).help(k).register(registry)
        }
    }

    private String sanitizeName(String name) {
        return name.replaceAll("[^a-zA-Z0-9_]", "_")
    }
}

The other 3 required elements are: PrometheusEndpoint, PrometheusMvcEndpoint, PrometheusEndpointContextConfiguration.

After all, the collected metrics could be reviewed by calling 'http://localhost:8080/prometheus' url.

And the result should look like this:

# HELP gauge_com_egalacoral_spark_stage_vto gauge_com_egalacoral_spark_stage_vto
# TYPE gauge_com_egalacoral_spark_stage_vto gauge
gauge_com_egalacoral_spark_stage_vto 4.0
# HELP gauge_com_egalacoral_spark_test_vfl gauge_com_egalacoral_spark_test_vfl
# TYPE gauge_com_egalacoral_spark_test_vfl gauge
gauge_com_egalacoral_spark_test_vfl 16.0

But, I am receiving an empty page. Here is test class for collecting some metrics.

class Runner {

    @Autowired
    GaugeService gaugeService

    private static final Logger logger = LoggerFactory.getLogger(Runner.class)

    private ZoneId zoneId = ZoneId.of('Europe/London')

    private Integer delay = 10

    @Scheduled(fixedDelay = 10000L)
    void runOnTest() throws Exception {

        def now = ZonedDateTime.now(zoneId)
        println "collect metrics"
        gaugeService.submit("test", 2.0)

    }
}

P.S: I have noticed that GaugeService has several implementation, and in my App this is DropwizardMetricServices. This implementation already contains it's own metric registry, which collects my metrics. So, when I am calling '/prometheus' endpoint, I am getting an empty page, as the CollectorRegistry' in my customPrometheusMetricWriter` is never populated with my metrics.

So please, tell me how could I solve this issue?

spring-boot-actuatorprometheus

Answers

answered 1 year ago I. Domshchikov #1

So, I found the reason why DropwizardMetricServices implementation of GaugeService is using.

According to the spring docs:

50.10 Dropwizard Metrics

A default MetricRegistry Spring bean will be created when you declare a dependency to the io.dropwizard.metrics:metrics-core library; you can also register you own @Bean instance if you need customizations. Users of the Dropwizard ‘Metrics’ library will find that Spring Boot metrics are automatically published to com.codahale.metrics.MetricRegistry. Metrics from the MetricRegistry are also automatically exposed via the /metrics endpoint

When Dropwizard metrics are in use, the default CounterService and GaugeService are replaced with a DropwizardMetricServices, which is a wrapper around the MetricRegistry (so you can @Autowired one of those services and use it as normal). You can also create “special” Dropwizard metrics by prefixing your metric names with the appropriate type (i.e. timer., histogram. for gauges, and meter.* for counters).

Somehow io.dropwizard.metrics:metrics-core library was included to the project dependencies. After, I removing it, all start to work as expected.

comments powered by Disqus