how to visualize prometheus endpoint metrics using grafana

prometheusnoob Source

I have a simple Spring-Boot application (with Maven), which I've instrumented using Prometheus to collect application-specific metrics. The instrumented Prometheus metrics are exposed on http://localhost:8080/prometheus by default, but I'm not sure how to visualize these metrics on Grafana. When I try to set the datasource in Grafana, I get an error:

HTTP Error Not Found

Grafana error when adding a new data source

Below is my working code, which exposes metrics on http:localhost:8080/prometheus

Example.java: Simply exposes a few endpoints. Instrumented so that Prometheus counts the number of times the /homepage endpoint is hit.

@SpringBootApplication
@RestController
public class Example {

    //Just a logger that keeps track of relevant information:
    private static final Logger LOGGER = Logger.getLogger(Example.class.getName());

    //counter for counting how many times an endpoint has been hit
    static final Counter myCounter = Counter.build()    
                                              .name("CounterName") //note: by convention, counters should have "_total" suffix
                                              .help("Total requests recorded by a specific endpoint")
                                              .labelNames("status")
                                              .register();

    //using a guage to set the time that a dummy process started, store the time it ends, and then calculate
    //the elapsed time between when the process started and when the process ended.
    static final Gauge myGauge =  Gauge.build()
                                .name("GaugeName")
                                .help("This is the Gauge help message")
                                .labelNames("label1")
                                .register();            

    @RequestMapping("/hello")
    String hello() {

        myCounter.labels("customLabel1").inc(); //increment the number of requests by one
        myGauge.labels("customLabel1").inc(3);
        LOGGER.log(Level.INFO, "Number of times /hello has been hit: " + myCounter.labels("customLabel1").get());
        LOGGER.log(Level.INFO, "Size of our Gauge: " + myGauge.labels("customLabel1").get());
        myGauge.labels("customLabel1").dec();
        LOGGER.log(Level.INFO, "size of out Gauge after decrementing by 1: " + myGauge.labels("customLabel1").get());

        return "Hello world! This is an example response!";
    }

    @RequestMapping("/homepage")
    String homePage() {

        myCounter.labels("customLabel2").inc(); //increment the number of requests by one
        LOGGER.log(Level.INFO, "Number of times /homepage has been hit: " + myCounter.labels("customLabel2").get());

        return "this is the home page!!";
    }


    public static void main(String[] args) throws Exception {
        SpringApplication.run(Example.class, args);
    }

}

MonitoringConfig.java: This is how Prometheus collects the application's metric information, and exposes them on localhostL8080/prometheus

@Configuration
public class MonitoringConfig {
    @Bean
    SpringBootMetricsCollector springBootMetricsCollector(Collection<PublicMetrics> publicMetrics){
        SpringBootMetricsCollector springBootMetricsCollector = new SpringBootMetricsCollector(publicMetrics);
        springBootMetricsCollector.register();

        return springBootMetricsCollector;
    }

    @Bean
    ServletRegistrationBean servletRegistrationBean() {
        DefaultExports.initialize();
        return new ServletRegistrationBean(new MetricsServlet(), "/prometheus");
    }
}

pom.xml: Project's Maven dependencies

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>springboot</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>1.5.8.RELEASE</version>
    </parent>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <!-- Prometheus dependencies -->
        <!-- The client -->
        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient</artifactId>
            <version>0.1.0</version>
        </dependency>

        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient_spring_boot</artifactId>
            <version>0.1.0</version>
        </dependency>

        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient_servlet</artifactId>
            <version>0.1.0</version>
        </dependency>
        <!-- Hotspot JVM metrics -->
        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient_hotspot</artifactId>
            <version>0.1.0</version>
        </dependency>
        <!-- Exposition HTTPServer -->
        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient_httpserver</artifactId>
            <version>0.1.0</version>
        </dependency>
        <!-- Pushgateway exposition -->
        <dependency>
            <groupId>io.prometheus</groupId>
            <artifactId>simpleclient_pushgateway</artifactId>
            <version>0.1.0</version>
        </dependency>

        <!-- Spring Boot Actuator for exposing metrics -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
            <version>1.5.8.RELEASE</version>
        </dependency>

    </dependencies>


</project>
spring-bootmetricsgrafanaprometheusgrafana-templating

Answers

answered 10 months ago Marc #1

You need to point grafana at a prometheus server, not your application.

See the prometheus docs for more details on how to run prometheus and how to visualize prometheus metrics in grafana dashboards.

answered 3 months ago Sania #2

Grafana is a visualization tool. It does not help in monitoring the metrics. It helps in creating visualizations/graphs for the metrics that are already being monitored.

You are right about the adding Data Source at Grafana WebUI but the data source should point to your Prometheus URL.

Here are the steps to map metrics to Grafana using prometheus.

  1. After installation of Prometheus, change the prometheus.yml to include your application endpoint. Prometheus scrapes metrics for monitoring from this endpoint

 scrape_configs
 - job_name: 'my_application'
      metrics_path: '/prometheus'
      static_configs:
      - targets: ['localhost:8080']

  1. Now prometheus monitors all the metrics under this endpoint. You can navigate to http://localhost:9090/ to observe the metrics.

  2. Mention this URL in your Grafana datasource underDataSources->New

    Mention URL: http://localhost:9090 and Access: Browser and name this datasource as Prometheus

comments powered by Disqus