Performance Metrics in Microservices

Performance Metrics in Microservices

Performance Metrics in Microservices

1. What Are Performance Metrics?

Performance metrics are measurable values used to evaluate the speed, reliability, resource utilization, and scalability of microservices.

In a microservices architecture, an application consists of multiple independent services that communicate through HTTP APIs, messaging systems, or other mechanisms. A performance issue in one service can affect the entire application.

Performance metrics help answer questions such as:

• How quickly does a service respond to requests?
• How many requests can a service handle per second?
• What percentage of requests fail?
• Is the service running out of CPU or memory?
• Which service is causing a performance bottleneck?
• Can the service handle increased traffic?
• Are database queries or external API calls slowing down the application?

Example: If an Order Service takes 3 seconds to process a request while other services respond within 100 milliseconds, performance metrics help identify where the delay occurs.

2. The Four Golden Signals

The four golden signals are a widely used framework for monitoring the health and performance of services.

Signal	Meaning	Example
Latency	Time taken to process a request.	95% of requests complete within 200 ms.
Traffic	Amount of demand on the service.	1,000 requests per second.
Errors	Rate of requests that fail.	2% of requests return errors.
Saturation	How close a resource is to its usable capacity.	CPU utilization is 85%.

These signals provide a useful starting point for monitoring individual microservices and the overall system.

3. Important Performance Metrics

3.1 Response Time (Latency)

Definition: The time taken to process a request and return a response.

Response time may include application processing, database access, network communication, and calls to other services.

Example: A request starts at 10:00:00.000 and completes at 10:00:00.250. Its response time is 250 milliseconds.

Monitor latency percentiles rather than relying only on the average.

• p50: Median response time; 50% of measured requests complete within this time.
• p95: 95% of measured requests complete within this time.
• p99: 99% of measured requests complete within this time.

Example:

Metric	Value	Interpretation
p50	80 ms	Typical request latency is relatively low.
p95	250 ms	5% of requests take longer than 250 ms.
p99	900 ms	The slowest 1% of requests take at least approximately 900 ms.

Why is p95 important? An average can hide slow requests. Percentiles help reveal the experience of users whose requests take longer than usual.

3.2 Throughput

Definition: The number of requests processed per unit of time.

Formula:

Throughput = Total Requests Processed / Time Period

Example: A service processes 60,000 requests in 60 seconds.

Throughput = 60,000 / 60
            = 1,000 requests per second

Throughput indicates how much work a service handles. However, high throughput alone does not guarantee good performance. Latency and error rates must also be considered.

3.3 Error Rate

Definition: The percentage of requests that fail out of the total requests processed.

Formula:

Error Rate (%) = (Failed Requests / Total Requests) * 100

Example: Out of 10,000 requests, 200 fail.

Error Rate = (200 / 10,000) * 100
           = 2%

Failures may include server errors, timeouts, dependency failures, and application exceptions. Define the error classification according to the service's requirements. For example, an expected HTTP 404 response is not necessarily an application failure.

3.4 CPU Utilization

Definition: The proportion of available CPU capacity being used by a service or its hosting environment.

Example: A container consistently using 90% of its CPU limit may experience increased latency when traffic rises.

High CPU utilization can indicate:

• CPU-intensive business logic.
• Excessive serialization or deserialization.
• Inefficient algorithms.
• Excessive parallel processing.
• Insufficient CPU capacity.

CPU utilization should be evaluated alongside latency, throughput, throttling, and available capacity.

3.5 Memory Utilization

Definition: The memory consumed by an application or its hosting environment.

For .NET applications, monitor:

• Managed heap size.
• Garbage collection frequency and duration.
• Allocation rate.
• Working set or container memory usage.
• Out-of-memory events.

Example: If memory consumption increases continuously under a steady workload, investigate possible memory leaks, retained objects, unbounded caches, or large allocations.

High memory usage is not automatically a problem. The important factors are memory limits, allocation patterns, garbage collection behavior, and application performance.

3.6 Database Performance

Microservices frequently depend on databases. Slow database operations can increase request latency and reduce throughput.

Important database metrics include:

• Query execution time.
• Slow query count.
• Database connection pool utilization.
• Connection acquisition wait time.
• Query timeout count.
• Lock waits and deadlocks.
• Database CPU, memory, and storage utilization.

Example: If the Order Service takes 800 ms to respond and its database query takes 650 ms, database optimization may provide the greatest performance improvement.

3.7 Dependency Latency

Definition: The time taken by a service's dependencies to respond.

Dependencies may include:

• Other microservices.
• Databases.
• Message brokers.
• External payment gateways.
• Third-party APIs.
• Cloud storage services.

Distributed tracing helps associate dependency calls with the request that triggered them.

Example: The Order Service responds slowly because the Payment Service takes 1.5 seconds to complete a request.

3.8 Availability

Definition: The proportion of time a service successfully provides its intended functionality.

A simple time-based availability formula is:

Availability (%) =
    (Total Time - Downtime) / Total Time * 100

Example: If a service is unavailable for 43.2 minutes during a 30-day period, its approximate availability is 99.9%, assuming a 30-day month and that the downtime definition is consistent.

Availability can also be measured using successful requests, depending on the service-level objective.

3.9 Saturation

Definition: The degree to which a resource is approaching its effective capacity.

Monitor saturation for:

• CPU capacity.
• Memory limits.
• Database connection pools.
• Thread pools.
• Request queues.
• Message broker consumer backlogs.

Example: If a database connection pool allows 100 connections and all 100 remain occupied, new operations may wait for a connection even when CPU utilization is low.

3.10 Queue Length and Message Processing Lag

Asynchronous microservices often use message brokers such as RabbitMQ, Kafka, or Azure Service Bus.

Important metrics include:

• Number of messages waiting to be processed.
• Age of the oldest unprocessed message.
• Message processing latency.
• Consumer throughput.
• Retry and dead-letter counts.
• Consumer lag, where supported by the messaging platform.

Example: If a service receives 1,000 messages per second but processes only 700 messages per second, the backlog grows by approximately 300 messages per second while those rates remain constant.

Growing backlog indicates that processing capacity is not keeping pace with incoming work.

4. Performance Metrics Summary with Formulas

Metric	Formula or Measurement	Purpose
Latency	Request completion time - Request start time	Measure response speed.
Throughput	Requests processed / Time	Measure processing capacity.
Error Rate	Failed requests / Total requests * 100	Measure request failures.
Availability	(Total time - Downtime) / Total time * 100	Measure service availability.
CPU Utilization	CPU usage relative to available capacity	Identify CPU constraints.
Memory Usage	Memory consumed by the process or container	Identify memory pressure.
Database Latency	Time spent executing or waiting for database operations	Identify database bottlenecks.
Queue Backlog	Number of pending messages	Identify processing delays.

5. Example: Monitoring an E-Commerce Application

Consider an application with the following microservices:

Customer
   |
   v
API Gateway
   |
   v
Order Service
   |
   +------------------+
   |                  |
   v                  v
Payment Service   Inventory Service
   |
   v
Payment Database

Suppose monitoring reports the following values:

Metric	Observed Value	Possible Interpretation
Order Service p95 latency	1,200 ms	Some requests are slower than expected.
Payment Service p95 latency	900 ms	Payment processing may contribute to the delay.
Order Service error rate	3%	Investigate failed requests and dependencies.
Order Service CPU utilization	45%	CPU is not obviously saturated.
Database query p95	700 ms	Database queries may be a significant bottleneck.

These values are illustrative. They do not establish the root cause by themselves.

Investigation approach:

1. Use distributed tracing to identify slow operations within the Order Service.
2. Inspect Payment Service spans and database operations.
3. Review database query plans, connection waits, and lock contention.
4. Check error logs and dependency timeouts.
5. Apply a targeted fix and compare metrics before and after the change.

The goal is to identify the actual bottleneck rather than scaling every service unnecessarily.

6. Implement Performance Metrics in ASP.NET Core Using C#

In .NET, you can use the built-in System.Diagnostics.Metrics API to create custom application metrics.

The following example counts incoming requests and measures their duration.

Step 1: Create a Metrics Class

using System.Diagnostics.Metrics;

public static class OrderMetrics
{
    public static readonly Meter Meter =
        new("OrderService.Metrics", "1.0.0");

    public static readonly Counter<long> Requests =
        Meter.CreateCounter<long>(
            "orders.requests",
            unit: "{request}");

    public static readonly Histogram<double> RequestDuration =
        Meter.CreateHistogram<double>(
            "orders.request.duration",
            unit: "ms");
}

Explanation:

• Meter groups related metrics under a name.
• Counter records values that increase, such as the number of requests.
• Histogram records a distribution of measurements, such as request durations.

Step 2: Record Metrics in an ASP.NET Core Endpoint

var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

app.MapGet("/orders/{id:int}", async (int id) =>
{
    var stopwatch = System.Diagnostics.Stopwatch.StartNew();

    try
    {
        // Simulate application processing.
        await Task.Delay(100);

        OrderMetrics.Requests.Add(
            1,
            new KeyValuePair<string, object?>(
                "outcome", "success"));

        return Results.Ok(new
        {
            OrderId = id,
            Status = "Processed"
        });
    }
    catch
    {
        OrderMetrics.Requests.Add(
            1,
            new KeyValuePair<string, object?>(
                "outcome", "error"));

        throw;
    }
    finally
    {
        stopwatch.Stop();

        OrderMetrics.RequestDuration.Record(
            stopwatch.Elapsed.TotalMilliseconds);
    }
});

app.Run();

This example records request counts and execution durations for the endpoint. The simulated delay represents application work and should be replaced with actual business logic.

For a production implementation, use middleware or appropriate instrumentation to capture all relevant requests consistently, including failures and requests rejected before reaching the endpoint.

Step 3: Export Metrics Using OpenTelemetry

Install the required packages:

dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Exporter.Prometheus.AspNetCore

Configure metric collection and expose a Prometheus scraping endpoint:

using OpenTelemetry.Metrics;

    var builder = WebApplication.CreateBuilder(args);

    builder.Services.AddOpenTelemetry()
    .WithMetrics(metrics => metrics
    .AddMeter("OrderService.Metrics")
    .AddAspNetCoreInstrumentation()
    .AddPrometheusExporter());

    var app = builder.Build();

    app.MapPrometheusScrapingEndpoint();

    app.MapGet("/orders/{id:int}", (int id) =>
    {
    return Results.Ok(new
    {
    OrderId = id,
    Status = "Processed"
    });
    });

    app.Run();

This configuration illustrates exporting metrics to Prometheus. It is a separate minimal example; combine the metrics configuration with the earlier endpoint and custom metrics class to collect the custom measurements.

Prometheus can scrape the configured metrics endpoint, and Grafana can visualize the collected metrics using a suitable data source configuration.

7. Common Monitoring Tools

Tool	Primary Purpose
Prometheus	Collects and stores time-series metrics.
Grafana	Builds dashboards and visualizes metrics from supported data sources.
OpenTelemetry	Provides instrumentation and collection/export mechanisms for telemetry.
Jaeger	Visualizes distributed traces.
Grafana Tempo	Stores and queries distributed traces.
Azure Monitor	Provides monitoring and observability capabilities for Azure workloads.
Application Insights	Provides application performance monitoring and request/dependency telemetry in Azure.

The choice depends on your hosting platform, application architecture, operational requirements, and budget.

8. What Metrics Should You Monitor in Production?

Start with a small set of metrics that reveal user impact and resource constraints.

Category	Recommended Metrics	Why It Matters
User Experience	Request latency p50, p95, p99	Reveals slow requests.
Reliability	Error rate, timeout rate, availability	Reveals failures affecting users.
Traffic	Requests per second	Shows demand and traffic changes.
Resources	CPU, memory, throttling	Reveals resource constraints.
Dependencies	Database latency, external API latency, connection waits	Identifies dependency bottlenecks.
Messaging	Queue depth, oldest message age, consumer lag	Reveals asynchronous processing delays.
Scaling	Instance count, requests per instance, scaling events	Helps assess capacity and scaling behavior.

9. Service-Level Indicators and Objectives

Monitoring becomes more useful when metrics are connected to explicit service-level goals.

• SLI (Service-Level Indicator): A measurement of service performance or reliability.
• SLO (Service-Level Objective): A target for an SLI over a defined time window.
• SLA (Service-Level Agreement): A formal agreement that may specify service commitments and consequences if they are not met.

Example SLOs:

• 99.9% of valid requests succeed over a rolling 30-day period.
• 95% of eligible API requests complete within 300 milliseconds over a defined measurement window.

These are example targets, not universal requirements. Set SLOs based on user needs, business impact, dependency behavior, and the cost of meeting the target.

10. Common Mistakes in Performance Monitoring

• Monitoring only CPU and memory: A service can be slow because of database waits or external API latency even when CPU usage is low.
• Using only average latency: Averages can hide a poor experience for the slowest requests.
• Collecting too many metrics: Unnecessary metrics increase storage and operational overhead.
• Ignoring business impact: Prioritize metrics that relate to user experience and critical workflows.
• Creating high-cardinality labels: Avoid using unique order IDs, customer IDs, or request IDs as metric labels.
• Alerting on every fluctuation: Use meaningful thresholds, evaluation windows, and SLO-based alerts to reduce noise.
• Scaling without investigation: Identify the bottleneck before increasing resources.

11. Summary

Performance metrics help developers and operations teams understand how microservices behave under real workloads.

The most important starting points are:

• Latency: How long requests take.
• Throughput: How much work the service processes.
• Error Rate: How often requests fail.
• CPU and Memory: How application resources are used.
• Dependency Latency: How databases and other services affect response time.
• Saturation: How close resources are to capacity.
• Queue Backlog: Whether asynchronous work is accumulating.
• Availability: Whether the service is usable when needed.

In ASP.NET Core, use OpenTelemetry and .NET metrics APIs to collect application measurements, Prometheus and Grafana or an equivalent platform to monitor them, and distributed tracing to investigate slow requests across services.

Best practice: Start with latency, traffic, errors, and saturation. Define meaningful SLOs, establish a baseline, and add further metrics only when they help answer a specific operational or business question.

Comments and Discussion