OpenTelemetry#

OpenTelemetry is an open-source observability framework designed to provide a unified standard for collecting and exporting telemetry data such as traces, metrics, and logs. It aims to make observability a built-in feature of software development, simplifying the integration and standardization of telemetry data across various services. For more details, you can read the official OpenTelemetry documentation.

Quick Start#

To use OpenTelemetry in FastStream, you just need to:

1. Install FastStream along with opentelemetry-sdk and opentelemetry-exporter-otlp

   pip install "faststream[otel]" opentelemetry-exporter-otlp
   

2. Configure TracerProvider with the gRPC exporter:

   1 2 3 4 5 6 7 8 9 10 11 12

   from opentelemetry import trace from opentelemetry.sdk.resources import Resource from opentelemetry.sdk.trace import TracerProvider from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter from opentelemetry.sdk.trace.export import BatchSpanProcessor resource = Resource.create(attributes={"service.name": "faststream"}) tracer_provider = TracerProvider(resource=resource) trace.set_tracer_provider(tracer_provider) exporter = OTLPSpanExporter(endpoint="http://localhost:4317") processor = BatchSpanProcessor(exporter) tracer_provider.add_span_processor(processor)

3. Add TelemetryMiddleware to your broker:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

from faststream import FastStream
from faststream.kafka import KafkaBroker
from faststream.kafka.opentelemetry import KafkaTelemetryMiddleware

broker = KafkaBroker(
    middlewares=(
        KafkaTelemetryMiddleware(tracer_provider=tracer_provider),
    ),
)
app = FastStream(broker)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

from faststream import FastStream
from faststream.confluent import KafkaBroker
from faststream.confluent.opentelemetry import KafkaTelemetryMiddleware

broker = KafkaBroker(
    middlewares=(
        KafkaTelemetryMiddleware(tracer_provider=tracer_provider)
    ),
)
app = FastStream(broker)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

from faststream import FastStream
from faststream.rabbit import RabbitBroker
from faststream.rabbit.opentelemetry import RabbitTelemetryMiddleware

broker = RabbitBroker(
    middlewares=(
        RabbitTelemetryMiddleware(tracer_provider=tracer_provider),
    ),
)
app = FastStream(broker)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

from faststream import FastStream
from faststream.nats import NatsBroker
from faststream.nats.opentelemetry import NatsTelemetryMiddleware

broker = NatsBroker(
    middlewares=(
        NatsTelemetryMiddleware(tracer_provider=tracer_provider),
    ),
)
app = FastStream(broker)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

from faststream import FastStream
from faststream.redis import RedisBroker
from faststream.redis.opentelemetry import RedisTelemetryMiddleware

broker = RedisBroker(
    middlewares=(
        RedisTelemetryMiddleware(tracer_provider=tracer_provider),
    ),
)
app = FastStream(broker)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

from faststream import FastStream
from faststream.mqtt import MQTTBroker
from faststream.mqtt.opentelemetry import MQTTTelemetryMiddleware

broker = MQTTBroker(
    middlewares=(
        MQTTTelemetryMiddleware(tracer_provider=tracer_provider),
    ),
)
app = FastStream(broker)

In the following documentation sections, you can find detailed information about all OpenTelemetry features available in FastStream.

OpenTelemetry FastStream Example#

You can also take a look at a pre-configured project and use it as a reference for your services and infrastructure.

To see how to set up, visualize, and configure tracing for FastStream services, checkout the example project.

The example includes:

• Three FastStream services
• Exporting traces to Grafana Tempo via gRPC
• Visualization of traces using Grafana
• Collecting metrics and exporting them with Prometheus
• A Grafana dashboard for metrics
• Examples with custom spans
• A pre-configured docker-compose with the entire infrastructure

Visualized via Grafana and Tempo