How OpenTelemetry HTTP instrumentation captures requests and propagates trace context

Distributed tracing lives or dies at the HTTP boundary. HTTP instrumentation libraries create the client and server spans that stitch a request path across services and the traceparent header that links them: @opentelemetry/instrumentation-http in Node.js, OpenTelemetry.Instrumentation.Http in .NET, and the OpenTelemetry Java agent’s HTTP client and server instrumentations.

This guide follows that path in order: patched libraries and installation requirements, the bootstrap, propagation and telemetry shape, filtering and enrichment, redaction and retry modeling, then the role of kernel-level eBPF capture alongside SDK instrumentation.

What OpenTelemetry HTTP instrumentation does

Each runtime’s HTTP instrumentation instruments the HTTP layer itself, so every request through that layer produces telemetry without per-endpoint code:

• Node.js: @opentelemetry/instrumentation-http monkey-patches the built-in node:http and node:https modules, covering both incoming server requests and outgoing client requests.
• .NET: OpenTelemetry.Instrumentation.Http instruments System.Net.Http.HttpClient and System.Net.HttpWebRequest; the companion OpenTelemetry.Instrumentation.AspNetCore package handles incoming server requests.
• Java: the OpenTelemetry Java agent instruments java.net.HttpURLConnection (Java 8+), java.net.http.HttpClient (Java 11+), Apache HttpClient 2.0+, OkHttp 2.2+, Spring RestTemplate 3.1+, Spring WebFlux 5.3+, Netty 3.8+, and a long list of other clients and servers in the supported libraries list.

After you register the instrumentation with a tracing pipeline, it produces a client span per outgoing request and a server span per incoming request. When you configure metrics collection, it also produces duration histograms for both directions. The instrumentation injects the W3C traceparent header on outgoing requests and extracts it from incoming ones, which is the mechanism that connects spans from different services into one trace.

Having established what the libraries instrument, the next question is what to install and which runtime versions they require.

Installation and runtime requirements

Package versions and minimum runtimes differ per language, and the stability guarantees differ too. Verified versions as of 2026-09-08:

One footnote on the Node.js row, because the number in the package README is stale. The README still lists Node.js >=14 under Supported Versions, but JS SDK 2.0 raised the floor to ^18.19.0 || >=20.6.0 and dropped Node 14 and 16. That applies to instrumentation-http from 0.200.0 onward. Node 18 itself reached end of life in March 2025, so target Node 20 or later in practice.

Install commands:

# Node.js
npm install @opentelemetry/instrumentation-http @opentelemetry/sdk-node \
  @opentelemetry/exporter-trace-otlp-grpc

# .NET
dotnet add package OpenTelemetry.Instrumentation.Http --version 1.18.0
dotnet add package OpenTelemetry.Instrumentation.AspNetCore --version 1.18.0

For Java, download the agent JAR from the v2.31.1 release; it bundles the OpenTelemetry Java SDK and requires no Maven or Gradle dependency for auto-instrumentation. Standalone library instrumentation exists (for example opentelemetry-spring-webmvc-5.3), but those artifacts are alpha; the agent is the stable path for Spring Web MVC.

One stability caveat applies before you pin versions. The package maintainers mark the Node.js package experimental, so breaking changes can land in minor releases; 0.221.0 changed the default attribute naming, covered below.

Basic setup: registering the instrumentation

The bootstrap shape differs per runtime, but the goal is the same: register the HTTP instrumentation with a tracer provider and wire an OTLP exporter behind a batching processor.

Node.js

The NodeSDK from @opentelemetry/sdk-node is the shortest production-shaped bootstrap. Load this file before any application code so the node:http patch lands before your framework imports it:

// tracing.js: require with node -r ./tracing.js app.js
const { NodeSDK } = require('@opentelemetry/sdk-node');
const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-grpc');
const { BatchSpanProcessor } = require('@opentelemetry/sdk-trace-base');

const sdk = new NodeSDK({
  spanProcessors: [new BatchSpanProcessor(new OTLPTraceExporter())],
  instrumentations: [new HttpInstrumentation()],
});
sdk.start();

If you manage the tracer provider yourself instead of using NodeSDK, call registerInstrumentations({ instrumentations: [new HttpInstrumentation()] }) from @opentelemetry/instrumentation after registering the provider. BatchSpanProcessor buffers and batches span exports, while the simple processor exports one span at a time. Choose between them based on whether you need batched export or immediate per-span export.

.NET

The extension methods AddAspNetCoreInstrumentation and AddHttpClientInstrumentation register server and client instrumentation on the tracer builder:

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddOtlpExporter());

The package targets show that AddHttpClientInstrumentation covers HttpClient on modern .NET and HttpWebRequest on .NET Framework. Configuration options for both packages hang off delegates passed to these methods, which is where filtering and enrichment plug in later.

Java

Java offers two registration paths, and the trade-off is control versus coverage. The agent path requires no code:

java -javaagent:opentelemetry-javaagent.jar \
     -Dotel.exporter.otlp.endpoint=http://collector:4317 \
     -jar app.jar

The agent instruments every supported HTTP client and server on the classpath at load time. The manual path builds an OpenTelemetrySdk in code and attaches standalone instrumentation artifacts per library, which gives you dependency-level control but leaves any library you forgot uninstrumented. For services that need broad supported-library coverage, use the agent; reach for manual SDK registration when startup constraints or packaging and policy requirements rule out the agent.

With spans flowing, the next question is how those spans connect across service boundaries.

How traceparent headers are injected and extracted

All three runtimes default to W3C Trace Context plus Baggage propagation. The SDK environment variable spec defines the default as tracecontext,baggage: outgoing requests get a traceparent header injected by the client instrumentation, and incoming requests have it extracted before the server span starts, so the server span becomes a child of the remote client span.

Swapping in B3 (common in Zipkin and older Istio environments) works differently per runtime:

• Node.js: install @opentelemetry/propagator-b3 and pass it to the SDK: new NodeSDK({ textMapPropagator: new B3Propagator() }). Multi-header B3 takes new B3Propagator({ injectEncoding: B3InjectEncoding.MULTI_HEADER }), as the package docs show. NodeSDK also honors OTEL_PROPAGATORS when code does not set a propagator.
• Java: set OTEL_PROPAGATORS=b3 (single header) or b3multi, or compose programmatically with B3Propagator.injectingSingleHeader() / injectingMultiHeaders(), as the Java configuration docs explain. Both factory variants extract from single and multi-header formats simultaneously.
• .NET: the core SDK does not implement OTEL_PROPAGATORS. Set the propagator in code: Sdk.SetDefaultTextMapPropagator(new B3Propagator()) from OpenTelemetry.Extensions.Propagators 1.18.0, with singleHeader: true for the single-header format. The env var works only under .NET zero-code automatic instrumentation.

That .NET gap bites teams standardizing propagator config through Helm-set environment variables: the same OTEL_PROPAGATORS value that reconfigures a Node.js and Java fleet silently does nothing in a code-instrumented .NET service. Once context flows, the spans themselves need consistent attribute names, which is where semantic conventions come in.

Default span attributes and HTTP semantic conventions

HTTP span attributes are stable, and the current semantic conventions release is v1.44.0. The required set differs between client and server spans. The HTTP spans specification defines these requirements:

If your dashboards still query http.method, http.status_code, or http.url, those are the old names from v1.20.0 and earlier. The HTTP attribute registry maps them to http.request.method, http.response.status_code, and url.full. The net.* namespace moved too: the network registry maps net.peer.name to server.address on client spans. The migration guide treats v1.23.1 as the stable endpoint for HTTP conventions.

This matters operationally in Node.js right now, and the migration window has already closed. Version 0.221.0 made the stable conventions the only option: instrumentation-http emits only stable HTTP semantic conventions, and OTEL_SEMCONV_STABILITY_OPT_IN no longer changes attribute or metric emission, because the old v1.7.0 names and the dual-emission http/dup mode were both removed in that release. The variable worked in versions 0.54.0 through 0.220.0, where http emitted only the new names and http/dup emitted both at once, as the HTTP metrics spec describes. If you are still on 0.220.0 or earlier, that dual-emission mode is your only staged path, so audit every alert and dashboard for the renamed attributes before upgrading to 0.221.0. There is no escape hatch on the other side.

HTTP metrics produced out of the box

When you configure metrics collection, the instrumentation emits two stable duration histograms: http.server.request.duration and http.client.request.duration. Both use seconds as the unit and carry advisory bucket boundaries of [0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10], as defined by the HTTP metrics semconv. The dimensions follow the stable span conventions, including http.request.method and http.response.status_code, so latency slices line up with trace queries. If you migrated from the pre-stable conventions, note that stabilization moved the unit from milliseconds to seconds and dropped the zero bucket boundary.

Four body-size metrics exist but remain at Development stability and are opt-in:

• http.server.request.body.size (histogram, bytes)
• http.server.response.body.size (histogram, bytes)
• http.client.request.body.size (histogram, bytes)
• http.client.response.body.size (histogram, bytes)

The spec defines several more at Development stability that instrumentations do not emit by default, including http.server.active_requests, http.client.active_requests, http.client.open_connections, and http.client.connection.duration.

Enabling the body-size metrics in Java takes more than the semconv opt-in variable. The Java instrumentation list requires you to set otel.instrumentation.http.client.emit-experimental-telemetry and otel.instrumentation.http.server.emit-experimental-telemetry (both default false), or their environment equivalents OTEL_INSTRUMENTATION_HTTP_CLIENT_EMIT_EXPERIMENTAL_TELEMETRY and OTEL_INSTRUMENTATION_HTTP_SERVER_EMIT_EXPERIMENTAL_TELEMETRY. Setting OTEL_SEMCONV_STABILITY_OPT_IN=http alone leaves them off.

Metrics and spans both inflate fast when instrumentation traces health checks and internal probes, which brings up filtering.

Filtering requests: excluding health checks and internal endpoints

A Kubernetes liveness probe every 10 seconds per pod produces spans nobody will ever query. Both Node.js and .NET expose predicate hooks to drop them at the instrumentation layer, and the two runtimes invert the predicate’s meaning.

In Node.js, HttpInstrumentationConfig takes ignoreIncomingRequestHook and ignoreOutgoingRequestHook. Returning true skips the span:

new HttpInstrumentation({
  ignoreIncomingRequestHook: (req) =>
    req.url === '/healthz' || req.url === '/readyz',
  ignoreOutgoingRequestHook: (options) =>
    options.hostname === 'metadata.internal',
});

In .NET, FilterHttpRequestMessage on HttpClientTraceInstrumentationOptions and Filter on AspNetCoreTraceInstrumentationOptions work the opposite way: returning true keeps the request. The HttpClient option source defines the client behavior, while the ASP.NET Core options define the server behavior. On .NET Framework the client-side equivalent is FilterHttpWebRequest.

.AddAspNetCoreInstrumentation(o =>
    o.Filter = ctx => !ctx.Request.Path.StartsWithSegments("/healthz"))
.AddHttpClientInstrumentation(o =>
    o.FilterHttpRequestMessage = req =>
        req.RequestUri?.Host != "metadata.internal")

That inversion is a real porting hazard: copy a Node.js predicate into .NET unchanged and you keep exactly the traffic you meant to drop. Three more behaviors to know:

• Exceptions fail safe but differently. Node.js catches hook exceptions and logs them via _diag.error without affecting the span decision, as the instrumentation source shows. In .NET, a filter that throws is treated the same as one returning false, so the instrumentation does not collect the request, and the error goes to the instrumentation EventSource.
• Filters run after the sampler in .NET. The instrumentation README states that the filter option does its filtering after the sampler is invoked, so filtered requests still pay sampler cost.
• .NET filters apply to traces only. Metrics filtering goes through SDK Views. Enrich and Filter support for ASP.NET Core metrics was removed before the instrumentation reached its stable release and is not available in any supported version (opentelemetry-dotnet #4981, planned in contrib #1733). The trace-side Enrich and Filter APIs are unaffected.

Java handles exclusion through agent configuration rather than in-process predicates. Filtering removes noise; the complementary need is adding signal, which header capture and enrichment hooks provide.

Capturing headers and enriching spans

By default, Node.js and Java capture no HTTP headers. The HTTP spans spec explains why: instrumentations should make header capture an explicit choice, because capturing every request header risks leaking sensitive information.

In Node.js, headersToSpanAttributes takes allowlists split by side and direction, as the config types define:

new HttpInstrumentation({
  headersToSpanAttributes: {
    server: { requestHeaders: ['content-type', 'x-request-id'] },
    client: { responseHeaders: ['content-length'] },
  },
});

Names match case-insensitively and land as attributes following the pattern http.request.header.<name> or http.response.header.<name>, always typed as string[]. The package README documents the naming change that came with the stable conventions: legacy mode converted hyphens to underscores (http.response.header.content_length), while the now-default stable mode preserves them (http.response.header.content-length). The behavior arrived in 0.212.0 gated on stable semconv and became unconditional in 0.221.0. Queries written against the old keys break on upgrade.

In Java, the agent captures headers through four comma-separated system properties, all experimental: otel.instrumentation.http.server.capture-request-headers, otel.instrumentation.http.server.capture-response-headers, and the http.client.* equivalents. The environment variable forms follow the usual uppercase-underscore mapping, such as OTEL_INSTRUMENTATION_HTTP_SERVER_CAPTURE_REQUEST_HEADERS. A header Content-Type: application/json becomes http.request.header.content-type → ["application/json"], lowercased with hyphens preserved.

For attributes beyond headers, each runtime exposes enrichment callbacks. Node.js HttpInstrumentationConfig accepts requestHook and responseHook functions that receive the span plus the raw request or response object, so you can set tenant IDs and feature flags as custom attributes. You can also add route metadata. .NET’s HttpClient and ASP.NET Core options expose enrich callbacks that hand you the Activity alongside the underlying HttpRequestMessage or HttpContext; per the metrics change noted above, those callbacks apply to traces only. Enrichment adds data to spans, which raises the opposite concern: keeping sensitive data out of them.

Query parameter redaction and sensitive data handling

URLs routinely carry credentials as query parameters, and instrumentation records those URLs verbatim in url.full and url.query unless redaction is configured. Node.js instrumentation-http ships built-in redaction for exactly this: redactedQueryParams for client spans and redactedQueryParamsServer for server spans. Both are experimental.

The spec asks instrumentations to redact five signed-URL keys by default: X-Amz-Signature, X-Amz-Credential, X-Amz-Security-Token, sig, and X-Goog-Signature. That requirement is itself at Development stability, and the list is expected to change. The Node.js built-in list adds Signature and AWSAccessKeyId on top of those five. The instrumentation replaces matching values with the literal string REDACTED before recording the URL in url.full (client) or url.query (server), and keeps the parameter key intact, as the package README documents.

The override semantics reward a careful read:

• Omit the option and the built-in list applies.
• Supply a non-empty array and it replaces the defaults entirely; it does not merge, so include the AWS and GCP signature params again if you still need them.
• Supply an empty array and redaction turns off on that side.
• The two options are independent. Configuring redactedQueryParams does nothing for server spans.
• Matching is exact and case-sensitive, which the spec also requires, so token does not catch Token.

Redaction handles the URL surface. The remaining modeling question is what a span even represents when a single logical request produces multiple wire attempts.

Edge cases: retries, redirects, and span modeling

The HTTP spans spec answers the span-per-try question directly. Instrumentations should create one span per attempt to send a request over the wire, and those that do should not also emit an encompassing logical client span. Repeated attempts carry http.request.resend_count with their ordinal; the first attempt carries no such attribute, and the count increments for any cause of resending, including redirects, authorization challenges, 503s, and network errors. When instrumentation cannot create per-attempt spans, it may fall back to one span for the top-most operation, in which case url.full must be the URL originally requested, before any redirects.

Implementation reality lags the spec unevenly across runtimes:

The operational consequence: identical retry behavior looks different per language in your trace backend. A .NET service that follows three redirects shows four client spans; the same flow through Node’s http module shows one span pointing at the original URL. This is also the sharpest argument in the automatic-versus-manual trade-off. Automatic instrumentation gives you the modeling the library authors implemented, gaps included; manual spans around your retry loop give you exact modeling at the cost of writing and maintaining that code in every service.

That per-service maintenance cost, multiplied across three runtimes and dozens of teams, is the setup burden a kernel-level approach removes.

How this fits with groundcover

Everything above requires per-service work: install the package and wire the bootstrap, then tune the filters before redeploying. groundcover’s Flora eBPF sensor captures HTTP requests from a different layer. Flora deploys as a single Kubernetes DaemonSet, one pod per node, and reads HTTP traffic directly from the Linux kernel. It captures supported HTTP traffic across the cluster with zero application instrumentation: no SDKs, no language agents, no restarts, no code changes. Full-cluster visibility arrives within hours instead of an instrumentation sprint across Node.js, .NET, and Java teams.

Existing OpenTelemetry investment carries over rather than competing. groundcover accepts OTel as a first-class data source: OTLP spans from the SDKs you configured above can appear alongside Flora-captured signals in the same interface. That coexistence covers the cases eBPF and SDKs each handle best; kernel capture sees services without application instrumentation, while your OTel spans keep the in-process context, custom attributes, and enrichment hooks described in this guide.

The data path stays in your infrastructure. groundcover’s BYOC architecture (Bring Your Own Cloud, where your data plane runs in your own cloud account) stores logs and traces, plus Kubernetes events, in ClickHouse. It stores metrics in VictoriaMetrics with PromQL compatibility. Both systems run inside your environment, while groundcover manages the control plane for UI and orchestration. Requirements are Kubernetes 1.21 or later and Linux kernel 4.16 or later. groundcover supports EKS and AKS, as well as GKE. If you want to validate the architecture on your own cluster, the free plan includes BYOC and requires no credit card: deploy Flora on one cluster and evaluate full-cluster visibility within hours.

FAQ

These answers summarize the setup, propagation, filtering, enrichment, metrics, and retry behavior covered above.

What does OpenTelemetry HTTP instrumentation capture by default?

Client and server spans for every request through the instrumented HTTP layer, with stable attributes including http.request.method, url.full (client), url.path and url.scheme (server), server.address, and http.response.status_code. When you configure metrics collection, the instrumentation also emits the http.server.request.duration and http.client.request.duration histograms. Node.js and Java capture no headers unless you configure an allowlist.

How do I install it in each runtime?

Node.js: npm install @opentelemetry/instrumentation-http (0.222.0, Node ^18.19.0 || >=20.6.0). .NET: dotnet add package OpenTelemetry.Instrumentation.Http plus OpenTelemetry.Instrumentation.AspNetCore (both 1.18.0). Java: attach the agent JAR (2.31.1, Java 8+) with -javaagent.

How does trace context propagate between services?

The client instrumentation injects a W3C traceparent header on outgoing requests and the server side extracts it, with tracecontext,baggage as the default propagator pair. Swap to B3 via OTEL_PROPAGATORS in Node.js and Java; the core .NET SDK ignores that variable, so call Sdk.SetDefaultTextMapPropagator(new B3Propagator()) in code.

How do I stop tracing health check endpoints?

Node.js: return true from ignoreIncomingRequestHook for /healthz-style paths. .NET: return false from the Filter predicate on AspNetCoreTraceInstrumentationOptions. Remember the inversion: true skips in Node.js and keeps in .NET.

Can I capture headers without leaking PII?

Yes, because capture is allowlist-only in Node.js and Java: name specific headers in headersToSpanAttributes (Node.js) or the otel.instrumentation.http.*.capture-*-headers properties (Java), and they appear as http.request.header.<name> attributes. Never allowlist Authorization, Cookie, or other credential-bearing headers, and pair header capture with Node’s query-parameter redaction so signed URLs don’t leak through url.full.

How do I add custom attributes to HTTP spans?

Use requestHook and responseHook on the Node.js config, which receive the span and the raw request or response object, or the .NET enrich callbacks that expose the Activity alongside the underlying request objects. Both run per request, so keep the logic cheap.

Which metrics come out of the box?

When you configure metrics collection, HTTP instrumentation emits two stable duration histograms, http.server.request.duration and http.client.request.duration, in seconds with 14 advisory buckets from 5 ms to 10 s. The four body-size histograms are Development stability and opt-in; Java additionally requires the emit-experimental-telemetry properties.

Are retries and redirects one span or many?

The spec asks for one span per wire attempt, with http.request.resend_count on resends. In practice, .NET emits per-attempt spans, Java’s OkHttp 3 and Apache HttpClient 4.3 instrumentations follow the spec, and Node.js instrumentation-http does not model retries or redirects at all, so verify what your backend shows before building alerts on span counts.

Do I still need OTEL_SEMCONV_STABILITY_OPT_IN in Node.js?

No. From 0.221.0, @opentelemetry/instrumentation-http emits only the stable conventions and the variable no longer affects attribute or metric names. It worked in 0.54.0 through 0.220.0, so if you are on one of those versions, use http/dup to run both naming schemes while you migrate dashboards, then upgrade.