feat(otlp): add gRPC protocol support for trace ingestion by jchrostek-dd · Pull Request #1105 · DataDog/datadog-lambda-extension

jchrostek-dd · 2026-03-16T17:41:20Z

Summary

Add gRPC support for OTLP trace ingestion on port 4317, complementing the existing HTTP endpoint on port 4318
Implement TraceService trait using tonic for gRPC server
Refactor OtlpProcessor to accept ExportTraceServiceRequest directly (avoids re-encoding for gRPC)
Wire up existing config: DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_ENDPOINT and DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_MAX_RECV_MSG_SIZE_MIB

Motivation

Users were configuring gRPC OTLP endpoints (DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_ENDPOINT) but the feature was not implemented, resulting in silent failures. This PR enables gRPC support. This was for an old ticket, but we should go ahead and add this, especially since OTLP is getting more popular.

Changes

File	Change
`bottlecap/Cargo.toml`	Add tonic with transport/server features
`bottlecap/src/otlp/grpc_agent.rs`	NEW - gRPC server implementing TraceService
`bottlecap/src/otlp/processor.rs`	Refactor to accept ExportTraceServiceRequest directly
`bottlecap/src/otlp/mod.rs`	Update enablement logic for HTTP/gRPC
`bottlecap/src/bin/bottlecap/main.rs`	Start both HTTP and gRPC servers

Binary Size Impact

Measured from CI layer size checks (arm64):

Metric	Main	PR	Difference
Zipped	5,074 KB	5,164 KB	+90 KB
Unzipped	11,183 KB	11,376 KB	+193 KB

This is a ~1.7% increase, well under historical concerns (OTLP was removed from Go agent for adding ~1.15MB).

Test plan

New integration tests for gRPC protocol (4 tests)
All existing OTLP tests continue to pass
cargo fmt, cargo clippy, cargo test pass
All CI checks passed (47 checks including integration suites)

🤖 Generated with Claude Code

Copilot

Pull request overview

Adds OTLP trace ingestion over gRPC (port 4317) alongside the existing HTTP OTLP endpoint (port 4318), wiring enablement through existing config and adding integration coverage for the new protocol.

Changes:

Introduce a tonic-based gRPC OTLP TraceService server and start it from the bottlecap binary when configured.
Refactor the OTLP processor to accept a pre-deserialized ExportTraceServiceRequest for reuse across HTTP and gRPC paths.
Extend integration test stack + test suite with a Node.js Lambda that exports traces via OTLP gRPC.

Reviewed changes

Copilot reviewed 8 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`integration-tests/tests/otlp.test.ts`	Adds a new test block covering gRPC OTLP trace export.
`integration-tests/lib/stacks/otlp.ts`	Provisions an additional Node Lambda configured to export traces via OTLP gRPC.
`integration-tests/lambda/otlp-node/package.json`	Adds the OTLP gRPC trace exporter dependency.
`integration-tests/lambda/otlp-node/grpc-handler.js`	Implements the Node Lambda handler that emits spans and exports via gRPC.
`bottlecap/src/otlp/processor.rs`	Splits decoding from processing to support gRPC’s already-decoded request type.
`bottlecap/src/otlp/mod.rs`	Adds gRPC module + enablement helpers and tests.
`bottlecap/src/otlp/grpc_agent.rs`	New gRPC server implementation for OTLP trace ingestion.
`bottlecap/src/bin/bottlecap/main.rs`	Starts HTTP and/or gRPC OTLP servers based on config.
`bottlecap/Cargo.toml`	Adds tonic dependency needed for the gRPC server.
`bottlecap/Cargo.lock`	Lockfile updates for the added tonic dependency.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

bottlecap/src/otlp/grpc_agent.rs

+    fn parse_port(endpoint: Option<&String>, default_port: u16) -> u16 {
+        if let Some(endpoint) = endpoint {
+            let port = endpoint.split(':').nth(1);
+            if let Some(port) = port {
+                return port.parse::<u16>().unwrap_or_else(|_| {
+                    error!("Invalid OTLP gRPC port, using default port {default_port}");
+                    default_port
+                });
+            }
+
+            error!("Invalid OTLP gRPC endpoint format, using default port {default_port}");
+        }
+
+        default_port
+    }


bottlecap/src/otlp/grpc_agent.rs

+        let max_recv_msg_size = self
+            .config
+            .otlp_config_receiver_protocols_grpc_max_recv_msg_size_mib
+            .map_or(DEFAULT_MAX_RECV_MSG_SIZE, |mib| {
+                mib.unsigned_abs() as usize * 1024 * 1024
+            });


integration-tests/tests/otlp.test.ts

+    it('should preserve span parent-child relationships', () => {
+      const result = getResult();
+      expect(result).toBeDefined();
+      const trace = result.traces![0];
+      // Verify that the trace has at least one span


bottlecap/src/otlp/grpc_agent.rs

+        let tracer_header_tags = DatadogTracerHeaderTags::default();
+        let body_size = size_of_val(&traces);
+        if body_size == 0 {
+            error!("OTLP gRPC | Not sending traces, processor returned empty data");
+            return Err(Status::internal(
+                "Not sending traces, processor returned empty data",
+            ));
+        }


Add gRPC support for OTLP trace ingestion on port 4317, complementing the existing HTTP endpoint on port 4318. Changes: - Add tonic dependency with transport/server features - Implement TraceService trait in new grpc_agent.rs module - Refactor OtlpProcessor to accept ExportTraceServiceRequest directly - Update enablement logic to check both HTTP and gRPC endpoints - Wire up max message size config from DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_MAX_RECV_MSG_SIZE_MIB - Add integration tests for gRPC protocol Users can now configure gRPC OTLP export by setting: DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_ENDPOINT=localhost:4317 Binary size impact: +16 bytes (negligible - tonic transport was already pulled in transitively). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Fix parse_port to handle URL schemes (http://, https://) - Validate max_recv_msg_size: reject negative values, cap at 64MiB - Use encoded_len() for body_size, traces.iter().all(Vec::is_empty) for empty check - Rename misleading test to match actual assertions - Add tests for URL scheme handling 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Simplify enablement logic by using only should_enable_http and should_enable_grpc functions directly. The combined check is now inlined in start_otlp_agent. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Previously, each OTLP agent created its own CancellationToken internally, but only the gRPC agent's token was captured and used for shutdown. The HTTP agent's token was never retrieved, so it couldn't be cancelled. Now both agents accept a CancellationToken parameter, and the caller creates a single shared token. One cancel() cleanly shuts down both. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Align with trace agent's TRACE_REQUEST_BODY_LIMIT (50MB) instead of 64MB. More appropriate for Lambda's memory-constrained environment. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Rename should_enable_http → should_enable_otlp_http Rename should_enable_grpc → should_enable_otlp_grpc 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

jchrostek-dd requested a review from a team as a code owner March 16, 2026 17:41

jchrostek-dd requested review from Copilot and litianningdatadog March 16, 2026 17:41

Copilot started reviewing on behalf of jchrostek-dd March 16, 2026 17:42 View session

Copilot AI reviewed Mar 16, 2026

View reviewed changes

jchrostek-dd removed the request for review from litianningdatadog March 16, 2026 18:20

jchrostek-dd changed the title ~~feat(otlp): add gRPC protocol support for trace ingestion~~ [WIP] feat(otlp): add gRPC protocol support for trace ingestion Mar 16, 2026

jchrostek-dd and others added 4 commits March 17, 2026 12:19

ci: retrigger CI for flaky arm64 fips alpine build

b3dd772

jchrostek-dd force-pushed the john/otlp-grpc-support branch from ab535e1 to b3dd772 Compare March 17, 2026 16:22

jchrostek-dd force-pushed the john/otlp-grpc-support branch from 92e6442 to ef1508e Compare March 17, 2026 16:46

jchrostek-dd and others added 3 commits March 17, 2026 12:48

refactor(otlp): rename enablement functions for clarity

a2df47f

Rename should_enable_http → should_enable_otlp_http Rename should_enable_grpc → should_enable_otlp_grpc 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

chore(otlp): remove redundant comments

650a404

🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

jchrostek-dd force-pushed the john/otlp-grpc-support branch from a6b99bc to 650a404 Compare March 17, 2026 17:00

jchrostek-dd changed the title ~~[WIP] feat(otlp): add gRPC protocol support for trace ingestion~~ feat(otlp): add gRPC protocol support for trace ingestion Mar 17, 2026

jchrostek-dd requested review from lym953 March 17, 2026 18:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(otlp): add gRPC protocol support for trace ingestion#1105

feat(otlp): add gRPC protocol support for trace ingestion#1105
jchrostek-dd wants to merge 8 commits intomainfrom
john/otlp-grpc-support

jchrostek-dd commented Mar 16, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

jchrostek-dd commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Changes

Binary Size Impact

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jchrostek-dd commented Mar 16, 2026 •

edited

Loading