Skip to content

feat: Adopt TraceExporter's Telemetry stuff #183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Aaalibaba42 wants to merge 5 commits into
base: main
Choose a base branch
from
+123 −41
Open

feat: Adopt TraceExporter's Telemetry stuff #183

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
aa886f6
feat: Adopt TraceExporter's Telemetry stuff
Aaalibaba42 Jul 20, 2026
f77f8a3
docs: fix libdatadog's behavior when 0 is passed
Aaalibaba42 Jul 21, 2026
0db60a5
chore: js tomfoolery
Aaalibaba42 Jul 21, 2026
40d34eb
fix: comments
Aaalibaba42 Jul 27, 2026
ab0de16
fix: use Cells uniformaly
Aaalibaba42 Jul 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 61 additions & 14 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions crates/capabilities/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ bytes = "1.4"
futures-core = "0.3"
anyhow = "1"
# TODO: Replace this temporary libdatadog PR rev with the official release/tag
# that contains DataDog/libdatadog#2235, then regenerate Cargo.lock.
libdd-capabilities = { git = "https://github.com/DataDog/libdatadog.git", rev = "8cd68ab922fb7aade0f089ccfc91291d874673af" }
# that contains DataDog/libdatadog#2172, then regenerate Cargo.lock.
libdd-capabilities = { git = "https://github.com/DataDog/libdatadog.git", rev = "73f23a2c03be39971c966e444785d63b6fd52e81" }

[dev-dependencies]
wasm-bindgen-test = "0.3"
17 changes: 9 additions & 8 deletions crates/pipeline/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,18 @@ serde = { version = "1.0", features = ["derive"] }
serde_json = "1"
libdatadog-nodejs-capabilities = { path = "../capabilities" }
# TODO: Replace these temporary libdatadog PR revs with the official release/tag
# that contains DataDog/libdatadog#2235, then regenerate Cargo.lock.
libdd-capabilities = { git = "https://github.com/DataDog/libdatadog.git", rev = "8cd68ab922fb7aade0f089ccfc91291d874673af" }
libdd-common = { git = "https://github.com/DataDog/libdatadog.git", rev = "8cd68ab922fb7aade0f089ccfc91291d874673af", default-features = false }
libdd-data-pipeline = { git = "https://github.com/DataDog/libdatadog.git", rev = "8cd68ab922fb7aade0f089ccfc91291d874673af", default-features = false }
libdd-trace-utils = { git = "https://github.com/DataDog/libdatadog.git", rev = "8cd68ab922fb7aade0f089ccfc91291d874673af", default-features = false, features = ["change-buffer"] }
libdd-trace-stats = { git = "https://github.com/DataDog/libdatadog.git", rev = "8cd68ab922fb7aade0f089ccfc91291d874673af", default-features = false }
libdd-trace-protobuf = { git = "https://github.com/DataDog/libdatadog.git", rev = "8cd68ab922fb7aade0f089ccfc91291d874673af", default-features = false }
libdd-shared-runtime = { git = "https://github.com/DataDog/libdatadog.git", rev = "8cd68ab922fb7aade0f089ccfc91291d874673af", default-features = false }
# that contains DataDog/libdatadog#2172, then regenerate Cargo.lock.
libdd-capabilities = { git = "https://github.com/DataDog/libdatadog.git", rev = "73f23a2c03be39971c966e444785d63b6fd52e81" }
libdd-common = { git = "https://github.com/DataDog/libdatadog.git", rev = "73f23a2c03be39971c966e444785d63b6fd52e81", default-features = false }
libdd-data-pipeline = { git = "https://github.com/DataDog/libdatadog.git", rev = "73f23a2c03be39971c966e444785d63b6fd52e81", default-features = false, features = ["telemetry"] }
libdd-trace-utils = { git = "https://github.com/DataDog/libdatadog.git", rev = "73f23a2c03be39971c966e444785d63b6fd52e81", default-features = false, features = ["change-buffer"] }
libdd-trace-stats = { git = "https://github.com/DataDog/libdatadog.git", rev = "73f23a2c03be39971c966e444785d63b6fd52e81", default-features = false }
libdd-trace-protobuf = { git = "https://github.com/DataDog/libdatadog.git", rev = "73f23a2c03be39971c966e444785d63b6fd52e81", default-features = false }
libdd-shared-runtime = { git = "https://github.com/DataDog/libdatadog.git", rev = "73f23a2c03be39971c966e444785d63b6fd52e81", default-features = false }
rmp-serde = "1"
bytes = "1"
http = "1"
web-time = "1"
console_error_panic_hook = "0.1"

[target.'cfg(target_arch = "wasm32")'.dependencies]
Expand Down
57 changes: 45 additions & 12 deletions crates/pipeline/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
use libdatadog_nodejs_capabilities::WasmCapabilities;
use libdd_data_pipeline::trace_exporter::agent_response::AgentResponse;
use libdd_data_pipeline::trace_exporter::{
TraceExporter, TraceExporterBuilder, TraceExporterOutputFormat,
TelemetryConfig, TraceExporter, TraceExporterBuilder, TraceExporterOutputFormat,
};
use libdd_data_pipeline::OtlpProtocol;
use libdd_shared_runtime::LocalRuntime;
Expand Down Expand Up @@ -197,13 +197,17 @@ pub struct WasmSpanState {
/// Datadog agent. libdatadog maps its internal traces to OTLP, so no
/// JS-formatted spans are involved. Like `use_v05`, only takes effect if
/// set before the first send (when the exporter is built).
otlp_endpoint: RefCell<Option<String>>,
otlp_endpoint: Cell<Option<String>>,
/// OTLP wire protocol (`http/json` default, or `http/protobuf`). Only
/// applied when `otlp_endpoint` is set.
otlp_protocol: Cell<Option<OtlpProtocol>>,
/// Extra HTTP headers for OTLP export (e.g. collector auth), as key/value
/// pairs. Only applied when `otlp_endpoint` is set.
otlp_headers: RefCell<Vec<(String, String)>>,
otlp_headers: Cell<Vec<(String, String)>>,
/// When set, the lazily-built exporter has telemetry enabled with this
/// config. Only takes effect if set before the first send
/// (when the exporter is built).
telemetry_config: Cell<Option<TelemetryConfig>>,
/// Latched message from a failed lazy `build_async`. Building is one-shot and
/// a failure is fatal (bad config), so once set every send returns it (as a
/// distinguishable error) instead of a misleading "builder already consumed",
Expand Down Expand Up @@ -335,9 +339,10 @@ impl WasmSpanState {
prepared_spans: RefCell::new(Vec::new()),
sending: Cell::new(false),
use_v05: Cell::new(false),
otlp_endpoint: RefCell::new(None),
otlp_endpoint: Cell::new(None),
otlp_protocol: Cell::new(None),
otlp_headers: RefCell::new(Vec::new()),
otlp_headers: Cell::new(Vec::new()),
telemetry_config: Cell::new(None),
build_error: RefCell::new(None),
})
}
Expand All @@ -359,7 +364,7 @@ impl WasmSpanState {
/// Takes precedence over `setUseV05` (OTLP bypasses the agent entirely).
#[wasm_bindgen(js_name = "setOtlpEndpoint")]
pub fn set_otlp_endpoint(&self, url: String) {
*self.otlp_endpoint.borrow_mut() = Some(url);
self.otlp_endpoint.set(Some(url));
}

/// Select the OTLP wire protocol: `http/json` (default) or `http/protobuf`.
Expand All @@ -374,6 +379,31 @@ impl WasmSpanState {
Ok(())
}

/// Enable telemetry on the lazily-built trace exporter.
///
/// Must be called before the first `sendPreparedChunk`. Later calls have
/// no effect.
///
/// # Arguments
///
/// - `heartbeat_ms`: sets the metric-flush cadence. Set to 0 to defer to
/// libdatadog's default interval.
/// - `runtime_id`: tags telemetry payloads with the tracer's runtime id when provided
/// - `debug_enabled`: toggles libdd-telemetry's verbose logging
#[wasm_bindgen(js_name = "enableTelemetry")]
pub fn enable_telemetry(
&self,
heartbeat_ms: u32,
runtime_id: Option<String>,
debug_enabled: bool,
) {
self.telemetry_config.set(Some(TelemetryConfig {
heartbeat: heartbeat_ms as u64,
runtime_id,
debug_enabled,
}));
}

/// Set extra HTTP headers for OTLP export as a flat `[key, value, ...]`
/// array (the host flattens its key/value map). Only takes effect with an
/// OTLP endpoint set, before the first send. A trailing unpaired element on
Expand All @@ -386,7 +416,7 @@ impl WasmSpanState {
.chunks_exact(2)
.map(|pair| (pair[0].clone(), pair[1].clone()))
.collect();
*self.otlp_headers.borrow_mut() = headers;
self.otlp_headers.set(headers);
}

#[wasm_bindgen]
Expand Down Expand Up @@ -520,16 +550,19 @@ impl WasmSpanState {
// When an OTLP endpoint is configured, libdatadog exports traces via
// OTLP HTTP to that endpoint instead of the Datadog agent (mutually
// exclusive with the agent v0.4/v0.5 path).
if let Some(url) = self.otlp_endpoint.borrow().as_deref() {
builder.set_otlp_endpoint(url);
if let Some(protocol) = self.otlp_protocol.get() {
if let Some(url) = self.otlp_endpoint.take() {
builder.set_otlp_endpoint(&url);
if let Some(protocol) = self.otlp_protocol.take() {
builder.set_otlp_protocol(protocol);
}
let headers = self.otlp_headers.borrow();
let headers = self.otlp_headers.take();
if !headers.is_empty() {
builder.set_otlp_headers(headers.clone());
builder.set_otlp_headers(headers);
}
}
if let Some(cfg) = self.telemetry_config.take() {

@yannham yannham Jul 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it ok to take it instead of clone? Do we only use the config exactly once? It looks like the next time send_prepared_chunk will be called, telemetry_config will be gone.

@Aaalibaba42 Aaalibaba42 Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exporter_slot goes from None to Some, so the entire build block is skipped on every subsequent send_prepared_chunk (l529), if the build fails, build_error is latched and the next call returns early before reaching the take (l534), so yes it should be okay by line 563.

@yannham yannham Jul 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I think I was confused by the fact that other values are cloned (headers, otlp endpoint) above instead of being taken out. Is that because we'll need them later?

@Aaalibaba42 Aaalibaba42 Jul 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a quick glance, build_error could be a Cell, but the others genuinely borrows from the content. If you think having all RefCell would help readability I don't mind too much, the impact of the borrow counting is probably minimal, otherwise I can make build_error also a Cell

@yannham yannham Jul 27, 2026 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but the others genuinely borrows from the content.

But then we clone them, right? self.otlp_protocol.get() clones, and otlp_headers is also cloned L560. So I'm just wondering which possibility is it:

  1. We use all those values otlp_protocol, otlp_headers, telemetry_config exactly only once for the builder. In this case it's strange that they have a different treatment than telemetry_config. We should rather use self.otlp_protocol.take() and std::mem::take(self.otlp_headers.borrow_mut()). Maybe it doesn't really matter for OtlpProtcol which is stack-allocated anyway, but the headers are a vec, so it does avoid a clone. And in that situation in fact the otlp_headers should probably not even be a RefCell, but a Cell? Unless it's built in several steps?

  2. Some of these values are re-used elsewhere afterwards, so we need to let them live in self. Then the current version makes sense, and this was basically my question above

    Is that because we'll need them later?

    It's just not obvious that e.g. headers and telemetry_config should be treated differently

@Aaalibaba42 Aaalibaba42 Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah you are right it's situation 1, otlp_protocol is Copy and otlp_headers is cloned, they should also be Cells for I don't think they are reused elsewhere

builder.enable_telemetry(cfg);
}
match builder.build_async::<WasmCapabilities>().await {
Ok(built) => *exporter_slot = Some(built),
Err(e) => {
Expand Down
Loading
Loading