Case Study 05 — Datadog Platform
Enter password
Carlos Diaz
Senior Product Designer
Work About
Case Study 05 — Datadog Platform

Natural Language
Queries

Redesigning how users interact with Datadog's query language — letting anyone describe what they need in plain language and receive a structured, editable, executable query.

335Beta users
43Organizations
6+Data sources
GAShipped 2025
Project overview video
01Context

Datadog's query language is powerful — Lucene for logs, a proprietary syntax for metrics, DDSQL for analytics. But for a growing number of users, that power comes at a cost.

New developers, business users, product managers, and support teams increasingly need to query observability data without mastering Datadog's syntax. Each product has its own query structure — Logs, RUM, Traces, Metrics, CCM — and knowledge doesn't transfer between them.

When a query returns unexpected results, users can't tell if the syntax is wrong, the field name is off, or the time range is bad. The result: trial-and-error, reliance on technical teammates, and eroding confidence in a tool they're supposed to use independently.

NLQ changes this. Users describe what they need in plain language — "Show me errors for the payment service in the last 24 hours" — and receive a structured, editable query they can run, refine, and trust.

RoleDesign Lead
TimelineQ4 2024 – Q2 2025
Shipped toLogs Explorer (expanding to RUM, Traces, Metrics)
TeamsNLQ, Search, Logs, AI Platform
Add image
02The Problem

NLQ existed in private beta across 40+ organizations — but 12% retention meant most users tried it once and stopped. Three blockers were clear.

12%

Beta retention

Most users tried NLQ and didn't come back. Not because the concept failed — because the entry point was buried inside the search bar as a secondary option. Discovery was the first failure.

1.93s

Average translation latency

In a search-first tool where users expect instant results, two seconds feels broken. Latency at that threshold doesn't just slow people down — it destroys trust in the AI's output before it's even evaluated.

0

Ways to flag a bad translation

Bad queries stayed bad. No feedback mechanism meant users couldn't rate, correct, or improve what the model returned. Every wrong translation was a dead end, not a learning signal.

Add image
03Process

Four months. Research-first, Logs-first — with a shared component framework built to scale to every Datadog explorer from the start.

Phase 1

Research + Benchmarking

Ran user interviews across new developers, business users, and power users. Benchmarked against Thoughtspot, Splunk NL, and Bits AI internal patterns. Three blockers emerged consistently: users couldn't find the feature, latency killed trust before results appeared, and there was no way to flag wrong output. Every design decision mapped back to these three findings.

Phase 2

Design Exploration

Explored multiple directions: a focus-mode overlay (discarded — redundant with Explorer), a heavily AI-branded experience (discarded — too invasive for frequent use), and the final approach: seamless in-Explorer integration where NLQ lives inside the existing query layout. Built Figma prototypes and interactive demos for stakeholder review.

Phase 3

Design Review + Ship

Ran a formal design review with PM, Engineering, and cross-functional stakeholders. Landed on the "Eager Translation + Submit" model over eager execution. Shipped the recommended version to Logs Explorer with feedback loop, expanding to RUM and Traces next.

Add image
04Solution

NLQ v2.0 — not a separate mode, but a layer woven into the Explorer. Five design decisions that make natural language feel native.

Discoverable Entry Point

An "Ask" CTA embedded inside the search bar — not next to it, not above it, inside it. This ensures consistency across every Datadog product that adopts NLQ. A tooltip explains the feature on hover. Power users can hide it via the Search Bar Configuration Hub. We also explored a "Space" shortcut to trigger NLQ from the keyboard, aligning with Datadog's broader AI shortcut strategy.

Entry point

Eager Translation

As the user types in natural language, the query editor below shows a real-time translation into Datadog syntax. The timeframe selection updates simultaneously. Users see exactly what the system understood before committing — no black box, no guessing. The NLQ component enters a highlighted state with a visible Submit button, making it clear that Enter or click is required to execute.

Real-time translation

Three Distinct States

The NLQ flow has three clear phases: Intro (type your prompt, see dynamic placeholder suggestions based on your org's data), Eager Translation (query preview generated, Submit enabled, refinement still possible), and Outcome + Feedback (query executed in Explorer, results visible, feedback prompt appears). Each state has distinct visual treatment so users always know where they are.

Three states

Seamless Explorer Integration

NLQ doesn't take you somewhere else — it opens directly inside the Explorer. Users stay in their workflow. After submission, they can modify the generated query in the standard query editor, adjust visualization settings, tweak facets — exactly like a manually written query. The transition between NLQ and traditional editing is invisible.

In-Explorer flow

Feedback Loop

After every query execution, a feedback prompt appears in the bottom-right of the NLQ component. Users rate the translation quality, and corrections feed back into the model. Aggregated feedback trains the translation engine on real-world usage patterns — the system gets smarter with every query. This was the missing piece that kept retention at 12%.

Feedback system
05Outcomes

12% retention in private beta. After v2.0 shipped: GA in 2025, expanding to RUM, Traces, CCM, and Metrics — with NLQ as the query translation layer behind every Bits AI workflow.

335

Beta Users

Active across 43 organizations in private beta. 66% of usage concentrated in Logs — validating the launch surface choice.

v2.0

Redesigned Experience

From buried entry point to embedded CTA. From 1.93s latency perception to eager real-time translation. From zero feedback to structured rating system.

6+

Data Sources

Logs, RUM, CCM, REDAPL already supported. Traces and Metrics in the pipeline. Framework designed to scale to every explorer search bar.

Bits AI

AI Foundation

NLQ is the query translation layer for Bits AI agents, Cmd+K natural language, and autonomous workflows. Every improvement to NLQ compounds across the platform.

GA

General Availability

Shipped in 2025. Expanded from private beta to full rollout, with pricing aligned to Datadog's AI product strategy and expansion across the platform.

What's Next

Complex contextual queries ("errors for my team"), RBAC-aware filtering, cold-start handling for new services, and DDSQL support for REDAPL and Workspaces.

Add image
← Content Packs Back toAll Projects →