Megatrend · Cloud & Digital Infrastructure

When software gets too complex to watch with your own eyes, who's the one seeing everything for you?

Every time a banking app freezes, a shopping site crashes mid-sale, or ChatGPT is slow to reply — behind the scenes there's a group of engineers who have to know instantly where it broke, inside a system made of thousands of parts all talking to each other. The tools that let them see a system too complex for any human brain to follow are called Observability. The tools that let them ship new code fast and safely are called DevOps — together, they're the engine room that keeps the entire software world running. And, without most people noticing, they've become a business that makes an enormous amount of money.

Category Cloud & Digital Infrastructure Level Sub-theme Layer platform Read time ~14 min
A calm control room with a wall of displays watching over a complex software city, with data routes flowing in and converging on a single screen
ภาพประกอบ (hero.png)
The control room of the digital world. When a single system has thousands of parts talking to each other, the most valuable thing is "seeing what's happening right now."

01What is it?

Think back to the old days. A website was one chunk of program running on one server. If it broke, an engineer just opened the "logs" on that machine and found the cause. But the apps you use today — Netflix, your banking app, a shopping site — aren't one chunk anymore. They're sliced into hundreds or thousands of "small services" (microservices), each running somewhere different on the cloud, constantly talking back and forth.

The problem: once a system gets this complex, when something slows down or breaks, no one can open each machine one by one anymore — it's like trying to figure out why a whole city is gridlocked by standing at one intersection at a time. This is the problem Observability (observing the system) was born to solve.

This node brings together two families of tools that work as a pair in a software engineer's life:

  • Observability (seeing how the system is doing): platforms that suck in data from every part of a system into one place, show you "how healthy the system is right now," and alert you the instant something goes wrong — including tools called APM (app performance monitoring), log management, and tracing
  • DevOps / CI-CD (shipping new code fast and safely): tools that let a team write code → test it → put it into production automatically, many times a day, without breaking things on release
Key terms
Observability vs Monitoring

Monitoring = deciding in advance which values to watch, like "alert me if CPU goes over 90%" — it can only answer questions you thought of beforehand · Observability goes a step further: it collects data detailed enough that you can ask new questions you never thought of, like "why do only customers on an iPhone in Thailand fail to pay at 3pm?" — without adding any code. That's what you need when a system is too complex to predict its problems in advance.

On the megatrend map, this node is a sub-theme of Cloud & Digital Infrastructure, sitting in the "platform layer" — not the hardware, not the cloud itself, but the software that sits on top of the cloud to watch over everything running on it.

02Why it matters — software's control room

The reason is simple: every minute a system is down = money disappearing. A shopping site down for an hour on a big sale day can mean tens of millions in lost sales. A stock-trading app freezing during a volatile market can mean a lawsuit. The more every business moves onto software, the more "the system must not go down — and if it does, it must be fixed fast" becomes a matter of life and death.

A tiny engineer standing bewildered in front of a giant black box, its insides full of tangled mechanisms, with no way to see what's happening inside
ภาพประกอบ (blackbox.png)
A system too big to understand by eye. The complexity of the modern cloud is exactly why observability went from a luxury to a necessity.

As a result, this group of tools has become a big — and fast-growing — market. The observability market alone is around $3B in 2025 and is expected to grow to roughly $6B by 2030 (about 12–16% a year, depending on the research firm). The broader DevOps/CI-CD market sits at roughly $16–20B in 2025, and many firms see it hitting $44–80B early next decade, growing around 20% a year.

Market size: observability vs DevOps
2025 market value (in billions of dollars) — DevOps/CI-CD is a far broader market
Source: Mordor Intelligence, Grand View Research (observability ~$2.7–2.9B), IMARC/Fortune Business Insights (DevOps ~$16–20B) — midpoint across several firms

But the market-size number is smaller than the value the stock market actually assigns, because the leaders here are seen as "infrastructure that's hard to rip out." Datadog, the observability leader, had full-year 2025 revenue of about $3.39B (up ~26%) — yet its company value runs into the tens of billions, because once a company "plugs" a tool into its own system, switching to someone else is very painful.

Plug it in, can't pull it out Once Datadog is embedded in every part of a customer's system, the rate at which existing customers keep paying more each year (net revenue retention) sits around 120% — meaning old customers keep spending more every year, without the company needing to find new ones.

03How it works (the three pillars)

At the heart of observability is an idea called the "three pillars" — three kinds of data that every part of a system emits, which a platform assembles into one big picture. Think of the system as a patient a doctor has to diagnose; the three pillars are three kinds of tests:

  • Logs — answer "what happened": messages each part writes down when something happens, like "at 14:03, this user failed to pay because their card expired" — like a very detailed diary of the system
  • Metrics — answer "how healthy is the system": summary numbers measured at intervals, like requests per second, average response time, % memory used — like vital signs (pulse, blood pressure) that tell you at a glance whether things are normal
  • Traces — answer "which way did this request go": when you press "pay" once, that request travels through maybe 20–30 small services; a trace follows the whole path — where it stopped and where it got slow — like GPS tracking a parcel through each checkpoint

One pillar alone isn't enough. The real power comes from combining all three: metrics alert you there's a problem → traces point to which small service has it → logs tell you the details of why it broke — all within minutes, instead of guessing all night.

The three pillars of observability Logs, metrics, and traces from multiple small services flow into an observability platform that shows the system's health and alerts when there's a problem Hundreds of small services (microservices) Logs "What happened" Metrics "How healthy" Traces "Which way did it go" Platform Observability System health + alerts
Three pillars flowing into one picture. Logs + metrics + traces from every part get sucked into a platform that shows the system's health and alerts you when something breaks (the colored dots are warning signals).
Key terms
OpenTelemetry (OTel)

The old problem: every vendor sold tools that collected data in their own "language," so switching vendors meant tearing everything out. OpenTelemetry is an open standard that defines how to collect logs/metrics/traces in one common format — like a "standard power plug" that lets data from your system plug into any platform. It reduces lock-in to a single vendor, and it's becoming the standard everyone has to support.

04Business model: pay by volume & "bill shock"

This is the part that makes this business both "very profitable" and "the most complained about" at the same time. Most of them charge by "how much data you use" (usage / data-volume pricing) — the more logs and metrics your system sends in, the more your bill balloons.

Datadog, for example, charges across dozens of stacked line items: server monitoring at about $15 per host per month, APM at about $31 per host per month, log ingestion at about $0.10 per gigabyte, plus another $1.70 per million events to make them searchable. It sounds cheap — until the system grows, data volume spikes, and the bill explodes.

A giant water meter whose numbers spin so fast they blur, as a flood of data flows through the pipe and a user watches anxiously
ภาพประกอบ (meter.png)
A meter that never stops spinning. The more data a system sends in, the bigger the bill — until "Datadog bill shock" became an in-joke in the industry.

The term "bill shock" has become an industry legend — engineering teams tell of bills 3x, 5x, sometimes 10x higher than they budgeted. Not because they used it wrong, but because the costs hide in many layers (e.g. every different "tag" counts as a whole new metric, so lots of tags blow it up). A mid-size company with 50 engineers might pay around $220,000 a year; an enterprise sending 1 terabyte of data a day might pay over $1 million a year for observability alone — the equivalent of 5–8 senior engineers' salaries.

Observability costs climb with the size of the org
Approximate annual cost (dollars) — grows far faster than team size, because it's priced by data volume
Source: estimates from analyzing Datadog's public pricing (OneUptime, Sentrial, Finout 2026) — real figures vary by usage

From the seller's side, this model is magic: revenue grows "naturally" every year, because every customer sends more data over time — no need to go sell anything extra (this is where net revenue retention ~120% comes from). But from the customer's side, it creates a love-hate relationship — you can't live without it, yet it stings every time you open the bill. So you get whole FinOps teams whose job is specifically "managing observability spend," and new challengers pitching "cheaper / keep your data longer for the same price."

05How it connects in the ecosystem

Observability & DevOps is the "layer that sees everything" in the cloud world, so it's tightly tied to other trends:

  • Sits on top of Hyperscale Cloud: these tools watch everything running on AWS, Azure, Google Cloud — the more the world moves to the cloud, the more there is to "watch" (though the cloud providers themselves also offer their own observability to compete)
  • Overlaps with Data Platforms & Analytics: because the core of observability is "storing and searching huge amounts of data fast" — Elastic started as a search tool before becoming an observability platform. The line between these two worlds keeps getting thinner
  • Converges with Cybersecurity & Digital Trust: the same logs data can be used to find "where the system is slow" and to find "whether anyone broke in" — this is why Cisco paid up to buy Splunk, and why observability and security are slowly merging
  • Feeds and is fed by AI Tooling & MLOps: AI models running in production are also systems that need "watching" — a new field called LLM observability has emerged to catch when AI answers wrong, answers slowly, or runs up costs

Put another way: every digital trend (AI, digital finance, smart city), once it builds software, always needs observability watching behind the scenes — so it's a "tax on complexity" for the whole industry. The more, and the more complex, the software the world builds, the more this market grows.

06Where it stands now + the players

The 2025–2026 picture has two big currents shaping this market.

The first current is consolidation into a "single platform". Engineering teams used to bolt together 5–10 tools; now everyone wants to cut down to one platform that handles logs/metrics/traces (and sometimes security too). The deal that shook the industry most was Cisco buying Splunk for $28B (closed March 2024, the largest deal in Cisco's history), adding about $4B of ARR to Cisco right away and folding the AppDynamics team under Splunk.

The second current is AI arriving both as new work to watch and as a tool inside the platform itself. Datadog revealed that "AI-native" customers (companies whose entire business is AI) made up about 8 percentage points of revenue growth in Q3 2025 — a sign the AI wave is creating an enormous amount of new things to watch.

Revenue/ARR of market players (approximate)
Full-year revenue or ARR (in billions of dollars) — 2025 · Grafana is a private company (ARR is an estimate)
Source: company 2025 earnings (Datadog FY2025, Dynatrace Q3 FY2026, Elastic), Cisco (Splunk ARR ~$4B), Sacra (Grafana ~$400M ARR)

The real leader of observability is still Datadog — a company that started as a server-monitoring tool and gradually grew into a platform that does almost everything. On the DevOps/CI-CD side there's a different crowd, led by GitLab and Atlassian, focused on writing-testing-shipping code. What's interesting is that the biggest names by stock value in this group are usually giants who do observability as a "side business" — Microsoft, Cisco, Broadcom — while the companies whose whole life is this are the smaller, more focused ones.

Key players in this field
Note
We rank players by role and market share, not raw market cap — to show who really leads each corner · Educational information, not investment advice
DatadogDDOG · US
US · observability leader
The most complete system-monitoring platform. 2025 revenue about $3.39B (+26%) · over 4,000 customers paying ≥$100k a year · NRR ~120% · pushing into AI with Bits AI agents and LLM Observability
core · market leader
Splunk (Cisco)CSCO · US
US · the giant that got absorbed
The original kingpin of log/security analytics, bought by Cisco for $28B in 2024 (ARR ~$4B) — a symbol of observability fusing with network and security
secondary · platform
DynatraceDT · US
US/Austria · enterprise-focused
An observability platform built around "automatic root-cause analysis with AI" (the Davis engine). ARR about $1.97B (end of 2025, +20%) — strong in the large-enterprise market
core · AIOps
ElasticESTC · US
US/Netherlands · search→observability
Started as a data-search tool (Elasticsearch), then expanded into an observability + security platform. ARR about $1.3B — reflecting the thinning line between data and observability
core · open-ish stack
GitLabGTLB · US
US · the DevOps/CI-CD side
An all-in-one platform for writing-testing-shipping code (SCM + CI/CD) — embedding AI (GitLab Duo) to help write and review code. The representative of this node's other half: the "ship fast" side
core · CI/CD
Grafana Labsprivate · US
US · the open-source challenger
The de facto standard for "dashboards" in the industry. The pitch: open-source + better value than the big players. ARR past ~$400M, last raised at about $9B — a dark horse many are waiting to go public
core · open-source challenger

07The road ahead: AI is both the burden and the tool

This node's future is tied to AI like two sides of one coin — and it's the most interesting story in the market.

An AI robot writing code at a desk while, on the other side, an eye keeps watch over a system — both the worker and the thing being watched
ภาพประกอบ (aiwatch.png)
AI writes code, and watches code. The same tools have to watch the AI that's running and use AI to help with the watching — both at the same time.

Side one — AI is new work to watch: once every company starts putting AI into its products, a flood of new things appear to keep an eye on — is the model answering correctly, is it slow, are the call costs (tokens) ballooning, is it "hallucinating"? This is a new field called LLM observability that everyone is racing to build, and one reason AI-native customers are visibly driving Datadog's growth.

Side two — AI is a tool inside the platform: observability used to only tell you "there's a problem" and leave humans to investigate. The new direction is to embed an "AI agent" that investigates the cause and fixes it itself — like Datadog's Bits AI or Dynatrace's Davis engine, which claim to pinpoint causes automatically. The endpoint is a "self-healing" system, with humans only watching the big picture.

A third direction is pressure for even more consolidation. Customers are tired of bill shock and tired of bolting together ten tools. That pressure will drive both more merger deals and the rise of open standards like OpenTelemetry that make switching easier — which is a double-edged sword for incumbent leaders who relied on "locking customers in" as their moat.

08Challenges & risks

This business's appeal (revenue that grows on its own with data volume + customers who are hard to dislodge) comes packaged with risks baked into the model itself.

The first risk is "bill shock" biting the owner back. The usage-based model makes revenue look great when the economy is good. But when companies tighten their belts, the first thing FinOps teams go after is "cut observability spend" — send fewer logs, keep data for less time, or move to a cheaper vendor. Being "ugly-expensive" is both a money-printer and a weakness that opens the door to cheap rivals like Grafana.

The second risk is open standards weakening lock-in power. As long as data is locked in each vendor's proprietary format, switching is expensive and painful. But OpenTelemetry is making observability data "portable" — once switching is easy, competition shifts back to price and quality rather than "being stuck," which pressures leaders' margins over the long run.

The third risk is the cloud and AI giants swallowing the market. AWS, Azure, Google each have their own "good enough" observability, already bundled with the cloud. Meanwhile the AI-agent wave could redefine this whole job — if one day AI can investigate and fix bugs almost entirely on its own, value might shift from "dashboards for people to look at" to "the smartest agent." Whoever adapts slowly risks being overtaken.

The bottom line for beginners Observability & DevOps is the invisible-but-essential "engine room" of the software world — three keys: (1) it collects a "tax on complexity" (the more software the world builds, the more there is to watch) · (2) the usage-based model is both a strength (revenue grows on its own) and a weakness (bill shock, opening the door to cheap rivals) · (3) AI is both new work to watch and the tool that will write and monitor code in place of people — whoever controls both sides first is the winner of the next round.

In short: every time your app "hasn't gone down" or "recovers fast after going down," there's a quiet industry behind the scenes seeing everything for the engineering team. It's not a flashy trend like AI or robots, but it's a "tax every digital company has to pay" — and as long as the world keeps building ever-more-complex software, the people selling the glasses to see it will have work for a long time to come.

Explore this theme — live data, stocks & news →