Bryan Cantrill and colleagues at Sun created DTrace in 2003: a dynamic tracing framework that could instrument a live production system — kernel and userspace simultaneously — with zero overhead when probes weren’t firing, and minimal overhead when they were. You could ask a running system arbitrary questions about its own behaviour without rebooting, recompiling, or touching a configuration file. It was, by any measure, a significant engineering achievement. DTrace shipped in Solaris 10 in 2005, arrived in macOS in 2007, and reached FreeBSD shortly after. It did not reach Linux until the 2010s, partly for licensing reasons and partly because the Linux kernel community built its own alternatives — ftrace, perf, eBPF — rather than adopt Sun’s implementation. Cantrill, who has opinions about Linux kernel politics and expresses them with considerable vividness, has noted this. At length.
