Ditto: End-to-End Application Cloning for Networked Cloud Services

Researchers and chip vendors need real cloud services to design against, and cannot have them — production code is proprietary, and open-source stand-ins lack the complexity of a real deployment; Ditto (Cornell with Meta and Intel) profiles a running service and generates a completely different program that behaves like it, capturing not just the CPU behaviour that earlier cloning work targeted but the system calls, network model, thread model and RPC graph where cloud services actually spend their time — and because it reconstructs statistics rather than copying code or memory traces, the clone can be published.

Why this problem is worth solving

The gap here is not technical so much as institutional, and it distorts an entire research field.

What makes cloud services hard to clone is where their time goes.

What people did before, and where it broke

• Use open-source applications as stand-ins. The community leans heavily on published benchmarks and open-source services, and they are genuinely useful. Where it broke. They lack the complexity of production deployments and the update cadence — real cloud services, especially microservices, change daily, and a benchmark suite does not.
• Replay a production trace. Traces come from the real thing, so they carry real behaviour. Where it broke. A trace is tied to the system configuration it was collected on. You cannot generalise it to a study of different hardware, a different allocation, or a different load, which is usually the whole point of the study.
• Generate a synthetic CPU-centric clone. A line of prior work profiles instructions per cycle, cache miss rates and dependency distances, then emits assembly matching them. Where it broke. Cloud services spend a large fraction of their execution in the kernel handling interrupts, I/O and scheduling, and none of that is captured by user-level CPU counters. These systems also target microarchitectural metrics rather than the latency numbers operators care about, and they handle single-node applications rather than distributed graphs of services.
• Use a coarse instruction taxonomy. Prior clones group x86 into a handful of categories — integer arithmetic, floating point, loads, stores, control — and pick one representative instruction each. Where it broke. Modern instructions within a category differ enormously. The paper example is a checksum instruction that takes three cycles and can only issue on one port, against ordinary integer arithmetic taking one cycle on any of four; repeat-prefixed and locked instructions can take tens of cycles.
• Replay the memory access trace. The most faithful way to reproduce memory behaviour. Where it broke. It takes ten to twenty million memory references to represent a pattern accurately, which means millions of hard-coded memory instructions that themselves distort every other characteristic. And memory access patterns have been shown to leak confidential information, so replaying them defeats the purpose of cloning.
• The gap. Nobody had cloned an application across the whole stack — kernel, network, I/O, and a distributed service graph — and nobody had asked how to reproduce behaviour statistically rather than by copying, which is what makes a clone both compact and publishable.

How Ditto builds a clone

The idea

The representativeness problem is structural, and cloning is a way around it rather than through it. Nobody outside a large operator can study production cloud services, so the field studies substitutes and hopes. Ditto proposes that an operator profile their service and publish a program that behaves like it — useful to a chip vendor sizing a new platform, to an operator writing performance requirements into a purchase agreement, and to researchers who currently have nothing. What makes this credible rather than wishful is that the generated program is a real program: it runs on new hardware, responds to load changes, and degrades under interference the way the original does. The general difficulty that the behaviour worth studying is only visible from inside a production fleet is the same one confronted by the warehouse-scale memory allocator characterisation covered elsewhere on this site, which had to be written by insiders because nobody else could see the data.

Cloud services are defined by how they wait, so the clone has to reproduce the waiting. This is the substantive departure from prior cloning work, which measured instructions per cycle and cache misses and stopped. A key-value store and a database can have similar CPU profiles and completely different bottlenecks — one wants memory and network bandwidth under a tight latency target, the other is limited by disk. So Ditto starts above the CPU: the RPC graph from distributed tracing, then the network model and thread model recovered by watching kernel functions, then system calls profiled with their arguments rather than merely counted. Only after that does it generate assembly. The ordering matters, because the skeleton determines queueing and queueing determines tail latency, which is the number an operator actually manages. Reconstructing the shape of a distributed service from the tracing infrastructure it already runs is the same starting point as the trace-based root-cause analysis work for microservices covered elsewhere on this site.

Rebuilding statistics instead of copying is what makes the clone both small and safe. The natural way to reproduce memory behaviour is to replay the access trace, and the paper rejects it for two reasons that turn out to be the same reason. Faithful replay needs ten to twenty million references, which would mean millions of hard-coded memory instructions distorting every other characteristic. And memory access patterns leak confidential information about the service. So Ditto profiles something coarser and sufficient: how many accesses fall within a working set of each size, measured by simulating caches of doubling sizes. It then generates a synthetic pattern that walks sequentially through working sets of those sizes, which provably produces the same hit and miss counts at every cache level regardless of hierarchy or inclusion policy. The same reasoning is applied to instruction locality. Producing the useful signal while withholding the identifying detail is the same balance struck by the private information retrieval work covered elsewhere on this site, from the opposite side of the same tension.

Walking one memory access pattern through Ditto

The second objection is security, and it is the one that decides the design. Prior work has demonstrated that memory access patterns reveal confidential information about a service. A clone that faithfully replays them is not shareable, which removes the entire reason for building it.

Why it works

• Working sets carry the information a cache hierarchy responds to. A sequential walk through a bounded region hits when the cache is bigger than the region and misses when it is smaller → matching the distribution of accesses across region sizes reproduces hit and miss counts at every level → the reproduction holds for any number of levels and any inclusion policy, so one profile transfers across machines.
• Everything profiled is a property of the program, not the machine. Instruction mix, branch rates, working set sizes and dependency distances describe what the application does rather than how a particular chip responds → the clone can be moved to different hardware without reprofiling → and it reacts to the move the way the original does, which is the actual test.
• The skeleton is what generates queueing, and queueing is what generates tail latency. Reproducing whether threads block, poll or wait on many sockets at once, and what wakes them → makes the clone respond to load the way the original does → so latency percentiles track without the clone ever having been profiled at that load.
• System call arguments carry more than system call counts. Profiling read sizes, offsets and descriptor flags rather than just tallying calls → reproduces disk bandwidth, page cache miss behaviour and network bandwidth structurally → the paper measures both bandwidths matching to within 0.1%.
• Feedback repairs what decomposition breaks. Profiling each subsystem separately misses their interactions → running the clone and comparing counters catches the residual → about ten cheap iterations reach over 95% accuracy without compromising portability, since the underlying features stay platform-independent.

What it costs (trade-offs and scope)

• The paper names the studies it is wrong for, in its own section. Because Ditto abstracts away individual instructions and memory accesses, it is less accurate for microarchitecture studies that depend on the exact implementation rather than its statistical pattern. The example given is designing hardware prefetchers by studying memory access patterns — explicitly a bad fit. The authors state the general form as a fundamental trade-off between the granularity captured in one subsystem and overall accuracy.
• Confidentiality is argued, not guaranteed, and the paper says so. Ditto may expose the RPC graph and some hardware counter statistics. The defence is that many companies already publish exactly these — RPC traces, kernel cycle breakdowns, workload traces — and that the skeleton is drawn from a small set of well-known network and threading models. That is reasonable and it is not a proof.
• Accuracy is uneven across metrics. Average errors are 4.1% for instructions per cycle, 5.1% for level-one data cache misses, 6.9% for level two, 7.1% for level-one instruction cache, 9.9% for branch mispredictions and 12.1% for last-level cache misses. The last is the one most likely to matter for memory-system studies.
• Tail latency diverges exactly where it gets interesting. The 99th percentile matches at low and medium load and separates at high load, which the authors attribute to queueing in the network stack at saturation — the regime a latency-critical service operator most wants to study.
• Two of the six workloads use a closed-loop load generator. For those, latency does not rise much at high load because only one request per connection is outstanding, which makes the latency agreement less informative than it looks for those two.
• The scope is narrow in language, ISA and framework. C and C++ only, x86 only, two RPC frameworks. The authors argue nothing depends on these and adapting is a matter of finding compatible profiling tools. Accelerators — GPUs, FPGAs, smart network cards — are left to future work and are noted as increasingly important.
• Fine tuning happens on a specific machine. The calibration loop runs the clone on one platform and adjusts. The argument that this does not compromise portability rests on the underlying features being platform-independent and is supported by the cross-platform validation, but the tuning itself is not machine-agnostic.
• Profiling perturbs the service. The instrumentation and emulation tools impose overhead on the original application while it is being profiled. This is one-time and does not affect the features collected, but an operator has to be willing to run it against something representative.
• The evaluation is six applications and one microservice graph. The graph is an open-source research benchmark with a small dataset, one replica per service, and the paper validates two of its tiers in detail while stating the rest are similar.

The evaluation

Check yourself

Sources

The paper is “Ditto: End-to-End Application Cloning for Networked Cloud Services” by Mingyu Liang, Yu Gan, Yueying Li, Carlos Torres, Abhishek Dhanotia, Mahesh Ketkar and Christina Delimitrou, published at ASPLOS 2023 Volume 2, pages 222-236, DOI 10.1145/3575693.3575751. The authors are at Cornell University, Meta and Intel, with the first two contributing equally. Ditto is released as open-source software.

This explainer is written from the full text, read through a proxy of the ACM Digital Library copy. Everything above comes from the paper’s own sections 3 through 7: the survey of what determines cloud application performance across inputs, binary, containers, kernel, CPU-memory subsystem and hardware devices; the four design principles of end-to-end modelling, portability, abstraction and automation; the extraction of the RPC dependency graph from distributed tracing; the network model classification across blocking, non-blocking and multiplexed server designs and synchronous or asynchronous client designs, and the thread model recovered by clustering call graphs with tree-edit distance and agglomerative clustering; the system call profiling with arguments; the instruction mix clustered by functionality, operand type and execution unit usage rather than coarse category, with the checksum instruction example; the branch taken and transition rate reproduction via a precomputed bit mask; the working-set-based data and instruction memory synthesis with its derivation and its 1.9% associativity error, plus the corrections for prefetching, coherence misses and memory-level parallelism through pointer chasing; the dependency distance profiling quantised into eleven bins; the feedback-based fine tuning reaching over 95% accuracy in about ten iterations; the validation across six services, three loads and three platforms with average errors of 4.1%, 9.9%, 7.1%, 5.1%, 6.9%, 12.1%, 0.1% and 0.1%; the top-down cycle breakdown; the incremental accuracy decomposition on MongoDB; the interference and frequency-scaling case studies; and the discussion sections on unsuitable use cases, confidentiality, application phases, virtualisation and the C, C++ and x86 scope limitation.