At a builder processing 10,000 bundles per block, a 1% simulation failure rate doesn’t cost 1%. It costs 40% of margin.

Here’s why: The bundles that fail simulation are disproportionately the most valuable ones-complex arbitrages with tight timing windows. When your simulation times out on a 12-hop DEX route, you don’t just lose a bundle. You lose the only bundle that mattered.

This post documents the infrastructure failures that turn profitable MEV strategies into losses, and the observability stack required to find them.

1. The Physics of Block Propagation

Ethereum produces a block every 12 seconds. Builders compete to have their block included by proposers. The competition is decided by bid value-the amount the builder pays the proposer.

But bid value is a function of bundle quality, and bundle quality degrades rapidly:

The “Stale State” Problem:

If a builder simulates a bundle against state that is 200ms old, the bundle may reference a DEX pool that has already been arbitraged. The simulation “succeeds” but the execution reverts.

• Source: Flashbots MEV-Boost Architecture

2. The Decision Matrix: Simulation Infrastructure

The Math:

In a 12-second block time, a builder receiving bundles at t=11.5s has 500ms to simulate and bid. If simulation takes 50ms and you process sequentially, you handle 10 bundles. If it takes 5ms, you handle 100. The builder with better infra wins by volume.

3. The “Silent Killer”: Partial Fills and Revert Rates

Most builders track “bundles received” and “bundles included.” They rarely track:

1. Simulation Timeout Rate: Bundles that took >100ms to simulate and were dropped.
2. Execution Revert Rate: Bundles that simulated successfully but reverted on-chain.
3. Partial Fill Rate: Multi-leg bundles where only some transactions succeeded.

Why This Matters

A 5% execution revert rate on 100 ETH/day of bundle value is 5 ETH of lost margin. At $3,000/ETH, that's **$5.4M/year** from a single observability gap.

4. The Kill: The “Bundle Half-Life” Metric

I propose a new observability metric: Bundle Half-Life.

Definition: The median time from searcher submission to block inclusion (or drop).

This single metric captures:

• Network propagation latency (searcher → builder)
• Simulation queue depth
• Block propagation latency (builder → proposer)

Implementation

# Pseudocode for Bundle Half-Life tracking
bundle_events = []

def on_bundle_received(bundle_id, timestamp):
    bundle_events.append({"id": bundle_id, "received": timestamp})

def on_bundle_included(bundle_id, block_number, timestamp):
    event = find_event(bundle_id)
    event["included"] = timestamp
    event["half_life"] = event["included"] - event["received"]
    emit_metric("bundle_half_life_seconds", event["half_life"])

Dashboard Alert:

• P50 Half-Life > 1s: Warning. You’re losing to faster builders.
• P99 Half-Life > 5s: Critical. Your tail bundles are worthless.

5. The Tool: Observability Stack for Builders

The Golden Signals for MEV infrastructure are not the standard RED (Rate, Errors, Duration). They are:

# Example Prometheus queries
sum(rate(bundle_received_total[5m])) by (searcher)
histogram_quantile(0.99, rate(bundle_half_life_seconds_bucket[5m]))

6. Systems Thinking: The Trade-offs

1. Infra Cost vs. Margin: Running a custom state trie requires 10x the engineering of a Geth node. Is your bundle volume worth it?
2. Latency vs. Correctness: Faster simulation can mean less accurate gas estimation, leading to underpriced bundles.
3. Observability Overhead: Every metric you emit adds latency. Trace judiciously.

7. The Philosophy

In MEV, infrastructure is not a cost center. It is a profit function.

The difference between a profitable builder and a money-losing one is often not strategy-it’s the 50ms of simulation latency that causes a 5% revert rate. DevOps in this domain is not “keeping the lights on.” It is directly moving the P&L.

When your interviewer asks about your DevOps philosophy, this is the answer: Infrastructure is alpha.

Need Help With MEV Infrastructure?

Building block infrastructure or optimizing your MEV strategy? I help trading firms and protocols design low-latency, censorship-resistant systems. Let’s discuss your architecture →