Building Real-Time Trade Processing at Scale

When you’re processing ~3 million trades per day across multiple desks — Spot, Options, Swaps — the architecture decisions you make early on compound fast.

The Challenge

Every trade needs to flow through booking, position calculation, real-time PnL, and risk aggregation. The latency budget is tight: traders need to see their positions update in near real-time to make hedging decisions.

Key Design Decisions

Vectorized Processing — Using Pandas with NumPy for batch computations on trade data. When you need to calculate PnL across thousands of positions, vectorized operations are orders of magnitude faster than row-by-row iteration.

Multiprocessing for Parallelism — Python’s GIL makes threading ineffective for CPU-bound work. We use multiprocessing to parallelize position calculations across instrument types.

ClickHouse for Analytics — OLAP queries on trade history need to be fast. ClickHouse handles our analytical workloads with sub-second response times on billions of rows.

---

More thoughts coming soon. This is a placeholder post — the real content will explore specific architectural patterns and trade-offs in detail.