Architectural Blueprint for Large-Scale System Design

Building software systems capable of scaling reliably to millions of active users requires understanding core trade-offs. This masterclass covers the foundational principles, caching patterns, database partitioning strategies, and message queue patterns used by top tech engineering teams.

1. Fundamental Architectural Theorems

The CAP Theorem

In any distributed data store, you can only guarantee two out of three properties simultaneously during a network partition:

PACELC Theorem Extension

PACELC extends CAP to address system behavior when there are no network partitions:

If there is a Partition (P), trade off Availability (A) vs Consistency (C); Else (E), trade off Latency (L) vs Consistency (C).

2. Caching Strategies & Preventing Cache Failure Modes

Caching stores pre-computed results or hot database entities in high-speed RAM (e.g., Redis or Memcached).

Code Implementation: Cache-Aside with Redis

import { createClient } from 'redis';

const redis = createClient();
await redis.connect();

export async function getCachedUserProfile(userId: string) {
  const cacheKey = `user:profile:${userId}`;
  
  // 1. Inspect Redis cache
  const cachedData = await redis.get(cacheKey);
  if (cachedData) {
    return JSON.parse(cachedData);
  }

  // 2. Fetch from primary database on Cache Miss
  const user = await db.user.findUnique({ where: { id: userId } });

  // 3. Populate Cache with TTL (Time To Live = 3600 seconds)
  if (user) {
    await redis.setEx(cacheKey, 3600, JSON.stringify(user));
  }

  return user;
}

Solving Deadly Caching Failure Modes

1. Cache Stampede (Thundering Herd):

   • Problem: A hot cache key expires, causing 10,000 concurrent requests to hit the database at the exact same millisecond.
   • Solution: Use Distributed Locks (Redlock) or Probabilistic Early Expiration (XFetch algorithm).

2. Cache Penetration:

   • Problem: Requests for non-existent IDs (e.g. user_999999) bypass the cache and repeatedly hit the database.
   • Solution: Cache null values with short TTLs or use a Bloom Filter upfront.

3. Cache Avalanche:

   • Problem: Thousands of keys expire at the exact same second.
   • Solution: Add a Random Jitter (+/- 1-5 minutes) to all TTL values.

3. Database Scaling: Replication vs Sharding

4. Message Queues & Event-Driven Architecture

Decouple synchronous HTTP request-response cycles by publishing heavy tasks to background worker queues.

import { Queue } from 'bullmq';

const videoQueue = new Queue('video-processing', {
  connection: { host: '127.0.0.1', port: 6379 }
});

export async function handleUploadRoute(req, res) {
  const { videoId, fileUrl } = req.body;

  // Enqueue job asynchronously
  await videoQueue.add('transcode', { videoId, fileUrl });

  // Return immediate 202 Accepted response to client
  return res.status(202).json({
    message: 'Video processing started',
    statusUrl: `/api/videos/${videoId}/status`
  });
}

import { Worker } from 'bullmq';

const worker = new Worker('video-processing', async (job) => {
  console.log(`Processing video ${job.data.videoId}...`);
  await transcodeVideo(job.data.fileUrl);
  console.log(`Finished transcoding video ${job.data.videoId}`);
}, {
  connection: { host: '127.0.0.1', port: 6379 },
  concurrency: 5 // Process 5 jobs concurrently per worker instance
});