Introduction to System Design

System design is the process of defining the architecture, interfaces, and data for a system that satisfies specific requirements. When designing systems at scale, we must move away from single-server setups and address scalability, availability, and reliability.

These fundamentals matter more than ever, not less, in a world where an AI copilot sits beside you in every design session. Ask a modern coding assistant how to fix a latency problem and it will answer instantly and fluently: "add a cache," "add more replicas," "put a load balancer in front of it." Those answers are often right. They are also exactly the kind of plausible-sounding suggestion that can be subtly, confidently wrong for your system, because the copilot is pattern-matching on the shape of the question, not reasoning about your actual constraint.

That is the real skill this lesson builds. The mechanics below (scaling, load balancing, redundancy) are the legitimate, load-bearing fundamentals you need to know. But knowing them is table stakes. The judgment that separates a strong engineer is the ability to take a fluent suggestion, whether it comes from a senior teammate or an AI copilot, and interrogate the frame: What is the actual bottleneck? What does this suggestion assume? What does it cost, and what does it fail to address? As you read each fundamental below, notice how it doubles as a place an AI copilot can hand you a tidy answer that dodges the real problem.

1. Vertical vs. Horizontal Scaling

When traffic grows, we have two primary options for scaling our infrastructure:

• Vertical Scaling (Scale Up): Adding more power (CPU, RAM, disk space) to an existing server. While simple to implement, it has a hard hardware ceiling and does not solve the Single Point of Failure (SPOF) problem.
• Horizontal Scaling (Scale Out): Adding more servers to our pool of resources. This is highly scalable and fault-tolerant but requires distributed system management and database consistency controls.

Where the copilot gets confident: Ask an AI assistant "how do I handle more traffic?" and "just add more replicas" is a near-universal reply. Sometimes that is right. But if the bottleneck is a single write-heavy database or a lock contention issue, horizontal scaling of the app tier adds cost and complexity while doing nothing for the actual constraint. The interrogation move is to ask what specifically is saturating before accepting a scaling recommendation. A fluent answer that never establishes the bottleneck is a signal to slow down, not speed up.

2. Load Balancing

To distribute client traffic across multiple horizontal servers, we introduce a Load Balancer.

Load balancers distribute user requests among a pool of application servers using routing algorithms:

1. Round Robin: Sequentially routes requests (Server A -> Server B -> Server C).
2. Least Connections: Routes to the server handling the fewest active connections.
3. IP Hash: Hashes the client's IP to assign them to a specific server (useful for session stickiness).

Where the copilot gets confident: "Just add a load balancer" and "just add a cache" are the reflexive suggestions here, and they are seductive because they are usually part of the answer. But a load balancer in front of stateful servers that assume sticky sessions can break auth flows, and a cache in front of data with a low read-to-write ratio or strong consistency needs can introduce staleness bugs that are worse than the latency it was meant to fix. When a copilot offers one of these, the judgment move is to name the assumption it is making about your traffic and data before you wire it in.

3. Eliminating Single Points of Failure (SPOF)

A Single Point of Failure is any part of a system that, if it fails, will stop the entire system from working. We eliminate SPOFs by introducing Redundancy:

• Running multiple active servers behind a load balancer.
• Implementing Database Replication (Primary-Replica setups) to prevent data loss.
• Using failover systems to automatically promote a standby resource if the active resource crashes.

Where the copilot gets confident: Redundancy is the area where a plausible suggestion can quietly move the point of failure instead of removing it. "Add a standby and fail over automatically" sounds airtight, but automatic failover with replication lag can promote a replica that is missing the last few seconds of writes, and a health check that is too naive can flap between nodes under load. An AI copilot will happily generate the failover config without surfacing these trade-offs, because they live in the consequences, not the syntax. The interrogation move is to trace what happens after the failover fires: what data is at risk, what does the client see, and what new single point of failure did you just create?

4. Bringing It Back to AI-Native Judgment

Every fundamental above is real and worth knowing. But notice the pattern: in each case, the dangerous failure mode was not not knowing the concept, it was accepting a fluent, reasonable-sounding suggestion without interrogating whether it addressed the actual constraint. That is precisely the environment you now work in every day. Your copilot is fast, articulate, and usually helpful, which is exactly what makes its occasional confident wrong turns so easy to follow.

The engineers who thrive in an AI-native workflow are not the ones who memorized the most patterns, and they are not the ones who refuse to use AI. They are the ones who treat every suggestion, human or machine, as a proposal to be tested against the real system: interrogating the frame, surfacing hidden assumptions, tracing consequences, integrating the actual constraints, and communicating the trade-off clearly. The scaling, load-balancing, and redundancy fundamentals in this lesson are the vocabulary. The judgment to know when a tidy answer is dodging the real problem is the skill. The rest of this course is about building that skill deliberately.