webserv: Systems Architecture & Codebase Guide

What is webserv?

webserv is an HTTP/1.1 compliant web server written from scratch in C++98. The objective of this project is to explore and implement how real-world industrial servers like NGINX, Apache, and Envoy function beneath the surface.

Core Engineering Constraints

No external libraries: No Boost, libevent, libuv, or modern C++11/17/20 STL features.
Single-threaded: Zero POSIX threads (no multi-threaded context switching).
Single poll() reactor: One central event loop drives all listening sockets, client streams, and CGI pipes simultaneously.
Non-blocking I/O: Every descriptor must use O_NONBLOCK. Never block the event loop on slow clients or network latency.

The C10K Challenge & The Reactor Pattern

In early web servers (like the original Apache prefork model), each incoming client connection was dispatched to a separate thread or spawned process. If 10,000 clients connected simultaneously, the operating system kernel wasted excessive CPU time switching thread contexts, allocating massive stack memory per thread, and exhausting descriptor tables. This is famously known as the C10K problem.

Thread-Per-Client (Anti-pattern)

A thread blocks on read() until the client sends data. If a slow client sends 1 byte every 10 seconds (Slowloris pattern), the thread remains frozen, consuming stack memory and blocking server capacity.

Single-Loop Non-Blocking Reactor (webserv)

The server sets all sockets to non-blocking with fcntl(). A single call to poll() sleeps until the operating system notifies that at least one file descriptor is ready for reading or writing. No threads, zero context switching overhead, virtually zero idle CPU usage.

The Reactor Multiplexing Loop src/net/Reactor.cpp

+-------------------------------------------------------------------------+
|                           THE REACTOR (poll)                            |
+-------------------------------------------------------------------------+
       |                                |                            |
       v                                v                            v
+--------------+               +------------------+         +---------------+
|  Listeners   |               | Client Sockets   |         |   CGI Pipes   |
| (port 8080,  |               | (HTTP keep-alive |         | (script stdin |
|  port 8081)  |               |  request/resp)   |         |  child stdout)|
+--------------+               +------------------+         +---------------+
       |                                |                            |
       | [POLLIN: new connection]       | [POLLIN / POLLOUT]         | [POLLIN / POLLOUT]
       v                                v                            v
Listener::handleRead()        Connection::handleRead()     CgiPipeRead/Write Handler
accept() -> new Connection    Parser -> Route -> Write     Feed stdin / read stdout

System Architecture & Six-Layer Decomposition

Clean architecture ensures strict separation of concerns and unidirectional data dependencies across six modular layers:

Layer Responsibilities

Core Layer: Bootstrap lifecycle, configuration parsing, signal dispatching.
Net Layer: Sockets, listeners, non-blocking I/O multiplexing via Reactor and PollRegistry.
Connection Layer: Client connection state machines, I/O buffering, keep-alive timers.
HTTP Layer: RFC 7230/7231 streaming request parsing, chunked decoding, response serialization.
Routing & Handlers: Virtual host dispatching, location resolution, static file delivery, uploads, deletion.
CGI Subsystem: RFC 3875 process spawning, asynchronous pipe IPC, timeout supervision.

The poll() Reactor Event Loop

The heart of the server is a single, centralized poll() invocation that monitors all registered descriptors with zero busy-waiting.

Reactor Execution Loop

void Reactor::run() {
    while (_running) {
        std::vector<struct pollfd> pollfds = _registry.buildPollFds();
        int ready = poll(&pollfds[0], pollfds.size(), POLL_TIMEOUT_MS);
        if (ready < 0) {
            if (errno == EINTR) continue;
            break;
        }
        for (size_t i = 0; i < pollfds.size(); ++i) {
            if (pollfds[i].revents == 0) continue;
            IEventHandler* handler = _registry.getHandler(pollfds[i].fd);
            if (!handler) continue;
            if (pollfds[i].revents & (POLLERR | POLLHUP | POLLNVAL)) {
                handler->handleError();
            } else {
                if (pollfds[i].revents & POLLIN) handler->handleRead();
                if (pollfds[i].revents & POLLOUT) handler->handleWrite();
            }
        }
        sweepTimeouts();
    }
}

Connection State Machine

Each client socket transitions deterministically through well-defined lifecycle states: STATE_READ_REQ_LINE → STATE_READ_HEADERS → STATE_READ_BODY → STATE_PROCESS → STATE_SENDING_RESP → STATE_LINGERING_CLOSE.

RFC 7230/7231 Streaming HTTP Parser

HTTP messages arrive across arbitrary TCP segment boundaries. The parser operates incrementally without buffering unbounded payloads in memory, streaming chunked or Content-Length bodies directly to disk or downstream pipes.

Chunked Transfer Decoding

Compliant with RFC 7230 §4.1: chunk headers, hex chunk-size parsing, CRLF verification, and trailer section termination.

Virtual Hosts, Routing & Handlers

Virtual servers are matched by the incoming Host header. Routes match using longest-prefix matching with support for method restrictions (405), HTTP redirection (301/302), custom error dispatching, autoindex directory listing, and uploads.

Asynchronous CGI Pipeline

CGI child processes execute in isolated subprocesses connected via non-blocking pipes. The reactor pumps input and reads output asynchronously, terminating runaway scripts with SIGKILL upon timeout.

Resilience: Slowloris & Lingering Close

Protection against connection exhaustion attacks: strict inactivity timeouts, FD_CLOEXEC on all socket handles, non-blocking pipe draining, and lingering close to prevent TCP RST resets.

Interactive Codebase Catalog

Explore module responsibilities, key classes, and design patterns across the source tree:

src/core/ Bootstrap & Lifecycle

Webserv (.hpp/.cpp): Central coordinator. Parses configuration, groups server blocks by unique (host, port) pairs, instantiates Listener objects, and starts the Reactor.
Signal (.hpp/.cpp): Installs global signal handlers using sigaction(). Flags atomic shutdown on SIGINT and ignores SIGPIPE so broken client sockets never crash the server.

Key Systems Architecture Decisions & Technical Q&A

Critical architectural rationales, edge cases, and design choices implemented across the webserv engine:

Q1: Why only one poll() loop? Where is poll() called in the architecture?

Architecture Rationale: The design mandates that all I/O operations (listening sockets, client TCP streams, and CGI pipes) must be multiplexed through a single central poll() event loop without secondary polling threads.

In src/net/Reactor.cpp, the single multiplexer is invoked:

int ready = poll(&pollfds[0], pollfds.size(), POLL_TIMEOUT_MS);

Every descriptor in the process lives inside PollRegistry. We never invoke poll() in any secondary function, thread, or CGI worker.

Q2: How do you guarantee that a client socket never blocks the reactor?

Implementation: Every socket accepted by accept() has fcntl(fd, F_SETFL, O_NONBLOCK) called on it immediately inside Connection::Connection (via Socket::setNonBlocking).

Furthermore, recv() and send() are only ever invoked when the Reactor confirms that poll() returned POLLIN or POLLOUT for that exact descriptor. If a client stalls, poll() simply does not flag readiness, allowing all other clients to be served without interruption.

Q3: What prevents CGI scripts from blocking the server or leaking zombie processes?

Safeguards in src/cgi/CgiProcess.cpp:

Non-blocking Pipes: Parent pipe ends are set to non-blocking and registered directly into the central PollRegistry. The server feeds stdin and reads stdout asynchronously via the main reactor loop.
Process Reaping: The parent never runs blocking wait(). It uses waitpid(_pid, &status, WNOHANG). If the child has not terminated, the reactor continues running.
Timeout and Kill: If a script exceeds 10 seconds (e.g. an infinite loop), checkTimeout() triggers kill(_pid, SIGKILL) and returns 504 Gateway Timeout.

Q4: Why is FD_CLOEXEC essential on listening sockets?

Descriptor Isolation: When webserv calls fork() to execute a CGI script, by default all open file descriptors in the parent process are duplicated in the child process.

If a CGI script inherits our listening socket (e.g. port 8080) and runs for minutes, and the server is restarted during that time, the kernel returns EADDRINUSE (Address already in use) because the child script still holds the socket open. Setting FD_CLOEXEC ensures the operating system automatically closes all listening and client sockets upon execve().

Q5: How do you verify zero memory leaks and zero descriptor leaks?

Run the automated regression and stress suite under Valgrind descriptor tracking:

# Run with Valgrind leak and descriptor tracking
valgrind --leak-check=full --track-fds=yes ./webserv config/default.conf

# In another terminal, run the full integration and siege stress suite:
python3 tests/integration/test_regression.py --all
siege -b -c 25 -t 30S http://127.0.0.1:8080/

Under continuous heavy load, open descriptor count remains strictly constant and Valgrind reports zero leaks.

Under the Hood of webserv