Cracking The Coding Interview Together – Problem 7.7: Chat Server Design

At first glance, building a chat server feels like a straightforward application: Alice sends a text string, and the server hands it off to Bob. But as senior engineers, we know the real challenge begins when we transition from a simple prototype to a high-scale production platform. What happens when Bob is offline? What if Alice and Bob are connected to entirely different server instances in separate data centers? How do we track real-time online presence without overwhelming our database with heartbeat writes?

Designing a real-time chat server is one of the classic Object-Oriented Design (OOD) and System Design challenges. It tests our ability to construct clean domain models while keeping an eye on real-world constraints like low-latency bi-directional network sockets, concurrency, state management, and horizontal scalability. In this guide, we are going to tackle Problem 7.7 from Cracking the Coding Interview step-by-step, building a clean Java domain model and architecting a production-ready solution together.

Problem Statement

CTCI Problem 7.7: Explain how you would design a chat server. Discuss the core architecture, object-oriented components, data structures, and key technical challenges you would face.

To give our design concrete boundaries, our chat system must support the following core capabilities:

• User Management: User sign-up, login, friend additions, and presence tracking (Online, Offline, Away).
• Direct Messaging: One-on-one real-time text chat between users.
• Group Messaging: Multi-user chat channels with dynamic member updates.
• Message History: Persistence and retrieval of historical conversation logs.
• Real-Time Delivery: Push-based message delivery with support for offline message queuing.

Analyze the Problem Together

Before jumping straight into class diagrams, let's analyze the fundamental mechanics of real-time communication. Traditional web applications rely on standard HTTP client-initiated request-response cycles. However, if Bob has to send an HTTP request every second to check if Alice sent him a message (short polling), we end up wasting millions of CPU cycles and network packets on empty responses.

To establish true real-time communication, we need full-duplex, bi-directional socket connections. The standard protocol for this is WebSockets. Once a client completes an initial HTTP handshake, the socket connection is upgraded to a persistent TCP connection where both the server and client can push frames at any time with minimal header overhead.

Let's compare the different network communication approaches we could choose:

The Architectural "Aha!" Moment: Real-time socket servers are inherently stateful. A specific WebSocket server instance holds the physical TCP file descriptor mapped to Bob's connection. If Alice is connected to Server A and Bob is connected to Server B, Server A cannot directly write to Bob's socket. Thus, our core architecture must decouple connection state handling from message routing logic using a distributed Message Broker (such as Redis Pub/Sub or Apache Kafka).

Questions for the Interviewer

When presenting this solution in an interview, never jump straight into writing code without clarifying the scope. Here are four essential questions a senior engineer would ask:

1. Scale & Concurrency: What scale are we targeting? Are we handling 10,000 concurrent users or 10 million concurrent WebSocket connections?
2. Delivery Guarantees: Do we need strict in-order, exactly-once message delivery guarantees, or is at-least-once delivery acceptable?
3. Media Handling: Are we supporting rich media attachments (images, video, audio snippets) or purely text-based payloads for this initial design?
4. Read Receipts & Presence: Is real-time typing indicator and read receipt status required in scope?

Think About the Solution

Let's organize our design into clean Object-Oriented building blocks. We can break down our domain into five major conceptual components:

• User & UserStatus: Encapsulates user profile details, status enum (ONLINE, OFFLINE, AWAY), and active connection handlers.
• Message: Implements the message entity containing payload text, unique sequential ID, timestamp, sender, and recipient target.
• Conversation (Abstract Base): Abstract entity extended by PrivateChat (1-to-1) and GroupChat (1-to-many).
• UserManager: Thread-safe registry that maps user IDs to online statuses and server sessions.
• ChatServerManager: Main orchestrator that handles incoming requests, routes messages via memory maps or pub/sub brokers, and delegates queueing for offline recipients.

Write Code and Explain

Let's write a production-ready, thread-safe Java implementation representing the core domain entities and local message dispatcher for our chat server.

package com.thedevlearnings.chat;

import java.time.Instant;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CopyOnWriteArrayList;

// Represents the status of a user in real-time
enum UserStatusType {
    OFFLINE, ONLINE, AWAY
}

class UserStatus {
    private UserStatusType statusType;
    private String statusMessage;
    private long lastActiveTimestamp;

    public UserStatus(UserStatusType statusType, String statusMessage) {
        this.statusType = statusType;
        this.statusMessage = statusMessage;
        this.lastActiveTimestamp = Instant.now().toEpochMilli();
    }

    public UserStatusType getStatusType() { return statusType; }
    public void setStatusType(UserStatusType statusType) { 
        this.statusType = statusType; 
        this.lastActiveTimestamp = Instant.now().toEpochMilli();
    }
}

// Core User domain model
class User {
    private final String userId;
    private final String username;
    private UserStatus status;
    private final Map<String, User> friendsList = new ConcurrentHashMap<>();

    public User(String userId, String username) {
        this.userId = userId;
        this.username = username;
        this.status = new UserStatus(UserStatusType.OFFLINE, "Offline");
    }

    public String getUserId() { return userId; }
    public String getUsername() { return username; }
    public UserStatus getStatus() { return status; }
    public void setStatus(UserStatus status) { this.status = status; }
    public Map<String, User> getFriendsList() { return friendsList; }

    public void addFriend(User user) {
        friendsList.put(user.getUserId(), user);
    }
}

// Encapsulates a chat message payload
class Message {
    private final String messageId;
    private final String senderId;
    private final String conversationId;
    private final String content;
    private final long timestamp;

    public Message(String messageId, String senderId, String conversationId, String content) {
        this.messageId = messageId;
        this.senderId = senderId;
        this.conversationId = conversationId;
        this.content = content;
        this.timestamp = Instant.now().toEpochMilli();
    }

    public String getMessageId() { return messageId; }
    public String getSenderId() { return senderId; }
    public String getConversationId() { return conversationId; }
    public String getContent() { return content; }
    public long getTimestamp() { return timestamp; }
}

// Abstract base class for Conversations
abstract class Conversation {
    private final String conversationId;
    private final List<User> participants = new CopyOnWriteArrayList<>();
    private final List<Message> messages = new CopyOnWriteArrayList<>();

    public Conversation(String conversationId, List<User> initialParticipants) {
        this.conversationId = conversationId;
        this.participants.addAll(initialParticipants);
    }

    public String getConversationId() { return conversationId; }
    public List<User> getParticipants() { return participants; }
    public List<Message> getMessages() { return messages; }

    public void addMessage(Message message) {
        messages.add(message);
    }
}

class PrivateChat extends Conversation {
    public PrivateChat(String conversationId, User user1, User user2) {
        super(conversationId, Arrays.asList(user1, user2));
    }
}

class GroupChat extends Conversation {
    private String groupName;

    public GroupChat(String conversationId, String groupName, List<User> members) {
        super(conversationId, members);
        this.groupName = groupName;
    }

    public void addMember(User user) {
        getParticipants().add(user);
    }
}

// Mock class representing an active WebSocket session interface
interface UserChannel {
    void sendSystemMessage(String text);
    void deliverMessage(Message message);
}

// Thread-safe Manager handling user routing and state logic
public class ChatServerManager {
    private static ChatServerManager instance;
    private final Map<String, User> users = new ConcurrentHashMap<>();
    private final Map<String, UserChannel> activeSessions = new ConcurrentHashMap<>();
    private final Map<String, List<Message>> offlineQueue = new ConcurrentHashMap<>();
    private final Map<String, Conversation> conversations = new ConcurrentHashMap<>();

    private ChatServerManager() {}

    public static synchronized ChatServerManager getInstance() {
        if (instance == null) {
            instance = new ChatServerManager();
        }
        return instance;
    }

    public void registerUser(User user) {
        users.put(user.getUserId(), user);
    }

    public void userConnected(String userId, UserChannel channel) {
        activeSessions.put(userId, channel);
        User user = users.get(userId);
        if (user != null) {
            user.setStatus(new UserStatus(UserStatusType.ONLINE, "Available"));
        }
        // Flush any queued offline messages
        flushOfflineMessages(userId, channel);
    }

    public void userDisconnected(String userId) {
        activeSessions.remove(userId);
        User user = users.get(userId);
        if (user != null) {
            user.setStatus(new UserStatus(UserStatusType.OFFLINE, "Offline"));
        }
    }

    public void sendMessage(Message message) {
        Conversation conversation = conversations.get(message.getConversationId());
        if (conversation == null) return;

        // Persist message to conversation history log
        conversation.addMessage(message);

        // Dispatch message to participants
        for (User recipient : conversation.getParticipants()) {
            // Don't echo back to sender unless specified by protocol
            if (recipient.getUserId().equals(message.getSenderId())) continue;

            UserChannel channel = activeSessions.get(recipient.getUserId());
            if (channel != null) {
                channel.deliverMessage(message);
            } else {
                // User is offline; push to offline queue
                queueOfflineMessage(recipient.getUserId(), message);
            }
        }
    }

    private void queueOfflineMessage(String userId, Message message) {
        offlineQueue.computeIfAbsent(userId, k -> new CopyOnWriteArrayList<>()).add(message);
    }

    private void flushOfflineMessages(String userId, UserChannel channel) {
        List<Message> pending = offlineQueue.remove(userId);
        if (pending != null) {
            for (Message msg : pending) {
                channel.deliverMessage(msg);
            }
        }
    }

    public void addConversation(Conversation conversation) {
        conversations.put(conversation.getConversationId(), conversation);
    }
}

Code Walkthrough

• Thread-Safe Collections: We use ConcurrentHashMap for global lookups (`users`, `activeSessions`, `offlineQueue`, `conversations`). This guarantees concurrent thread safety without relying on coarse-grained, blocking synchronization methods on every operation.
• CopyOnWriteArrayList: For group chat members and conversation logs, read actions heavily outnumber write updates. CopyOnWriteArrayList lets concurrent reader threads read without locking while thread-safe copies are created during updates.
• Singleton Architecture: ChatServerManager acts as the localized registry managing sessions, user presence updates, and local message routing.
• Offline Message Queuing: When a recipient isn't found in activeSessions, the message is gracefully buffered in the offlineQueue and automatically flushed as soon as the user triggers userConnected().

Time & Space Complexity

Let's evaluate the complexity of our localized chat server core operations:

• Sending a 1-to-1 Message:
  • Time Complexity: O(1) lookup time to fetch conversation details and retrieve recipient channel from ConcurrentHashMap.
  • Space Complexity: O(1) memory allocation for the `Message` object entity.
• Sending a Group Message:
  • Time Complexity: O(K), where K represents the number of active participants in the channel to deliver socket payloads.
  • Space Complexity: O(1) heap overhead per active broadcast.
• User Connection/Disconnection:
  • Time Complexity: O(1) map update and O(M) to flush M queued offline messages.
  • Space Complexity: O(U) overall space complexity, where U is total connected concurrent users stored in heap state maps.

Why This Problem Matters (The Senior Perspective)

In a technical interview, solving the problem for a single node instance is only halfway to standard expectations. Senior software engineers must demonstrate how this object model scales across distributed systems in production.

Here are three key architectural challenges you must address in a production system:

1. Scaling Beyond a Single Machine (Pub/Sub Router)

A single server can comfortably host around 50,000 to 100,000 concurrent WebSocket connections (bounded by ephemeral ports, file descriptors, and RAM memory overhead). To scale to millions of users, we deploy a fleet of Chat Socket Nodes behind a load balancer with TCP pass-through capabilities.

When Alice on Node 1 sends a message to Bob on Node 50, Node 1 publishes the event to a central Pub/Sub cluster (such as Redis Pub/Sub or NATS). Each chat server node subscribes to channels matching its locally connected active user IDs. When Node 50 receives the pub/sub payload for Bob, it routes it down Bob's local WebSocket connection.

2. Data Persistence Strategy

Writing every single chat message synchronously to a traditional relational database (like MySQL) creates severe write bottlenecks at scale. Instead, high-throughput chat applications rely on write-optimized NoSQL datastores like Apache Cassandra, ScyllaDB, or Amazon DynamoDB.

By defining our partition key as conversation_id and clustering key as message_id (or time-based UUIDv7), we can perform sequential disk appends for writes and achieve lightning-fast, time-ordered pagination reads.

3. Presence Tracking at Scale

Updating database tables every time a user toggles online/offline generates excessive write traffic. Instead, client connections periodically push lightweight heartbeat pings to an in-memory Redis cluster using key TTLs (Time-To-Live). If a client fails to send a heartbeat within 30 seconds, the key expires, and an event fires triggering the user status to OFFLINE.

Final Thoughts

We've successfully taken CTCI Problem 7.7 from a high-level system prompt to an object-oriented Java domain model and extended it into a distributed architecture design. Designing a chat server showcases your grasp of object modeling, concurrency data structures, real-time protocols, and distributed systems messaging patterns.

To take your learning further, ask yourself: How would you design end-to-end encryption (E2EE) where the chat server acts purely as an untrusted relay? How would you handle client message deduplication using idempotency keys?

Happy coding, and see you in the next installment of The Dev Learnings!