Design Instagram

1. The system should be highly available , prioritizing availability of photos/videos over consistency (eventual consistency is fine, up to 2 minutes).
2. The system should deliver feed content with low latency (< 500ms end-to-end response time for feed requests).
3. The system should render photos and videos instantly (low latency media delivery).
4. The system should be scalable to support 500M DAU.

• The system should be secure and protect user data (authentication, authorization, encryption).
• The system should be fault-tolerant and highly reliable (no data loss).
• The system should provide analytics on user behavior and engagement.

IG Requirements

The Set Up

Defining the Core Entities

1. User: This entity will store user information like username, profile details, etc. (We'll keep it minimal for now).
2. Post: This will include a reference to the media file, a caption, and the user who created it. Crucially, a single Post entity will handle BOTH photos and videos.
3. Media: Represents the actual bytes of the media file. We'll use S3 for this.
4. Follow: Represents the relationship between two users, where one user (the follower) is following another user (the followee). This captures the uni-directional "follow" relationship.

IG Entities

API or System Interface

POST /posts -> postId
{
  "media": {photo or video bytes},
  "caption": "My cool photo!",
}

POST /follows
{
  "followedId": "123"
}

GET /feed?cursor={cursor}&limit={page_size} -> Post[]

High-Level Design

1) Users should be able to create posts featuring photos, videos, and a simple caption

Create Post

1. Clients: Users will interact with the system through the clients website or app. All client requests will be routed to the system's backend through an API Gateway.
2. API Gateway: This serves as an entry point for clients to access the different microservices of the system. It's primarily responsible for routing requests to the appropriate services but can also be configured to handle cross-cutting concerns like authentication, rate limiting, and logging.
3. Post Service: Our first microservice is responsible for handling post creation requests. It will receive the media and caption, store the post metadata in the database, the actual bytes on the media in a blob store, and return a postId to the client.
4. Posts DB: Stores metadata about each post including who created it, when it was created, the caption, and a link to the media in the blob store.
5. Blob Store: Stores the actual bytes of the media. We'll use S3 in our case as it's the most popular solution.

1. The client makes a POST request to the API Gateway with the media and caption.
2. The API Gateway routes the request to the Post Service.
3. The Post Service receives the media and caption, stores the post metadata in the database, and the actual bytes on the media in a blob store.
4. The Post Service returns a postId to the client.

2) Users should be able to follow other users

Followers

3) Users should be able to see a chronological feed of posts from the users they follow

1. Get followees: Query the Follow table to get a list of user_id s that the current user follows.
2. Get Posts: For each of those followed users, query the Post table to get their recent posts.
3. Merge and Sort: Combine all the retrieved posts and sort them chronologically (by timestamp or postId ).
4. Return: Return the sorted posts to the client.

Feed Generation

1. The client makes a GET request to the API Gateway with the cursor and limit .
2. The API Gateway routes the request to the Post Service.
3. The Post Service queries the Follow table to get the followed users of the current user.
4. The Post Service queries the Post table for each followed user to get their recent posts.
5. The Post Service combines and sorts the posts and returns them to the client, limited by the cursor and limit .

Potential Deep Dives

1) The system should deliver feed content with low latency (< 500ms )

1. Read Amplification: Every time a user refreshes their feed, we generate a large number of database reads. With 500M daily active users refreshing their feeds multiple times per day, this quickly becomes unsustainable. This is going to get expensive fast.
2. Repeated Work: If two users follow many of the same accounts (which is common), we're repeatedly querying for the same posts. At Instagram's scale, popular posts might be retrieved millions of times.
3. Unpredictable Performance: The latency of a feed request would vary dramatically based on how many accounts a user follows and how active those accounts are. This makes consistent performance nearly impossible.

• Each feed refresh might need to process 10,000 posts (1,000 followed accounts × 10 posts/day)
• With 500M DAU, if each user refreshes their feed 5 times daily, that's 2.5 billion feed generations per day
• During peak usage (e.g., evenings, major events), we might see 150,000+ feed requests per second

1. Need to massively overprovision our database capacity, making the system prohibitively expensive, or
2. Accept higher latencies during peak usage, violating our requirements

Bad Solution: Simple Caching (Improvement on Fan-out on Read)

Approach

The most obvious improvement to the fan-out on read approach is adding a cache in front of the Posts table to cache each users recent posts. We can use Redis for this. The idea is simple: before querying the database for a user's followed users' posts, we check the cache. If the posts are in the cache, we return them. If not, we query the database and then store the results in the cache for future requests.

We can key the cache by a combination of the user_id and a cursor (or timestamp). This lets us get a specific "page" of the feed from the cache. An example key would be: feed:{user_id}:{cursor} . The value would be a list of postId s.

Feed Generation with Simple Cache

Challenges

Caching helps but doesn't fundamentally solve our scalability challenges. The cache needs to be massive to achieve a meaningful hit rate at Instagram's scale, and we still perform expensive fan-out reads to aggregate posts from all followed users for every feed request. While caching is a useful optimization, it's treating the symptom rather than addressing the root problem of our fan-out read architecture.

Good Solution: Precompute Feeds (Fan-out on Write)

Approach

A much better approach is to precompute the feeds. Instead of generating the feed when the user requests it (fan-out on read), we generate it when a user posts (fan-out on write).

Here's how it works: When a user creates a new post, we query the Follows table to get all users who follow the posting user. For this to be efficient, we need a Global Secondary Index (GSI) on the Follows table with followedId as the partition key and followerId as the sort key. This lets us quickly find all followers of a user who just posted. For each follower, we prepend the new postId to their precomputed feed. This precomputed feed can be stored in a dedicated Feeds table (in DynamoDB, for example) or in a cache like Redis.

Let's use Redis. It provides very fast read and write access, perfect for feeds. We can store each user's feed as a sorted set (ZSET in Redis). The members are postId s, and the scores are timestamps (or postId s, if they are chronologically sortable).

Precomputed Feeds

Data Model (Redis):

• Key: feed:{user_id}
• Type: Sorted Set (ZSET)
• Members: postId
• Scores: timestamp (or postId )

When a user requests their feed, we read the top N posts from their sorted set in Redis, which is a single, fast operation. However, we still need to hydrate the posts based on these postId s. To do this we have 3 options:

1. For each postId in the cache, go fetch the metadata from the Posts table in DynamoDB. This is simple but requires an additional database query for every feed request.
2. Rather than caching the postId, we could cache the entire post metadata in Redis. This way we don't have to make a second query to the Posts table to get the post metadata. This is faster but uses more memory and introduces data consistency challenges.
3. Use a hybrid approach with two Redis data structures: one for the feed's postIds (ZSET) and another for a short-lived post metadata cache (HASH). When we need to hydrate posts:

   • First, try to get all post metadata from the Redis post cache
   • For any cache misses, batch fetch from DynamoDB using BatchGetItem
   • Update the Redis post cache with the fetched metadata
   • Return the combined results

The hybrid approach (option 3) gives us the best balance of performance and resource usage. We can tune the TTL of the post metadata cache based on our memory constraints and consistency requirements. For example, if a post were to get updated, like a change to the caption, we would just invalidate the cache for that post and it would be updated in the next feed request.

To recap, here is what happens when a new post is created.

1. We store the post metadata in the Posts table and the media in the blob store like before.
2. We put the postId onto a queue to be asynchronously processed by the Feed Fan-out service.
3. The feed fan-out service will query the Follows table to get all users who follow the posting user.
4. For each follower, it will prepend the new postId to their precomputed feed.
5. The feed fan-out service will store the new feed in Redis.

Then, when a user requests their feed, we:

1. Read the top N posts from their sorted set in Redis.
2. Hydrate the posts based on these postId s. We first check if the post metadata is in the Redis post cache. If it is, we use that. If it's not, we batch fetch from DynamoDB using BatchGetItem.
3. Combine the results and return them to the user.

Challenges

This 'fan-out on write' approach significantly improves read performance, making feed retrieval a fast, single Redis query (plus metadata lookups). However, we've traded read-time complexity for write-time complexity and increased storage. The primary challenge is write amplification, especially the 'celebrity problem'. A single post by a user with millions of followers triggers millions of writes to Redis, potentially overwhelming our system and increasing write latency for popular users. This is because we need to update the feed cache for the millions of people following them.

Great Solution: Hybrid Approach (Precompute + Real-time)

Approach

We can address the main limitation of the precompute approach, that celebrities will result in massive write amplification, by using a hybrid approach. We combine fanout-on-write for most users with fanout-on-read for popular accounts. This provides a good balance: fast reads for most users and manageable write amplification.

Here's how it works: We define a threshold for the number of followers. Let's say, 100,000 followers. For users with fewer than 100,000 followers, we precompute their followers' feeds just like in the "good" approach above. For users with more than 100,000 followers (the "celebrities"), we DON'T precompute their posts into their followers' feeds.

Instead:

1. When a "celebrity" posts, we add the post to the Posts table and do not trigger an asynchronous feed update for their followers.
2. When a user requests their feed: We fetch the precomputed portion of their feed from Redis (posts from users with < 100,000 followers). Then, we also query the Posts table for recent posts from the "celebrities" they follow. We then merge the precomputed feed with the recent posts from celebrities, chronologically and return the merged feed.

Thus, we end up with an effective mix between pre-computation and real-time merging.

1. Fanout-on-write for the majority of users (follower count < 100,000)
2. Fanout-on-read for the few "celebrity" users (follower count > 100,000)

Challenges

As is always the case, more complexity comes with its own tradeoffs. The 100,000 follower threshold needs to be carefully tuned - set it too low and we don't solve the write amplification problem, set it too high and we impact read performance for too many users.

Users following many celebrities will experience slower feed loads than those who don't, creating an inconsistent user experience. We'll need to set clear SLAs for different user segments and potentially implement additional caching specifically for celebrity posts. We also introduce storage complexity by maintaining two separate systems - the precomputed feeds in Redis for regular users and the real-time query system for celebrity posts. This increases operational complexity and requires careful monitoring of both systems.

Despite these challenges, this hybrid approach has proven effective in practice. Instagram actually uses a similar approach in production, demonstrating that the benefits outweigh the added complexity.

2) The system should render photos and videos instantly, supporting photos up to 8mb and videos up to 4GB

1. Upload efficiency.
2. Download/viewing latency.

1. First, we will call POST /posts to create the post metadata and get a postId as before, but now it will also return a pre-signed URL that can be used to upload the media. This URL is valid for a limited time (e.g., 1 hour) and allows the user to upload directly to S3 without having to go through our servers.
2. Client side, we use the multipart upload API to upload the file in chunks to the pre-signed URL.
3. S3 will automatically handle reassembling the chunks and store the final file in S3.

1. Client-driven approach: The client sends a PATCH request to update the post metadata with the S3 object_key and sets the media upload status to complete once the multipart upload finishes.
2. Server-driven approach: We configure S3 event notifications to trigger a Lambda function or background job when the multipart upload completes. This job then updates the post metadata automatically.

Upload

Bad Solution: Direct S3 Serving

Approach

The simplest approach would be to serve media directly from S3. When a user views a post, we return the S3 URL for the media file, and their client downloads it directly from S3. This is straightforward to implement and works fine for small applications, but it doesn't scale well for a global application like Instagram.

Direct S3 Serving

Challenges

This approach falls short in several ways. First, users far from our S3 bucket's region will experience high latency when loading media. A user in Singapore accessing media stored in us-east-1 might wait seconds for images to load. Second, every request hits S3 directly, which becomes expensive at scale and doesn't take advantage of any caching. Finally, we're serving the same high-resolution files to all users regardless of their device or network conditions, wasting bandwidth and slowing down the user experience.

Good Solution: Global CDN Distribution

Approach

A significant improvement is to put a CDN like CloudFront in front of our S3 bucket. The CDN maintains a global network of edge locations that cache our media files closer to users. When a user requests media, they're automatically routed to the nearest edge location. If the media isn't cached there, the CDN fetches it from S3 and caches it for future requests.

This way, instead of our Singaporean user waiting seconds to load an image originally stored on the US East Coast, they can load it from a server in the region closest to them.

We can configure the CDN to cache files based on their type and popularity. For example, we might cache images for 24 hours at edge locations since they rarely change once uploaded. This dramatically reduces latency for users and load on our origin S3 bucket. We can also use the CDN's built-in compression to reduce file sizes during transfer.

Global CDN Distribution

Challenges

While this solves our global distribution problem, we're still serving the same files to all users. A user on a mobile device with a slow connection receives the same high-resolution image as someone on a fast desktop connection. We also have limited control over the caching strategy, which means popular media might get evicted from edge locations due to cache size limits, leading to unnecessary origin fetches.

Great Solution: CDN with Dynamic Media Optimization

Approach

The most effective solution combines CDN distribution with dynamic media optimization. When media is uploaded, we use a media processing service (like Cloudinary or Imgix) to generate multiple variants optimized for different devices and network conditions. For images, this includes different resolutions and formats like WebP for supported browsers. For videos, we create multiple quality levels and use adaptive streaming (more details in the YouTube breakdown).

The CDN then serves these optimized variants based on the requesting device and network conditions. Mobile users automatically receive appropriately sized images, while desktop users get higher resolution versions. The client includes device information and network conditions in its requests, allowing the CDN to serve the most appropriate variant.

We also implement intelligent caching strategies. Popular media is cached more aggressively at edge locations, while less accessed content might have shorter TTLs. For viral content that suddenly gains popularity, we can proactively warm caches in regions where we predict high viewership.

CDN with Dynamic Media Optimization

Challenges

This approach significantly improves user experience but introduces more complexity in our system. We need to manage multiple versions of each media file, increasing storage costs and complexity. The media processing pipeline adds latency to uploads and requires careful monitoring. Despite these challenges, this is the approach most large-scale media platforms use as it provides the best balance of performance and manageability.

3) The system should be scalable to support 500M DAU

1. Precomputed Feeds (Hybrid Approach): The cornerstone of our scalable feed generation is the hybrid approach. By precomputing feeds for the vast majority of users (those following accounts with fewer than our defined threshold of followers), we drastically reduce read-time load. The real-time merging for "celebrity" posts is a carefully considered trade-off to manage write amplification.
2. Content Delivery Network (CDN): Using a CDN like CloudFront for media delivery is essential for global scalability and low-latency access. Dynamic media optimization further improves performance for diverse devices and network conditions.
3. Chunked Uploads: For large files (especially videos), chunking uploads improves reliability and user experience, and allows for parallel uploads.
4. Database Choice and Indexing: Our choice of DynamoDB (or a similarly scalable NoSQL database) provides horizontal scalability for metadata storage. The careful use of partition keys and sort keys ensured efficient queries.

Final Design

What is Expected at Each Level?

Mid-level

Senior

Staff+