Python Developer Interview Questions & Answers

121. Tell me how you would diagnose and fix a slow Python endpoint or background job.PerformanceHard

Question Details

Explain how you would measure latency, profile CPU and memory, inspect database queries and network calls, find blocking work, test changes under load, and verify that an optimization improves the real bottleneck.

Short Interview Answer (30-60 seconds)

I would start by defining what slow means with a measurable target, such as p95 latency for an endpoint or total processing time and queue delay for a background job. Then I would capture a baseline using application metrics, logs, and distributed traces so I can see whether time is spent in Python code, database queries, network calls, queue waits, locks, or resource pools. After narrowing the problem, I would use the right profiler, such as py spy or cProfile for CPU work and tracemalloc for Python memory allocations. I would make one change that targets the measured bottleneck, run the same representative load again, compare the same metrics, verify correctness, and check that the bottleneck did not move to another dependency. The main tradeoff is that profiling and tracing can add overhead, so I would use controlled runs, sampling, or a limited production rollout.

Detailed Explanation

I would begin by defining the symptom and the success metric. For an endpoint, I would usually inspect p50, p95, and p99 latency, throughput, error rate, request queue time, and resource saturation. For a background job, I would separate queue wait time from worker execution time and also inspect retry rate, failure rate, throughput, and completion time. This creates a baseline and prevents me from optimizing code that is not responsible for the real delay.

Useful Questions to Ask the Interviewer

What user-visible symptom and measurable performance target define success?
What workload, environment, data size, and concurrency level should I assume?
What profiling evidence is available, and which tradeoffs or system changes are allowed?

Next, I would reproduce the problem with representative requests, job payloads, data sizes, dependency behavior, and concurrency. A single fast local request is not enough because production delays may appear only with a large database, multiple workers, connection pool pressure, or many concurrent requests. I would keep the workload stable so that the before and after results are comparable. I would also keep worker count, data state, cache state, warmup period, test duration, and dependency conditions consistent between runs.

I would then break down the execution path using metrics, logs, and distributed tracing. For an endpoint, I would inspect request queue time, middleware, application code, database calls, external network calls, serialization, and response time. For a background job, I would inspect queue delay, worker startup, task code, retries, database calls, external services, locks, batching, and result publication. This helps separate active CPU time from time spent waiting.

If the evidence points to CPU work, I would normally prefer low overhead metrics, tracing, and a sampling profiler such as py spy for a running production process. I would use cProfile and pstats in a controlled environment because deterministic profiling can materially change execution timing. I would look for hot functions, repeated work, inefficient loops, expensive serialization, and algorithms whose cost grows badly with input size. A sampling profiler can miss very short events, so I would treat every profiler result as evidence rather than absolute proof.

If memory is the problem, I would inspect process memory trends and use tracemalloc snapshots to compare Python allocations over time. I would look for retained objects, large temporary objects, allocation churn, unbounded caches, growing queues, and data loaded fully into memory. I would compare tracemalloc results with process level resident memory because native extensions, allocator behavior, and child processes may increase memory without appearing fully in Python allocation snapshots.

If database time is high, I would inspect query logs, query count, execution plans, indexes, result size, transaction contention, and connection pool wait time. Common issues include repeated queries, missing indexes, selecting too much data, long transactions, and a pool that is too small or already saturated. If network time is high, I would inspect timeout values, connection reuse, payload size, retries, remote service latency, and whether independent calls can be safely combined or run concurrently.

I would also look for blocking work. In synchronous code, this may appear as thread pool starvation, lock contention, or long blocking calls. In asyncio code, a synchronous database driver, blocking file operation, CPU heavy loop, or time.sleep call can block the event loop and delay unrelated requests. An awaited asyncio.sleep normally yields control and does not block the event loop. Event loop lag and task timing can help confirm this. The fix may be an async compatible library, moving CPU work to a process, using an executor for controlled blocking work, or reducing the work itself.

After identifying the bottleneck, I would make one evidence based change. Examples include improving a query, removing repeated calls, batching operations, reducing serialization, streaming large results, changing an algorithm, adding a bounded cache, tuning a connection pool, or moving CPU intensive work to a process. I would not add more threads, workers, caching, or concurrency without checking resource limits and failure behavior because those changes can increase memory use, create contention, or overload a dependency.

I would verify the change by running the same representative workload and comparing the same baseline metrics. I would check latency percentiles, throughput, CPU, memory, queue delay, errors, pool waits, and dependency timing as relevant. I would also run correctness tests because a faster result is not useful if it is stale, incomplete, duplicated, reordered, or incorrect. Finally, I would check whether the original bottleneck was reduced or simply moved to the database, network, queue, or another worker.

For production validation, I would prefer a staged rollout, canary release, or feature flag when the risk justifies it. I would monitor the same metrics after deployment and be ready to roll back if errors, saturation, memory use, or tail latency become worse. The key idea is to measure first, optimize the proven bottleneck, and verify with the same workload.

Tell me how you would diagnose and fix a slow Python endpoint or background job. diagram

Technical Approach

Define the symptom and success metric. Measure endpoint latency or job queue and execution time.
Capture a baseline with percentiles, throughput, errors, CPU, memory, saturation, and dependency timing.
Reproduce the issue with representative requests, data, dependency behavior, and concurrency.
Use traces, logs, metrics, and query information to divide total time into application work and waiting time.
Classify the bottleneck as CPU, memory, database, network, disk, queue, lock, pool, or event loop related.
Use a focused profiler or diagnostic tool that matches the suspected problem.
Make one evidence based change that directly addresses the measured bottleneck.
Repeat the same load test and compare the same metrics.
Verify functional correctness and check whether the bottleneck moved elsewhere.
Release carefully and monitor the original success metric in production.

Practical Insights

There is no single algorithmic complexity for this investigation because the cost depends on the endpoint, job, data size, and dependency behavior. Profiling adds some CPU and timing overhead. Distributed tracing adds instrumentation and storage cost. Load testing uses compute, database connections, network capacity, and engineering time. Some fixes also introduce new costs. Caching uses memory and can serve stale data. More concurrency uses additional connections and can overload downstream systems. Multiprocessing increases process startup, serialization, and memory costs. The correct choice is the smallest change that improves the measured bottleneck without creating an unacceptable reliability or maintenance cost.

Why Interviewers Ask This

Interviewers ask this question to see whether the candidate measures a real production symptom before changing code. They want evidence that the candidate can separate CPU work from waiting on databases, networks, queues, locks, and other dependencies. They also want to know whether the candidate can choose suitable profiling tools, test a change under realistic load, preserve correctness, and verify that the original bottleneck was actually reduced.

Common interview mistakes

A common mistake is changing code before defining the symptom and baseline. Another is using average latency only and missing slow p95 or p99 requests. Candidates may profile unrealistic input, treat a timeit result as proof of service performance, or confuse CPU time with time spent waiting for a database or network call. Other mistakes include ignoring query count and connection pool waits, blocking the asyncio event loop with synchronous input and output, CPU work, or time.sleep, adding unbounded threads or tasks, comparing different workloads before and after the change, and assuming more workers will fix every problem. It is also a mistake to trust one profiler as complete proof, ignore profiler overhead, skip correctness checks, or improve one component while moving the bottleneck to another dependency.

Interview tip

Explain the investigation as a measured sequence. Start with the symptom and baseline, show how you divide total time into CPU and waiting, name the tool that confirms the suspected bottleneck, describe one targeted fix, and finish with the same load test, correctness checks, and production monitoring.

Interviewer may ask next

What would you do if the endpoint is fast in local profiling but slow in production?

I would treat that as evidence that the local workload or environment does not reproduce the production boundary. I would compare production traces, queue time, database latency, connection pool waits, external calls, payload sizes, data volume, concurrency, and resource limits. The exact boundary is the complete request path, not only the Python function measured locally. I would build a representative test using production like data and dependency timing, then use low overhead production metrics or sampling profiles where safe. The tradeoff is that production observation provides realistic evidence but must be limited to avoid excessive overhead and exposure of sensitive data.

How would you verify that adding more workers is the correct fix?

I would add workers only when measurements show available downstream capacity and a workload that can benefit from more parallel processing. The boundary includes the worker pool, queue, CPU, memory, database connections, network dependencies, and any shared locks. I would run the same representative load with controlled worker counts and compare queue delay, throughput, task time, CPU, memory, errors, retries, and dependency saturation. It matters because more workers can reduce queue delay but can also exhaust memory, increase lock contention, or overload the database. The main tradeoff is higher parallel capacity versus greater resource use and pressure on shared dependencies.

122. How do you decide between threading, multiprocessing, and asyncio for a Python workload?PerformanceHard

Question Details

Compare I/O-bound and CPU-bound work, the effect of the GIL, process overhead, event-loop behavior, cancellation, shared state, and how you would choose a concurrency model for a production service.

Short Interview Answer (30-60 seconds)

I would first confirm the Python runtime and measure whether the workload is mostly waiting for input and output or spending time on CPU work. I would use threading when I must run blocking input and output libraries concurrently and the shared state is manageable. I would use multiprocessing for substantial pure Python CPU work when parallel execution is worth the process startup, serialization, communication, memory, and cancellation costs. I would use asyncio for high concurrency input and output when the libraries support async behavior and the service can handle cancellation, timeouts, backpressure, and event loop discipline. On the standard GIL enabled CPython build, threads usually do not run pure Python bytecode in parallel, but they can still help while waiting, and native extensions may release the GIL. Optional free threaded Python builds can run Python threads in parallel, so I would confirm the interpreter build and extension compatibility before relying on that behavior. I would validate the choice with representative load, latency, throughput, CPU, memory, queue growth, and failure behavior rather than assuming one model is always fastest.

Detailed Explanation

I would begin by confirming the Python runtime and measuring the workload instead of deciding from the task name. I would check whether the service uses the standard GIL enabled CPython build or an optional free threaded build, because that changes how Python threads can use CPU cores. I would then use application metrics, traces, profiling, queue measurements, event loop lag, and resource utilization to determine whether the workload is mainly CPU work, blocking input and output, or high concurrency async input and output under representative load.

Useful Questions to Ask the Interviewer

What user-visible symptom and measurable performance target define success?
What workload, environment, data size, and concurrency level should I assume?
What profiling evidence is available, and which tradeoffs or system changes are allowed?

A workload is CPU bound when most time is spent performing Python or native computation. It is input and output bound when most time is spent waiting for a database, network service, disk, queue, or another external resource. I would also check whether the important libraries are synchronous, async capable, or native extensions that may release the GIL. This matters because the same business task can behave differently depending on the runtime, library implementation, task size, and data movement.

Threading is usually suitable when the code uses blocking input and output libraries and several operations need to wait concurrently. While one thread waits for network, database, or disk input and output, another thread can run. On the standard GIL enabled CPython build, only one thread normally executes Python bytecode at a time, so threading is usually not the first choice for heavy pure Python CPU work. However, the GIL does not make threading useless. Native extensions may release it, and optional free threaded Python builds can allow Python threads to execute in parallel across CPU cores. I would confirm interpreter support and extension compatibility before relying on free threading because it is optional and some extensions may not behave the same way. Threading also has costs such as context switching, thread stack memory, lock contention, race conditions, limited cancellation, and difficult shared state management. I would use a bounded thread pool rather than creating unlimited threads.

Multiprocessing is usually suitable for substantial pure Python CPU work that can be divided into independent units. Separate processes have separate Python interpreters and can execute Python code in parallel across CPU cores. The costs are process startup, serialization, data copying, interprocess communication, larger memory use, result collection, and more complex failure handling. Small tasks may become slower because the overhead is larger than the useful work. Shared in memory state is not automatically available between processes, so I would prefer immutable inputs, coarse task sizes, and clear message based boundaries. I would also remember that cancellation is limited after work has started in another process, so shutdown and task termination need explicit design.

Asyncio is usually suitable for services with many concurrent input and output operations when the libraries support nonblocking async interfaces. Many tasks share one event loop and cooperate by yielding control at await points. This can provide high concurrency without one operating system thread per task. It does not automatically make code faster, and one blocking synchronous call or CPU heavy loop can delay every task on the event loop. Blocking work should be moved to a controlled executor, process, or separate worker when appropriate.

Asyncio also requires careful timeout, cancellation, and cleanup behavior. Cancellation is cooperative, so code must reach await points and handle cleanup correctly. Background tasks must not leak after a request ends. The service also needs bounded concurrency and backpressure so that it does not create unlimited tasks, fill queues, exhaust connection pools, or overload downstream systems. Shared mutable state can still create races because tasks may interleave at await points.

For a production service, I would use a clear decision order. First, confirm the runtime and whether the GIL is enabled. Second, measure CPU time versus waiting time. Third, check whether required libraries are blocking, async capable, or native code that releases the GIL. Fourth, estimate concurrency, task size, serialization cost, and memory use. Fifth, evaluate shared state, cancellation, backpressure, graceful shutdown, observability, deployment, and team maintenance cost. Only then would I select the simplest model that satisfies the measured workload.

If the database driver and HTTP client are synchronous, a bounded thread pool may be simpler than converting the whole service to asyncio. If the service already uses async libraries and must handle many concurrent network requests, asyncio may be the clearest choice. If a request contains expensive pure Python computation on a standard GIL enabled build, I may keep the service input and output path async or threaded but move that CPU work to a process pool or separate worker. If a native library releases the GIL or the runtime is a compatible free threaded build, threads may also be worth benchmarking for CPU work.

The choice can be mixed, but each boundary must be explicit. For example, an async service may use a bounded process pool for CPU heavy work. A threaded service may send large computation to worker processes. I would avoid mixing models without a measured need because the combination increases cancellation, shutdown, debugging, and deployment complexity.

I would validate the chosen model with the same representative workload and compare latency percentiles, throughput, CPU use, memory use, queue growth, connection pool pressure, event loop lag, errors, timeouts, cancellation behavior, and graceful shutdown. The correct model is the one that improves the measured workload while keeping correctness, resource use, reliability, and maintenance cost acceptable.

How do you decide between threading, multiprocessing, and asyncio for a Python workload? diagram

Technical Approach

Confirm the Python runtime and whether the GIL is enabled.
Measure where the workload spends time.
Classify it as mainly CPU work, blocking input and output, or high concurrency async input and output.
Check whether the libraries are synchronous, async capable, or native code that may release the GIL.
Choose threading for bounded concurrent blocking input and output when shared state is manageable.
Benchmark threads for CPU work only when native code releases the GIL or a compatible free threaded build is in use.
Choose multiprocessing for large enough pure Python CPU tasks that justify process and serialization overhead.
Choose asyncio for high concurrency input and output when the full path can cooperate with the event loop.
Define limits for threads, processes, tasks, queues, and connection pools.
Design cancellation, timeouts, backpressure, shutdown, and failure handling.
Test the model with representative traffic and data.
Compare latency, throughput, CPU, memory, queue growth, errors, and operational complexity before adopting it.

Practical Insights

The main cost is not one simple algorithmic complexity. Threading adds thread memory, context switching, lock management, and shared state risk. Multiprocessing adds process startup, serialization, communication, duplicated memory, and worker management. Asyncio can support many waiting tasks with lower per task overhead, but it adds event loop rules, cancellation handling, and the risk that one blocking call delays all tasks. Every model also uses database connections, network sockets, queues, and downstream capacity. The best choice is the model whose useful concurrency is greater than its coordination and operational cost for the measured workload.

Why Interviewers Ask This

Interviewers ask this question to see whether the candidate can classify a workload using evidence instead of choosing a concurrency model by habit. They want to know whether the candidate understands the GIL, blocking input and output, process overhead, event loop behavior, cancellation, shared state, backpressure, and production operations. They are also checking whether the candidate validates the choice with realistic load and resource measurements.

Common interview mistakes

A common mistake is choosing a model from intuition without measuring whether the work is CPU bound or waiting. Another is describing the GIL as universal without checking whether the service uses the standard GIL enabled build or an optional free threaded build. Candidates may also claim that threads are useless, even though they can help blocking input and output, native extensions may release the GIL, and compatible free threaded builds can run Python threads in parallel. Other mistakes include using threads for heavy pure Python CPU work on the standard build, ignoring process startup and serialization cost, assuming process work can always be cancelled after it starts, placing blocking calls inside the event loop, creating unbounded threads or async tasks, sharing mutable state without synchronization, and assuming async cancellation stops work immediately. Candidates also forget backpressure, connection pool limits, graceful shutdown, task cleanup, process failure handling, extension compatibility, and the need to test under realistic concurrency.

Interview tip

Start by confirming the Python runtime and classifying the measured workload. Then explain threading for blocking input and output, multiprocessing for substantial pure Python CPU work, and asyncio for high concurrency async input and output. Mention the standard GIL behavior, optional free threaded builds, native extensions that may release the GIL, one major cost of each model, cancellation and backpressure, and finish by saying that the choice must be verified under representative production load.

Interviewer may ask next

What would you do if an asyncio service contains a CPU heavy Python function?

I would not run that function directly on the event loop because it would delay unrelated tasks. The exact boundary is the CPU heavy function, while the surrounding network and database path can remain async. On a standard GIL enabled build, I would first reduce or optimize the computation, then move the remaining substantial pure Python CPU work to a bounded process pool or separate worker and await its result. If the work is inside a native extension that releases the GIL, or the runtime is a compatible free threaded build, I would also benchmark a bounded thread pool. This matters because the event loop must remain responsive. The tradeoff is process or thread overhead, serialization, memory use, cancellation complexity, and the need to limit queued work.

How would you choose between a thread pool and asyncio for a high concurrency network service?

I would choose based on library support, concurrency level, cancellation needs, and measured operational cost. The boundary is the network input and output path and its database or HTTP clients. If the required libraries are blocking and the expected concurrency is moderate, a bounded thread pool may be simpler and safer. If the full path supports async interfaces and the service needs many concurrent connections, asyncio may use fewer operating system threads and provide clearer timeout and cancellation control. The tradeoff is simpler synchronous code with thread and lock costs versus event loop discipline, async library requirements, and blocking call risk.

123. What is system design?System DesignEasy

Question Details

Define system design as deciding how software components, data stores, interfaces, and infrastructure work together to meet clear requirements. Explain the beginner interview sequence: clarify scope and users, identify functional and non-functional requirements, estimate scale, define APIs and data, draw a simple architecture, and then discuss bottlenecks, failures, security, observability, and tradeoffs.

Short Interview Answer (30-60 seconds)

At a high level, system design means deciding how software parts work together. The main challenge is meeting user needs while balancing speed, reliability, security, scale, and cost. I would explain it in three parts: understand requirements and scale, define APIs and data, then draw the architecture and discuss risks. In the example, clients reach services through a Load Balancer and API Gateway. Supporting parts include data stores, Cache, Message Queue, and CDN. The main trade-off is simplicity versus more scale and flexibility.

Detailed Explanation

System design means planning how a complete software product should work. You decide what users need and how much traffic may arrive. You also decide how software parts communicate and where different data belongs. The difficult part is choosing a design that stays useful as traffic grows. It should also remain reliable, secure, observable, and affordable. The diagram teaches a simple interview process first. It then uses a photo-sharing app to show how those decisions become a real architecture.

Useful Questions to Ask the Interviewer

Who are the main users?
What are the most important use cases?
What traffic and growth should we expect?
Which quality goals matter most, such as speed or reliability?
Are there important security or cost limits?

How to Explain It in an Interview

1. Clarify the scope and users

A good opening is, “First, I want to understand the product and its users.” Ask what problem the system solves. Ask which use cases are most important. This keeps the design focused before choosing technology.

2. Identify requirements and estimate scale

Next, separate functional requirements from non-functional requirements. Functional requirements describe what the product does. The diagram gives sign-up, login, create, read, update, delete, search, and notifications as examples.

Non-functional requirements describe how well the system should work. The diagram includes performance, availability, scalability, reliability, security, and cost. Then estimate scale. Its example uses one million users, 200,000 daily active users, a 10:1 read-to-write ratio, one terabyte of data, and 20% monthly growth. These numbers help guide later choices.

3. Define APIs and data

Then explain the main APIs and the data they use. List the important API operations and their request and response formats. Define the data model and relationships. After that, choose databases that fit those needs. This gives the architecture a clear contract before drawing the main components.

4. Draw the simple architecture

For the photo-sharing example, the clients are the Mobile App and Web App. Normal application requests move through the Load Balancer and then the API Gateway. The gateway connects to the User Service, Photo Service, and Feed Service.

The User Service connects to User DB, which is relational. The Photo Service connects to Photo Storage, which is object storage. The Feed Service connects to Feed DB, which is NoSQL. The diagram also shows Cache (Redis), Message Queue (Async Jobs), and CDN (Static Files) as supporting parts. A separate path from the Clients area reaches Cache. Application services send background work to the Message Queue. Photo Storage provides static content to the CDN.

5. Discuss bottlenecks, failures, security, and trade-offs

Finally, explain what could become difficult as the system grows. The diagram lists database pressure, storage pressure, hot keys, and network limits as bottlenecks. It also calls out server failure, database failover, and retries.

Security includes authentication, authorization, data protection, and HTTPS. Observability means using logs, metrics, traces, and alerts to understand system behavior. Then discuss trade-offs. SQL can offer strong consistency, while NoSQL can support high scale for suitable workloads. Cache can make reads faster but may return stale data. Background work gives better reliability but may finish later. Replication can improve availability but increases cost.

Engineering Considerations / Design Trade-offs

The benefit is that each part has a clear job. Cache can make repeated reads faster. CDN can serve static files without sending every request through the main application path. Message Queue lets some work happen in the background. The downside is more moving parts. More parts can cost more and need more monitoring. SQL can give strong consistency, while NoSQL may scale more easily for some workloads. Cache can be fast but may show stale data. Background work may finish later. Replication can improve availability, but it also increases cost. These choices depend on the requirements.

Why Interviewers Ask This

Interviewers ask this question to see how you organize a large problem. They want to know whether you start with users and requirements before choosing technology. They also check whether you can estimate scale, define APIs and data, draw a simple architecture, and discuss failures and trade-offs. The goal is not memorizing one diagram. It is showing clear thinking, good judgment, and simple communication.

Interviewer may ask next

How would this design change if photo traffic grew ten times larger?

I would keep the same basic architecture, but I would pay more attention to the photo path. The Mobile App and Web App would still use the Load Balancer and API Gateway for normal application requests. The Photo Service would still connect to Photo Storage. The CDN would become more important because the diagram uses it for static files coming from Photo Storage.

I would watch the bottlenecks already shown in the design. Storage and network limits could become important first. Hot keys could also create uneven load. Logs, metrics, traces, and alerts would help show where pressure is growing.

I would not replace the existing services just because traffic increased. I would first measure which part is limiting the system. This keeps the design simple and follows the original architecture. The main downside is cost. Higher traffic means more storage, network use, monitoring, and operating work.

What should we discuss if Cache (Redis) becomes unavailable during heavy traffic?

I would first explain that losing Cache removes one supporting path shown in the diagram. The main application structure still contains the Load Balancer, API Gateway, application services, and their data stores. User Service still connects to User DB. Photo Service still connects to Photo Storage. Feed Service still connects to Feed DB.

The important concern is extra load. Requests that depended on Cache may become slower or put more pressure on other parts of the system. I would watch logs, metrics, traces, and alerts because those are the observability tools shown in the diagram. They help us see whether databases, storage, or network limits are becoming bottlenecks.

After Cache is healthy again, it can return to its supporting role. The main downside is lower performance while it is unavailable. Heavy traffic can make that slowdown more serious.

124. What is a microservice?System DesignEasy

Question Details

Define a microservice as a small independently deployable service centered on a focused business capability. Compare microservices with a modular Python monolith, and explain service boundaries, APIs or events, data ownership, deployment, scaling, observability, network failures, consistency, and operational cost. State clearly that microservices are a tradeoff rather than a default.

Short Interview Answer (30-60 seconds)

At a high level, a microservice is a small service focused on one business capability. The main challenge is getting independent deployment and scaling without creating too much operational complexity. I would explain this by comparing a modular Python monolith with separate microservices, then looking at communication, data ownership, and operations. Microservices can use APIs or events, own separate data, and scale independently. The trade-off is more network failures, harder consistency, and higher operating cost.

Detailed Explanation

A microservice is a small part of an application that handles one clear business job. An online store might separate User, Product, Order, Payment, and Notification work. The difficult part is deciding whether that separation gives enough value to justify the extra work. A modular Python monolith keeps those capabilities inside one application. Microservices move them into independently deployable services. I would explain the monolith first, then the service boundaries, communication, data ownership, deployment, scaling, failures, and operating cost.

Useful Questions to Ask the Interviewer

How large is the application and engineering team?
Do different business areas need independent deployments?
Do some parts need much more scaling than others?
Is fault isolation important between different business capabilities?
Does the team already have strong monitoring and operations practices?

How to Explain It in an Interview

1. Start with the modular Python monolith

I would start with the simpler design. In the diagram, User, Product, Order, Payment, and Notification are modules inside one Python application.

They run in one process and share one database. Modules can call each other directly inside the application. This makes the system easier to build, deploy, and operate.

The downside is that the whole application is one deployment unit. Scaling or changing only one part is harder.

2. Explain the microservice boundary

A microservice takes one focused business capability and makes it an independent service. The diagram separates User, Product, Order, Payment, and Notification into different services.

Each service has a clear boundary. Each service also owns its own database. This means the User Service owns User DB, while Order Service owns Order DB, and so on.

Independent ownership gives teams more freedom. It can also provide technology flexibility because services are separated.

3. Explain APIs, events, and data ownership

Once services are separate, they need network communication. The diagram shows APIs, such as HTTP calls, and asynchronous events.

Client, web, or mobile requests enter through the API Gateway. The gateway sends each request to the needed service. Services can also communicate through the Message Broker or Event Bus for events such as OrderCreated and PaymentReceived.

The important idea is that each service owns its own data. Unlike the monolith, there is no single shared database for every business capability.

4. Explain deployment, scaling, and observability

The main benefit is independent deployment. One service can be deployed without deploying the whole application.

Scaling also becomes more focused. If Order Service needs more capacity, that service can scale without scaling every other service.

Observability also becomes more detailed. Each service needs its own logs and metrics. Tracing helps follow one request across several services.

5. Explain failures, consistency, and the trade-off

The downside is that network calls can fail. The diagram shows timeouts, retries, and circuit breakers as ways to handle those failures.

Data consistency is also harder because each service owns separate data. Some updates may use eventual consistency, which means services can agree after a short delay. The diagram also mentions sagas for coordinating work across several services.

Microservices cost more to build and run. There are more deployments, databases, network calls, monitoring needs, and failure cases. They can give better scalability and fault isolation, but they are a trade-off, not a default. I would start with a modular monolith and move to microservices when independent scaling, deployment, team growth, or fault isolation clearly justify the extra complexity.

Engineering Considerations / Design Trade-offs

The benefit is that each service can deploy and scale independently. A busy Order Service can grow without scaling the whole application. Problems may also stay inside one service, which improves fault isolation. The downside is more complexity. Network calls can time out or fail. Separate databases make some updates harder to keep in sync. Each service also needs logs, metrics, tracing, and monitoring. Operating many services costs more time and infrastructure. Microservices are therefore a trade-off, not a default. Start with a modular monolith, then split services when the benefits clearly outweigh the added complexity.

Why Interviewers Ask This

Interviewers want to see whether you understand why microservices exist, not just their definition. They look for judgment about service boundaries, data ownership, deployment, scaling, monitoring, network failures, and consistency. They also want to see whether you understand the simpler modular monolith option. A strong answer explains both the benefits and the extra operational cost, then chooses microservices only when those benefits are worth it.

Interviewer may ask next

What would you do if the Order Service suddenly needed much more scaling than the other services?

I would keep the same microservices design and give the Order Service more capacity. Independent scaling is one of the main benefits shown in the diagram.

The API Gateway would still send order requests to the Order Service. User, Product, Payment, and Notification services would not need the same scaling unless their own load also increased.

I would watch Order Service logs, metrics, and traces closely. Its Order DB also needs enough capacity because that service owns its own data. If higher order traffic creates more events, I would also watch the Message Broker or Event Bus.

The service boundary does not change, so the design stays conceptually the same. The main downside is higher operating cost. More capacity also creates more monitoring work and can increase traffic toward services that Order Service depends on.

What happens if the Payment Service is temporarily unavailable while an order is being processed?

I would treat that as a normal network failure that the system must handle. A call to Payment Service may time out because separate services communicate over the network.

The diagram shows retries and circuit breakers. A retry means trying the request again when the problem may be temporary. A circuit breaker stops repeated calls when a service appears unhealthy, which prevents more failed traffic from piling up.

The services may also communicate through the Message Broker or Event Bus. Events such as OrderCreated and PaymentReceived let related work happen without requiring every step to finish at the same moment.

Because each service owns separate data, some information may become consistent after a short delay. A saga can coordinate work across services when needed. The main downside is added complexity. The team must handle partial failures and monitor the workflow carefully.

125. Design a batch inference API for a GPU cluster.System DesignHard

Question Details

Design an API that accepts large batches of machine-learning inference jobs and runs them on a shared GPU cluster. Explain job submission, durable queues, scheduling, batching, GPU allocation, retries, idempotency, result storage, tenant quotas, backpressure, monitoring, and recovery when a worker or GPU node fails.

Short Interview Answer (30-60 seconds)

At a high level, this system accepts large inference jobs and returns a job ID quickly. The main challenge is to protect the shared GPU cluster while many tenants submit work. I would explain it in three flows: job submission, GPU processing, and result lookup. The API validates the job, checks tenant quota, saves metadata, and puts work in a Durable Job Queue. Workers run inference on GPU Nodes, store results, and retry failures. The trade-off is that larger batches use GPUs better, but users may wait longer.

Detailed Explanation

The goal is to accept large machine-learning inference jobs and run them safely on a shared GPU cluster. The hard part is that GPUs are expensive and limited. The system must return quickly, protect tenants from each other, and recover when workers or GPU nodes fail. The diagram solves this by separating job submission, queued GPU processing, result lookup, and recovery.

Useful Questions to Ask the Interviewer

Which user flows and system capabilities are required for the first version?
What traffic, data volume, latency, and availability targets should I design for?
Which consistency, security, geographic, and cost constraints matter most?

Design a batch inference API for a GPU cluster. diagram

How to Explain It in an Interview

1. Start with the main idea

I would start by saying that this is a queued processing system. The client should not wait while the whole batch runs on GPUs. Instead, the API accepts the job, saves its state, and returns a job ID.

This keeps the request path short. It also lets the system handle traffic spikes. The Durable Job Queue absorbs extra work when many jobs arrive at once.

2. Explain job submission

The Client sends a batch job to the API Gateway. The gateway forwards it to Auth + Tenant Quotas. This step checks who the tenant is and whether they have capacity left.

The Idempotency Store handles repeated submits. An idempotency key means the same request can be sent again safely. If the client retries, the store can return the same job ID.

After that, the Job Validator checks the job payload. It saves job metadata in the Job Metadata Store. Then it puts the job into the Durable Job Queue and returns a job ID.

3. Explain GPU processing

The Scheduler pulls ready jobs from the Durable Job Queue. It sends compatible work to the Batch Builder. Batching means grouping jobs so GPUs do more useful work at once.

The GPU Worker Pool runs inference on GPU Nodes. A GPU Node is the machine that has the GPU hardware. When inference finishes, the worker writes results and marks the job complete.

This design protects GPUs from direct client traffic. Clients never talk to workers or GPU nodes directly. That keeps scheduling, quotas, and recovery under service control.

4. Explain result lookup

The client later asks for job status through the Status API. The Status API reads the Job Metadata Store. If the job is complete, it uses the Result Reader to fetch the output.

The result itself is treated like a stored output file. The Result Reader returns it to the same Client. This keeps reads separate from GPU execution.

5. Explain recovery, backpressure, and monitoring

If a worker times out, the Retry Controller handles the failure. It can put the job back into the Durable Job Queue with backoff. Backoff means waiting longer before trying again.

If retries are exhausted, the job goes to the DLQ. A DLQ is a dead-letter queue for failed work that needs inspection. The system also records failure state so clients can see what happened.

Backpressure protects the cluster during spikes. The Scheduler sends a queue pressure signal to Auth + Tenant Quotas. Then the API Gateway can slow or reject excess jobs.

Observability & Monitoring tracks request rate, queue depth, queue wait, batch size, GPU utilization, worker health, retries, DLQ count, errors, and request ID traces. The main trade-off is batch size. Bigger batches improve GPU efficiency, but they add queue wait.

Engineering Considerations / Design Trade-offs

The benefit is that the API returns quickly with a job ID. The heavy GPU work happens after the request is accepted. The Durable Job Queue helps during traffic spikes because it stores work until workers are ready. The downside is that users must poll for status or result later. Larger batches use GPUs better, but they can make jobs wait longer in the queue. Strict tenant quotas protect the shared cluster, but they may reject bursty tenants. Retries improve recovery, but repeated failures need a DLQ for later review.

Why Interviewers Ask This

Interviewers ask this to see if the candidate can separate a fast API path from heavy background work. They want to see judgment around queues, scheduling, batching, GPU capacity, and retries. They also want to know if the candidate protects shared resources with tenant quotas and backpressure. A strong answer explains trade-offs clearly instead of only listing components.

Interviewer may ask next

How would you change the design if one tenant submits many huge jobs and hurts other tenants?

I would keep the same basic design, but I would make tenant quotas stronger. Auth + Tenant Quotas would check both request rate and queued work per tenant. The Scheduler would also consider tenant fairness before picking jobs from the Durable Job Queue.

This means one large tenant cannot fill all GPU capacity. The Scheduler can reserve some capacity for other tenants or use weighted scheduling. Weighted scheduling means bigger tenants can get more capacity, but not all of it. It also lets small tenants keep making progress during a large batch spike.

Backpressure would still flow from Scheduler to Auth + Tenant Quotas and then to API Gateway. If the tenant is over limit, the gateway can slow or reject new jobs. The downside is more scheduling complexity and possibly lower GPU utilization. Some GPUs may wait briefly while the scheduler protects fairness.

What changes if GPU nodes fail while a batch is running?

I would keep the same flow and make the Retry Controller handle the failed work. The GPU Node failure is sent to the Retry Controller. The controller records the failure state in the Job Metadata Store and sends safe work back to the Durable Job Queue.

This keeps the client-facing API simple. The client still polls the Status API and sees the current job state. If the job succeeds after retry, the worker writes results and marks the job complete. The retry should use the same job ID, so the client does not see a duplicate job.

If the job keeps failing, the Retry Controller sends it to the DLQ. That prevents endless retries from wasting GPUs. The downside is that some jobs may finish later because they must wait for retry. A very sick GPU node may also reduce cluster capacity until it is replaced.

126. Design a system for an LLM responding to user queries.System DesignHard

Question Details

Design a production service that accepts user prompts and returns responses from a large language model. Explain request routing, prompt validation, model serving, context management, streaming responses, rate limiting, caching, observability, safety checks, failure handling, and scaling during traffic spikes.

Short Interview Answer (30-60 seconds)

At a high level, this system accepts a user prompt and returns a safe streaming LLM response. The main challenge is balancing speed, safety, cost, and model quality. I would explain it in three flows: the synchronous request flow, the cache and context flow, and the background event flow. The request passes through validation, safety checks, orchestration, GPU serving, output checks, formatting, and streaming. The main trade-off is that bigger models improve quality, but they cost more and add latency.

Detailed Explanation

The goal is to take a user prompt, process it safely, and stream a useful LLM response back to the client. The hard part is that the system must feel fast while still checking safety, controlling cost, and protecting GPU capacity. The diagram solves this by separating the direct request path, the cache and context path, and the background event path.

Useful Questions to Ask the Interviewer

Which user flows and system capabilities are required for the first version?
What traffic, data volume, latency, and availability targets should I design for?
Which consistency, security, geographic, and cost constraints matter most?

Design a system for an LLM responding to user queries. diagram

How to Explain It in an Interview

1. Explain the goal and the main idea

I would start by saying that the system receives prompts and returns safe streaming answers. The user should not wait for slow background work. The main response path must stay focused on validation, model selection, generation, safety, and streaming.

The design also separates work that can happen later. Usage tracking, analytics, audit logs, and evaluation go through the Event Bus. This keeps the main answer path simpler and faster.

2. Explain the main request flow

The Client sends the prompt to the API Gateway. The API Gateway is the front door of the system. Then Auth + Rate Limits checks who the user is and whether they are sending too many requests.

Next, the Prompt Validator checks whether the request is shaped correctly. Input Safety Checks look for unsafe or blocked input before the model sees it. After that, the LLM Orchestrator controls the main decision flow.

The request then goes to the Inference Queue. The queue protects GPU Model Serving from too much work at once. GPU Model Serving runs the selected model and produces the answer.

3. Explain cache, context, and model routing

Inside the LLM Orchestrator, the Response Cache is checked first. On a cache hit, the system can reuse a safe answer and send it to Output Safety Checks. This saves GPU cost and reduces delay.

If the cache misses, the system builds context. The Context Manager reads from Context Sources. Those sources include Conversation Store, Read Replicas, Object Storage, and Optional RAG Search.

Then the Prompt Builder creates the final prompt. The Model Router chooses which model should answer. The Configuration Store provides model versions and routing policy.

4. Explain output safety and streaming

After GPU Model Serving creates the answer, Output Safety Checks review it. This step helps stop unsafe responses before the user sees them. Then the Response Formatter prepares the answer for delivery.

The Streaming Gateway sends the response back to the Client. Streaming means the user can see tokens as they arrive. This improves the user experience because the user does not wait for the full answer.

5. Explain background events and failures

The Event Bus handles work that should not block the response. LLM Orchestrator can send events to it. Output Safety Checks also send a safety decision event.

Feedback API receives user ratings or corrections from the Client. That feedback goes to the Event Bus. The Event Bus sends events to Usage & Billing, Analytics, Audit Logs, and Evaluation & Feedback.

Failed Events uses Retry with Backoff and DLQ. Retry with backoff means the system waits longer between retry attempts. DLQ means dead letter queue, which stores events that could not be processed.

6. Explain scale, security, and trade-offs

The Inference Queue protects GPUs from sudden traffic spikes. The scheduler can batch requests, which means it groups work together. The API layer and model workers can scale when traffic grows.

Security comes from input safety, output safety, PII redaction, tenant isolation, and audit logs. PII means private user information, such as names or phone numbers. Tenant isolation keeps one customer’s data separate from another customer’s data.

The trade-off is cost versus quality and speed. Bigger models usually answer better, but they cost more and take longer. Caching lowers cost, but cached answers must be safe to reuse.

Engineering Considerations / Design Trade-offs

The benefit is that the system separates fast user work from background work. The user gets a streaming response while billing, analytics, audit logs, and evaluation happen through the Event Bus. The cache can lower cost because it avoids GPU work for safe reusable answers. The downside is that cached answers must be checked carefully. The Inference Queue protects GPUs, but it can add wait time. Bigger models may improve answer quality, but they also increase cost and latency. We accept these trade-offs because safety and predictable performance matter in production.

Why Interviewers Ask This

Interviewers ask this to see how a candidate breaks a large LLM system into clear flows. They want to know if the candidate can balance safety, latency, cost, scaling, and reliability. They also check whether the candidate understands caching, queues, streaming, observability, and background processing. A strong answer explains the trade-offs without adding unnecessary complexity.

Interviewer may ask next

How would the design change if traffic suddenly doubled during peak hours?

I would keep the same basic design, but I would focus on protecting the GPU Model Serving layer. The Inference Queue becomes more important because it controls how much work reaches the GPUs at once.

I would autoscale the API layer and model workers. Autoscale means adding more running copies when traffic grows. The scheduler can also batch requests, which means it groups similar work so GPUs are used more efficiently.

The Response Cache also helps during traffic spikes. If many users ask similar questions, cache hits can bypass the expensive generation path. Those cached answers still go through Output Safety Checks before returning.

Correctness is kept because the same validation, safety, and formatting steps remain in place. The downside is that queue wait time may increase if GPU demand grows faster than capacity.

How would you handle unsafe model output in this system?

I would keep Output Safety Checks as the main guard before the answer reaches the user. This part of the diagram is important because the model may produce unsafe text even when the input was allowed.

If Output Safety Checks reject the answer, the system should not stream that unsafe text. It can return a safe refusal or a safer formatted response, depending on policy. The safety decision should also be sent to the Event Bus.

That event can feed Audit Logs and Evaluation & Feedback. Audit Logs help explain what happened later. Evaluation & Feedback helps improve future behavior.

Correctness is kept by placing safety before Response Formatter and Streaming Gateway. The downside is extra latency because the answer must be checked before it is returned.

127. Design an online collaborator platform?System DesignHard

Question Details

Design an online collaboration platform where many users can create or join a shared document and edit it at the same time. The system should show live changes, user presence, cursors, comments, version history, and reconnect users after a network failure. Explain how edits are ordered and merged, how duplicate operations are prevented, how document state is stored, how WebSocket connections are scaled, how offline edits are handled, and how the system recovers when a collaboration server fails.

Short Interview Answer (30-60 seconds)

At a high level, this system lets many users edit the same document in real time. The hard part is keeping every user in the same order while handling reconnects and server failures. I would explain it in four parts: route each document to one Session Owner, commit edits to the Operation Log, broadcast live updates, and recover clients from snapshots plus missed operations. The main trade-off is strong ordering versus more coordination.

Detailed Explanation

The goal is to let many users edit one shared document and see changes quickly. The difficult part is keeping every client in the same order when edits happen at the same time. The system must also handle comments, presence, offline work, reconnects, and server failures. The diagram solves this with one Session Owner per document, Operational Transformation, a durable Operation Log, snapshots, and short-lived connection and presence stores.

Useful Questions to Ask the Interviewer

Which user flows and system capabilities are required for the first version?
What traffic, data volume, latency, and availability targets should I design for?
Which consistency, security, geographic, and cost constraints matter most?

Design an online collaborator platform? diagram

How to Explain It in an Interview

1. Create or join the document

A user creates a document or joins an existing one through the WebSocket Gateway. The gateway checks authentication, permissions, request limits, and tenant rules.

The Document Router hashes the document_id. It sends every connection for that document to the same Collaboration Session Owner.

The client receives the current snapshot and any operations after the snapshot sequence. This gives the client the latest document state before live editing starts.

The Connection Registry stores which WebSocket node owns each user and device connection. This is short-lived routing data and is updated when clients reconnect.

2. Keep one safe owner for each document

Each Session Owner holds a lease for one document. The lease contains document_id, owner_epoch, and lease_expiry.

The owner renews the lease with heartbeats. If renewal fails, it must stop accepting new edits.

When another node takes over, it receives a higher owner_epoch. Only the newest epoch may append operations. This stops an old server from writing after a network delay.

3. Order and commit edits

The client sends an edit with operation_id and base_sequence. The Session Owner checks permissions and ignores duplicate operation IDs.

The service uses Operational Transformation, or OT. OT changes a new edit so it still works after other users' earlier edits.

The owner assigns the next document sequence. It appends the operation to the Operation Log with document_id, sequence, owner_epoch, and operation_id.

The Operation Log is the source of truth. It rejects writes from an older owner_epoch. The sender is acknowledged only after the log commits the operation.

The Session Owner broadcasts the committed operation directly after the log commit. It does not wait for the Operational Store update.

The current document state may update immediately after. If that update fails, it can be retried or rebuilt from the log.

4. Handle pressure and slow clients

Each document session uses a bounded input queue. The service limits operations per user and document, pending bytes per connection, and queue length.

If a limit is reached, the service returns a retry response such as 429. Clients that cannot receive updates fast enough may be slowed or disconnected.

These limits stop one busy document or slow client from hurting the whole system.

5. Handle presence, comments, and side work

Presence and cursor updates go to the Presence Store. This data is temporary and expires when heartbeats stop. It is not written to the permanent Operation Log.

Comments are saved in the Operational Store. They use their own IDs and timestamps. They do not use the document edit sequence.

After a comment is saved, it is broadcast to connected clients. Mentions can create notifications in the background.

Search indexing, notifications, audit logs, and analytics also run in the background. Their failure does not block the edit acknowledgement or live broadcast.

6. Reconnect and recover safely

A reconnecting client sends document_id, last_applied_sequence, and pending operation IDs. The server sends all missed operations after that sequence.

The client applies those operations first. Its offline edits are then transformed against the newer server edits and sent again.

The Snapshot Store creates snapshots from committed operations. Every snapshot stores snapshot_sequence, which is the last operation included. Recovery loads the snapshot and then applies later operations.

If a Session Owner fails, a new owner reads the last committed sequence from the Operation Log and takes over with a higher epoch. Uncommitted client edits may need to be resent. WebSocket nodes keep temporary connection state, but permanent document data stays in the log and stores.

Engineering Considerations / Design Trade-offs

The benefit is that every document has one clear edit order. The Operation Log keeps edits safe before users see them. The downside is that one Session Owner must handle all edits for one document, so very active documents can become hot. OT keeps edits consistent, but it adds transform work. Bounded queues and connection limits protect the system, but some clients may receive a retry response. Snapshots make loading faster, while background indexing and analytics may appear later.

Why Interviewers Ask This

The interviewer wants to see whether the candidate can handle real-time ordering, duplicate edits, reconnects, offline work, and safe failover. They also want to test WebSocket scaling, durable logs, snapshots, comments, presence, and conflict handling. A strong answer explains both the normal edit path and how the system recovers after a client or server failure.

Interviewer may ask next

What happens if the current Session Owner fails while users are editing?

A new Session Owner must take over safely. The Document Router selects another healthy node for that document.

The new owner receives a higher owner_epoch and a new lease. It reads the last committed sequence from the Operation Log. The log rejects any later write from the old owner because its epoch is now stale.

Connected users reconnect through the WebSocket Gateway. The Connection Registry is updated with their new WebSocket node. They send their last_applied_sequence and pending operation IDs.

The server returns missed committed operations first. Clients then rebase and resend any uncommitted edits.

The benefit is that committed edits are not lost. The downside is a short pause while clients reconnect, and some uncommitted edits may need to be sent again.

How would you handle edits made while a user is offline?

The client stores offline edits locally with unique operation IDs and the last known base_sequence.

When the connection returns, the client first asks for all committed operations after its last_applied_sequence. It applies those operations to reach the latest server state.

The offline edits are then transformed against the missed edits using OT. After that, the client sends them to the current Session Owner.

The owner checks duplicate operation IDs, assigns new document sequences, and commits them to the Operation Log. If an operation was already accepted before the disconnect, the duplicate check prevents it from being applied twice.

The benefit is that users can keep working offline. The downside is that large offline changes may need more transform work and may create visible conflicts.

128. Design a scalable URL-shortening service?System DesignHard

Question Details

Design a scalable URL-shortening service similar to Bitly or TinyURL. The system must generate unique short links, redirect users to the original URLs with low latency, support optional custom aliases and expiration dates, prevent alias collisions, and collect click analytics asynchronously. Explain how you would generate globally unique keys, partition and replicate the URL mappings, cache popular redirects, prevent database-to-cache inconsistency, apply rate limits, and preserve availability when cache or database nodes fail.

Short Interview Answer (30-60 seconds)

At a high level, this is a read-heavy system. Creating a short link must be correct, but redirects must be very fast because they happen much more often. I would explain it in three parts: create the short link, redirect the user, and record clicks in the background. The Metadata Database keeps the official mapping. The Redirect Cache makes reads fast. The trade-off is that cache or replica data may be a little old.

Detailed Explanation

The goal is to turn a long URL into a short code and return the original URL very quickly when someone opens that code. The hard part is keeping link creation correct while making redirects fast and available. The diagram solves this with a separate create path, redirect path, and analytics path.

Useful Questions to Ask the Interviewer

Which user flows and system capabilities are required for the first version?
What traffic, data volume, latency, and availability targets should I design for?
Which consistency, security, geographic, and cost constraints matter most?

Design a scalable URL-shortening service? diagram

How to Explain It in an Interview

1. Explain the main idea

This is mainly a read-heavy system. A link is created once, but users may open it many times.

The Metadata Database keeps the official mapping between the short code and long URL. The Redirect Cache is only used to make reads faster. Click analytics runs in the background, so it does not slow down the redirect.

2. Create the short link

The client sends a long URL. The request may also include a custom alias and an expiration time.

The API Gateway and Rate Limiter checks the request, applies rate limits, and blocks bad input. Then the URL Service decides how to create the short code.

If the user gives a custom alias, the service changes it to lowercase and reserves it with a UNIQUE rule in the Metadata Database. This prevents two users from using the same alias.

If there is no alias, the service creates a Snowflake-style number. This number uses time, a unique worker ID, and a sequence. The service then changes the number into Base62 text. Base62 only makes the code shorter. It does not make it unique.

The URL Service writes the mapping to the Metadata Database first. Only after the database write succeeds does it populate or invalidate the Redirect Cache. This keeps the cache from holding data that was never saved.

3. Redirect the user

When a user opens a short link, the request first reaches the Edge or CDN. It handles DNS, TLS, and DDoS protection. Then it sends the request to the Redirect Service.

The Redirect Service checks the Redirect Cache first. If the code is found and not expired, the service gets the long URL and expiration time. It then returns a 302 or 307 redirect.

If the cache misses or the entry has expired, the service reads from a Metadata DB replica. A replica is a read copy of the main database. It may be a little behind.

The service checks the expiration time again. If the link is valid, it returns the long URL and adds the result to the cache. The cache time must not go past the link's expiration time. If the link is missing or expired, the service returns 410 Gone or a custom page.

4. Record click analytics

After a valid redirect lookup, the Redirect Service sends a ClickRecorded event to the Event Queue. Workers process the event and store the result in the Analytics Store.

The create path may also send a separate ShortLinkCreated event. These are two different events. Because analytics runs in the background, slow reports do not delay the redirect.

5. Scale and handle failures

The Metadata Database is split by hash(short_code). The leader handles writes, while replicas handle reads. Replicas may have a small delay.

If the cache fails, the Redirect Service reads from replicas. Rate limits, circuit breakers, short timeouts, and request coalescing protect the database. Request coalescing means many requests for the same code share one database lookup.

If the database leader fails, writes may stop briefly while a new leader takes over. Redirects can still continue from the cache during that time. The system also validates URLs, blocks abuse, uses HTTPS, and applies separate rate limits to create and redirect requests.

Engineering Considerations / Design Trade-offs

The benefit is fast redirects because popular links come from the cache. The downside is that cache or replica data may be a little old. Writing to the database first keeps the mapping correct. The cache is updated only after that. A short cache time finds changes sooner, but it causes more database reads. Replicas improve availability, but they may be behind the leader. Analytics runs in the background, so click reports may appear later.

Why Interviewers Ask This

The interviewer wants to see whether the candidate can separate a write path from a much busier read path. They also want to check unique code generation, custom aliases, caching, expiration, database scaling, failure handling, rate limits, and background analytics. A strong answer keeps the data correct while making redirects fast and available.

Interviewer may ask next

How would you make a new short link work immediately if the read replicas are behind?

I would keep the same design, but I would change the first read after creation.

After the Metadata Database leader saves the new mapping, the URL Service can place it in the Redirect Cache before returning success. Then the new short link can work right away from the cache.

If the cache update fails, the first read can go to the leader instead of a replica. This is useful because replicas may be a little behind.

The leader still keeps the official data. The cache only helps with speed.

The benefit is that the user can open the new link immediately. The downside is more routing logic and a little more load on the leader.

What would you do if the Redirect Cache failed during heavy traffic?

The Redirect Service would read from the Metadata DB replicas instead.

But it should not send every request to the database without limits. That could overload the replicas.

I would use rate limits, short timeouts, and circuit breakers. A circuit breaker stops sending requests when the database is already failing.

I would also use request coalescing. If many users ask for the same short code, the service sends one database request. The other requests wait for the same result.

If the replicas are still too busy, the service may return a controlled 503 error for some requests. The benefit is that the database stays healthy. The downside is that some redirects may fail for a short time.

129. Design a distributed job scheduler?System DesignHard

Question Details

Design a distributed job scheduler in Python that lets users submit one-time and recurring jobs, cancel jobs, view status, and run jobs at their scheduled time. The system must support priorities, bounded retries with exponential backoff, worker heartbeats, execution leases, and safe recovery when a scheduler or worker fails. Explain how you would store schedules durably, prevent two workers from owning the same job, handle long-running and stuck jobs, apply backpressure and tenant quotas, and use Python processes, containers, or asyncio workers for different workload types.

Short Interview Answer (30-60 seconds)

At a high level, this system stores jobs safely and runs them at the right time. The hard part is preventing duplicate ownership while still recovering from crashes. I would explain it in four parts: submit and store the job, schedule due jobs, run them with leases, and report results. The Job Store keeps the official data. The trade-off is at-least-once execution, so a job may run more than once after a failure.

Detailed Explanation

The goal is to let users create one-time or recurring jobs and run them at the correct time. Users must also cancel jobs and check their status. The difficult part is handling crashes without losing jobs or letting two workers own the same job. The design solves this with durable storage, atomic scheduler claims, execution leases, heartbeats, retries, and separate worker types.

Useful Questions to Ask the Interviewer

Which user flows and system capabilities are required for the first version?
What traffic, data volume, latency, and availability targets should I design for?
Which consistency, security, geographic, and cost constraints matter most?

Design a distributed job scheduler? diagram

How to Explain It in an Interview

1. Submit and store the job

The user sends a create, cancel, or status request through the API Gateway. This layer checks the user, validates the request, applies rate limits, and checks the tenant quota.

For a new job, the Job Service saves the job and schedule in the Job Store. The first state is SCHEDULED. The Job Store is the source of truth, which means it keeps the official job data.

A one-time job has one planned run. A recurring job also stores its schedule and next_run_at. A JobCreated event may be sent for notifications or reporting, but it does not send the job to the Ready Queue.

2. Find jobs that are ready to run

The Scheduler Cluster reads jobs whose next_run_at is now or earlier. Several scheduler instances may read at the same time, so each job must be claimed with one atomic update.

The winning scheduler changes the state from SCHEDULED to QUEUED. It also saves claimed_at and claim_expiry. Only the winner sends the job to the Ready Queue.

If a scheduler crashes before queueing the job, another scheduler finds the expired claim. It can reset the job and try again. This lets the system recover claimed jobs after the scheduler comes back.

3. Run the job safely

A worker pulls a job only when it has free capacity. Before running it, the worker checks the latest job state. If the job is CANCELLED or already finished, the worker skips it.

If the job is valid, the worker gets an execution lease. A lease gives one worker temporary ownership. It includes a lease_version and lease_expiry.

The worker sends heartbeats while the job is running. A heartbeat extends the lease. Long-running jobs must keep sending them. If the worker stops sending heartbeats, the lease expires and another worker may try the job.

For a running job, the worker receives a cancellation request when the job type supports it. Stopping is best-effort, so the system cannot always stop the job immediately.

The worker type depends on the work. asyncio workers are good for network and database waiting. Python processes are better for CPU-heavy code. Containers are useful for isolated or dependency-heavy jobs. Simple blocking jobs can use normal synchronous workers.

4. Save results, retry failures, and update status

When the worker finishes, it reports the result with the lease_version. The Job Service checks that this version is still current. It accepts the current worker and rejects an old worker.

A successful job becomes SUCCEEDED. A cancelled job becomes CANCELLED. A failed job uses bounded retries. The Retry Manager calculates a later next_run_at with exponential backoff. If the job has used all allowed attempts, it becomes FAILED. Otherwise, it returns to SCHEDULED.

For a recurring job, success creates the next planned run. This is different from a retry. A retry repeats the same failed run.

JobStateChanged events go through the Event Bus. The Status View Updater uses them to update the Status Read DB. The Job Query Service reads that view when users ask for status.

5. Handle scale and failure

The Ready Queue is split by priority and tenant. Queue limits and worker concurrency provide backpressure, which means the system accepts only the work it can handle.

Tenant quotas stop one customer from using all workers. The Job Store uses a leader or consensus group for writes and replicas for availability. The main trade-off is at-least-once execution. A job may run twice after a crash, so job handlers should be safe for retries.

Engineering Considerations / Design Trade-offs

The benefit is that jobs are not lost when a scheduler or worker fails. Leases and heartbeats let another worker take over stuck work. The downside is that a job may run more than once after a failure. Job code should therefore be safe when repeated. Short leases find failures faster, but they need more heartbeats. Long leases use fewer heartbeats, but recovery is slower. Priority queues help urgent jobs, but fair ordering across many tenants is harder.

Why Interviewers Ask This

The interviewer wants to see whether the candidate can store schedules safely, stop two workers from owning one job, and recover from crashes. They also want to check retries, cancellation, recurring jobs, backpressure, tenant fairness, and Python worker choices. A strong answer explains both the normal flow and what happens when part of the system fails.

Interviewer may ask next

How would you handle a worker that finishes after its lease has already expired?

I would reject the old worker’s result. Every execution lease has a lease_version. The worker sends that version when it reports success or failure.

The Job Service reads the current lease from the Job Store. If the versions match, the result is accepted. If they do not match, the worker is stale and its update is rejected.

This protects the job state after another worker takes ownership. However, the old worker may already have made an outside change, such as sending an email. That is why the job handler should use a stable request key or check whether the action was already completed.

The benefit is safe job state. The downside is that outside actions may still happen twice unless the job code also protects them.

How would you stop one tenant from filling the whole queue?

I would apply limits before the job enters the system and again during scheduling. The API Gateway and Job Service check how many jobs the tenant may create. The Scheduler also checks how many jobs that tenant already has running or waiting.

The Ready Queue uses tenant partitions, priorities, and tenant quotas. This prevents one tenant from taking all queue and worker capacity. Workers also use bounded concurrency, so they accept only a safe number of jobs.

When the queue reaches its limit, the API can reject or slow new submissions according to the configured backpressure rule. This keeps the system stable during a traffic spike.

The benefit is fair use and better stability. The downside is more scheduling logic, and a tenant may wait even when another tenant is not using its full share.

130. When you are in a leadership role, how do you motivate team members?BehavioralHard

Question Details

Describe a real situation where you led or influenced Python developers without relying only on authority. Explain how you understood individual needs, clarified the goal, removed obstacles, encouraged ownership, handled low motivation, and measured the team's progress.

I motivate team members by first understanding what is making the work difficult for them. Then I connect each person’s work to a clear goal, give them useful ownership, remove blockers, and make progress visible without using pressure as the main tool.

Interview tip:

Use the STAR method. Explain how you understood each person, clarified the shared goal, removed obstacles, encouraged ownership, addressed low motivation, and tracked progress.

Situation

During a previous Python project, I helped lead a small development team that was improving a data processing service. The work had become repetitive, several technical issues were slowing us down, and one developer had become less engaged because tasks were being assigned without enough context. The team was still completing work, but discussions were quiet and progress was becoming less predictable.

Task

My responsibility was to help the team complete the planned improvements while rebuilding energy and ownership. I did not want to motivate people only by pushing deadlines. I needed to understand what each developer needed, explain why the work mattered, remove avoidable friction, and create a simple way to see whether the team was moving forward.

Action

I started with short individual conversations. I asked each developer what was slowing them down, which tasks they felt confident owning, and what type of support would help. I learned that one person wanted more challenging backend work, another needed clearer acceptance criteria, and the less engaged developer felt that decisions were being made before the team could contribute. I then explained the shared goal in practical terms. We were not only changing Python code. We were making the service easier to maintain and reducing failures during data processing. I divided the work into clear outcomes and invited developers to choose ownership where their interests and skills matched the need. I gave the less engaged developer ownership of reviewing the processing flow and proposing a safer error handling approach. This mattered because it gave that person a real decision to make instead of another isolated coding task. I also removed obstacles. I clarified unclear requirements, arranged a focused review for a difficult dependency, and created small examples that made expected behavior easier to test. During team check ins, I asked about progress and blockers instead of asking only whether tasks were finished. We tracked completed outcomes, open risks, and the next useful step. When motivation dropped, I addressed it privately and directly. I listened first, adjusted the task when the concern was reasonable, and explained any constraint that could not change. I also recognized useful contributions during team discussions, especially when someone prevented a defect or helped another developer, because those actions supported the whole team even when they did not produce a large visible feature.

Result

The team became more active in planning and review discussions, and work moved with fewer repeated questions. The developer who had been less engaged presented the new error handling approach and helped the team adopt it. We completed the planned improvements with clearer ownership and a more reliable development process. I learned that motivation is usually stronger when people understand the purpose, have a meaningful area to own, and can see that their concerns lead to practical action.

Why Interviewers Ask This

Interviewers ask this question to learn whether a candidate can lead through trust, clarity, and support instead of relying only on authority. A strong answer shows that the candidate understands individual needs, creates ownership, removes obstacles, handles low motivation respectfully, and uses visible progress to keep a team aligned.

Interviewer may ask next

How did you handle the developer who had become less engaged?

I spoke with the developer privately and asked what was causing the low engagement. I learned that the person felt excluded from technical decisions and was receiving tasks without enough context. I gave the developer ownership of reviewing the processing flow and proposing the error handling approach. I still set clear expectations, but I also made sure the person had a meaningful decision to own and regular support when blockers appeared.

How did you know your approach was improving the team's motivation?

I looked for changes in behavior and delivery rather than relying only on a general feeling. The team raised blockers earlier, contributed more during planning and code reviews, and needed fewer repeated clarifications. Ownership also became clearer because developers could explain their next step and the reason behind it. Those signs showed that the team was more engaged and that progress was becoming more predictable.

460 Python Developer Interview Questions & Answers

Questions with Detailed ExplanationsWith Detailed Explanations

121. Tell me how you would diagnose and fix a slow Python endpoint or background job.PerformanceHard

122. How do you decide between threading, multiprocessing, and asyncio for a Python workload?PerformanceHard

123. What is system design?System DesignEasy

124. What is a microservice?System DesignEasy

125. Design a batch inference API for a GPU cluster.System DesignHard

126. Design a system for an LLM responding to user queries.System DesignHard

127. Design an online collaborator platform?System DesignHard

128. Design a scalable URL-shortening service?System DesignHard

129. Design a distributed job scheduler?System DesignHard

130. When you are in a leadership role, how do you motivate team members?BehavioralHard