Connection Pool

Track: Protocols and sockets. 5 stages. Languages: Go, Python, JavaScript.

Opening a new network connection is slow. A program that talks to a database many times a second should not open a fresh connection for every request. Instead, it keeps a few connections open and shares them. That shared set is a connection pool. Build one.

Your pool hands out real network connections to a real server. It never keeps more than a fixed number open at once. When a caller is done, the connection goes back into the pool so the next caller can use it again, without opening a new one. When the pool is full and every connection is busy, a new caller waits until one comes back, or until its own time runs out.

Implement these, using your language's own names for them:

Many callers use your pool at the same time, so it must stay correct when calls overlap. Standard library only.

The tester puts a small relay between your pool and the server, so it sees every connection you open and close, and it can make some connection attempts fail, hang, or answer slowly on purpose. It counts real connections, so a connection you forget to close shows up, even if your own counts say it was closed. For help with the ideas, read the notes. New to code that many callers run at the same time? Read Many callers at once first.

Where it is used

A pool that forgets to close a connection after a failed or hung attempt leaks it. After enough failures, the server runs out of connection slots and nobody can connect.

Stages

kp4: share 4 connections between 12 workers (easy)

12 workers share one pool with maxSize = 4. Each worker gets a connection 10 times in a row, uses it for 2 milliseconds, and gives it back. Each Acquire may wait up to 2 seconds.

pool = New(dial, 4)
worker A: Acquire -> new connection #1     (1 open)
...       Acquire x3 more -> #2, #3, #4    (4 open, the limit)
worker E: Acquire -> waits                 (all 4 are busy)
worker A: Release(#1)                      (#1 goes back, still open)
worker E: Acquire -> #1 again              (reused, nothing new opened)
end:      Close()                          (all 4 closed)
The tester will verify: never more than 4 connections open at once, 4 open at once at some point, no more than 4 opened in total for all 120 uses, every connection you hand out really works, and every connection is closed by the time Close() returns.

tp7: get and give back connections quickly (L2, hard)

Keep Acquire and Release fast. With maxSize = 8, a loop keeps 4 connections out and swaps them in and out 50,000 times, with no real network. Every swap must still hand back a connection from the pool. Example: if our solution takes 100 ms, the limit is 150 ms. Your solution takes 140 ms, so it passes. In Go, your code may also make at most twice as many memory allocations as ours.

The tester will verify: your time is at most 1.5 times our solution's time, and the connections you hand out show that the pool really reused them.

sv8: keep going when a connection attempt fails (L3, hard)

Networks fail for a moment. When dial fails, wait a short time, about 10 milliseconds, and try again, until the caller's time limit runs out. Two runs:

run 1: maxSize 2, one Acquire with 2 seconds to spare
       dial -> cut               (the tester cuts the first connection at once)
       dial -> works             -> Acquire returns a connection
run 2: maxSize 1, 6 Acquire calls one after another, 800 ms each
       the first 200 connections are all cut at once
       Acquire #1 keeps trying, runs out of time, returns an error
       a later Acquire -> works  (the failed call gave its space back)

A cut connection usually shows up as a "connection reset" error, sometimes as the connection closing before any reply. Treat both as a failed dial. Do not open connections before the first Acquire. The tester counts from the first attempt.

The tester will verify: the first cut attempt does not reach the caller, in run 2 an Acquire stops at its time limit and returns an error, a later Acquire still works after that, and every connection opened is closed by the time Close() returns.

sv9: keep going when a connection attempt hangs (L3, hard)

Sometimes a server accepts a connection and then never answers. If one attempt is allowed to wait for the caller's whole time limit, one bad attempt costs the whole call. Give each attempt its own shorter limit, between 200 and 500 milliseconds, and try again after it. The slow answer takes about 50 milliseconds. Leave extra time for scheduling and network delays. Use the caller's remaining time if it is shorter. Two runs:

run 1: maxSize 2, one Acquire with 2 seconds to spare
       dial -> hangs             (the tester accepts and never answers)
       dial -> works             -> Acquire returns a connection
run 2: the same, but every answer from the server arrives slowly, about 50 milliseconds
       dial -> works, slowly     -> Acquire returns a connection
The tester will verify: in run 1 Acquire still returns a connection, in run 2 a slow answer is not treated as a failure, and every connection opened is closed by the time Close() returns.

tm5: report your own numbers correctly (L4, medium)

The same work as kp4: 12 workers, maxSize = 4, 10 uses each. At the end, the tester compares your Metrics() with what it saw. Count a connection as opened only when dial really worked. Count a connection as handed out from the moment Acquire takes it until Release gives it back.

tester saw: 4 connections opened, at most 4 open at once
Metrics() -> dials 4, max_concurrent 4      (correct)
Metrics() -> dials 120, max_concurrent 4    (wrong: counted every Acquire)
The tester will verify: your opened count sits between the most connections the tester saw open at once and the total it saw opened, and your busiest moment sits between that same number and 4.

Rules

Read the Connection Pool notes in the wiki