How grading works

The stages

Faultline runs your code as a separate program. It checks what your program does, not how you wrote it. A submission starts at the first stage. It stops at the first stage that does not pass.

• L1: Correctness. Your code must return the right answers.
• L2: Performance. Your code must be fast enough. It is compared with a working solution in your language. Skipping work cannot earn a pass.
• L3: Resilience. Your code must handle problems such as dropped connections.
• L4: Observability. Your counters must match what your code really did.

Not every challenge has all four levels. The challenge page shows the stages you need to pass.

Timing and retries

Timing checks allow small changes in how long work takes. Some checks run more than once. If only some runs pass, the result shows as FLAKY. Check for shared state bugs and assumptions about timing.

A problem with our service is never counted as your failure. We retry it automatically. Tell us if your submission stays queued for several minutes.

Checks and progress

Use faultline check to check your code. Every check saves your progress. Use faultline check --stage <id> to check only up to one stage. A failing result prints a command that repeats the same failing case. That run is saved in your history, but it does not update your progress. You can stop waiting with Ctrl-C. Your submission keeps running. Use faultline status <id> to read the result later.

Read Reading a verdict for help with a result.