Parallel email testing: fixing the shared-inbox race.

Your email tests pass with one worker and fail with eight. Searchers land here from "parallel email tests" and from "flaky email tests," and both queries describe the same bug — because the flakiness almost never comes from email being slow. It comes from email being shared.

The short answer. Email tests race in parallel because the inbox is shared mutable state. Give every test its own inbox — created in a fixture, deleted in teardown, with a short TTL as the backstop — and there is nothing left to race: the address itself becomes the correlation key, retries get a fresh mailbox, and no coordination or locking is needed at any worker count.

The race, reproduced

Here is the suite that produces it. One inbox, minted once, used everywhere — the "optimization" every email suite drifts toward:

import { MailFixture, Inbox } from 'mailfixture';

const mail = new MailFixture();

export let sharedInbox: Inbox;

export async function setupInbox() {
  sharedInbox = await mail.createInbox(); // one inbox for the whole run
}

const otp = await sharedInbox.waitForOtp({ timeout: 30_000 });
await page.fill('#otp', otp);

Run it serially and it's green. Run it with Playwright's default parallelism — spec files spread across workers — or pytest -n auto, and it fails a few percent of the time with errors that point everywhere except the real cause:

Error: expect(locator).toHaveText('Welcome')
  Received: "That code is invalid or has expired."

The interleaving is mundane. Worker 1 triggers a signup email; worker 3 triggers a password reset. Both are long-polling the same inbox. The signup email lands first, both waits resolve with it, and worker 3 submits a signup code to a reset form. Or worse: both flows produce six-digit codes, the codes are single-use, worker 1's test consumes worker 3's code, and worker 3 fails — a test that did nothing wrong, failed by a test that passed.

Three properties make this race uniquely nasty to diagnose:

1. It only reproduces under load. --workers=1 is green, so bisecting "fixes" it, and the bug gets filed as environmental flakiness.
2. The failing test is not the guilty test. The test that stole the message passes. Blame lands on whichever test lost the race, which changes run to run.
3. Timing hides it locally. On a laptop running four tests, the windows rarely overlap. In CI with twelve shards and slower email delivery, they always do.

If your suite has a qa+test@company.com address, an inbox minted in globalSetup, or a scope="session" inbox fixture, you have this bug — whether or not it has fired yet.

The decision: inbox-per-test vs inbox-per-worker

There are two honest ways to isolate parallel email tests. One of them is almost always right.

Inbox-per-test — every test mints a fresh address, uses it, deletes it. Isolation is total: no two tests can ever see each other's mail, because no two tests share an address. The recipient address is the correlation ID — you never ask "which message is mine?" because only yours can arrive. Retries are safe by construction (each attempt builds a new inbox). Teardown is local to the test. This is the default, and you should need a reason to leave it.

Inbox-per-worker — each parallel worker mints one inbox and its tests use it serially. The cross-worker race disappears (workers don't share an address), and you create fewer inboxes. But you inherit three problems inbox-per-test doesn't have:

• History bleeds between tests. Within a worker, test 7 waits on an inbox that already holds tests 1–6's messages. The SDK wait helpers (waitForOtp, waitForMessage, waitForLink) start each call from the inbox's full history — so a wait can match a previous test's email, including one that arrived late, after its own test already finished. You must clear() the inbox between tests or pass a match filter specific enough to exclude every earlier message — and since every message in the inbox has the same recipient, to: can't discriminate; you're left hoping subjects differ.
• Retries reuse the inbox. A retried test waits on an inbox still holding the failed attempt's messages, and can re-read the exact stale code that just failed.
• Worker-scoping is easy to get wrong. Playwright worker fixtures and pytest scope="session" under xdist both look like per-worker scoping and drift into per-run scoping with one config change.

When is inbox-per-worker worth it? Not for correctness — only as a throughput trade when your parallelism genuinely outruns the inbox-creation rate limit (the cost section below has the arithmetic). If you do choose it, treat clear() between tests as mandatory, not optional.

What is never right is inbox-per-suite: one address in globalSetup, a session-scoped fixture, or a hardcoded shared mailbox. That's the reproduction case above.

Inbox-per-test, implemented

The pattern is the same in any runner: mint in setup, delete in teardown, keep both in one place so no spec ever constructs its own inbox.

Playwright — a test.extend fixture. Test-scoped, deliberately: a worker-scoped fixture would share one inbox across every test that worker runs, which is the disease this guide exists to cure.

import { test as base } from '@playwright/test';
import { MailFixture, Inbox } from 'mailfixture';

const mail = new MailFixture(); // reads MAILFIXTURE_API_KEY — Node-side, specs run here anyway

export const test = base.extend<{ inbox: Inbox }>({
  inbox: async ({}, use) => {
    const inbox = await mail.createInbox({ ttlSeconds: 900 });
    await use(inbox);
    await inbox.delete(); // teardown runs pass or fail; TTL covers aborted runs
  },
});
export { expect } from '@playwright/test';

import { test, expect } from './fixtures';

test('signup sends a one-time code', async ({ page, inbox }) => {
  await page.goto('/signup');
  await page.fill('#email', inbox.address);
  await page.click('text=Send code');

  const otp = await inbox.waitForOtp({ timeout: 30_000 }); // server-held long-poll
  await page.fill('#otp', otp);
  await expect(page.locator('h1')).toHaveText('Welcome');
});

pytest — a function-scoped fixture in conftest.py. Under pytest -n auto, each xdist worker is a separate process running this fixture independently; because no inbox outlives its test, workers need zero coordination.

import pytest
from mailfixture import MailFixture

@pytest.fixture(scope="session")
def mail():
    return MailFixture()  # reads MAILFIXTURE_API_KEY

@pytest.fixture
def inbox(mail):
    inbox = mail.create_inbox(ttl_seconds=900)
    yield inbox
    inbox.delete()  # runs pass or fail; TTL covers killed runs

Note the units: the JS SDK counts timeouts in milliseconds, the Python SDK in seconds. timeout=30_000 transplanted into Python is eight hours, and pytest imposes no per-test time limit to catch it.

The waits are server-held long-polls, which matters for parallelism twice over. First, "wait for the code" is one request held open until the message arrives, not a polling loop — so forty concurrent tests are forty idle connections, not forty tests hammering a list endpoint into a rate limit. Second, the timeout is a ceiling, not a duration: waits resolve the moment mail lands, so parallel suites don't stack sleep time.

Retries and reruns

Retries are where partially-fixed suites relapse, because a retry replays the trigger — your app sends a second email — against whatever inbox state the first attempt left behind.

With inbox-per-test, both major runners rebuild fixtures on retry, and that's the whole fix:

• Playwright (retries: 2): each attempt sets up its fixtures from scratch. Attempt two gets a brand-new inbox and cannot re-read the code attempt one consumed.
• pytest (pytest-rerunfailures, --reruns 2): each rerun tears down and rebuilds function-scoped fixtures. Same guarantee.

Both guarantees hold only if the inbox comes from the test-scoped fixture. An inbox minted in globalSetup, at module import, or in a session/worker-scoped fixture is the same inbox on every attempt — the retry re-waits on a mailbox that already contains the failed attempt's message, matches it instantly, submits a consumed code, and fails again. A test that fails identically on every retry attempt while looking timing-related is this bug's signature.

One more retry interaction: give the runner a longer leash than the SDK. Playwright's default test timeout is 30 seconds — exactly the waitForOtp deadline above — so a slow email dies as "Test timeout of 30000ms exceeded" instead of the SDK's MailFixtureTimeout, which is the error that names the real failure. Set the test timeout above the SDK deadline (and in pytest, if you use pytest-timeout, the same rule applies).

Correlation: match filters and since cursors

Isolation by address is the primary correlation mechanism, but two API-level tools sharpen it:

match filters. Message listing and every wait helper accept a match string with subject:, from:, and to: prefixes (a bare term matches the subject). Even with a dedicated inbox, match earns its keep the moment one test triggers two emails — a welcome mail and an OTP mail, say:

const msg = await inbox.waitForMessage({
  match: 'subject:verification',
  timeout: 30_000,
});

The filter is applied server-side, inside the long-poll — non-matching messages don't wake your wait.

since cursors. Every message summary carries a received_at timestamp usable verbatim as a since cursor on the list endpoint. Within a single wait call, the SDK tracks this internally: in a resend flow, one waitForOtp call that inspects and skips a message will not re-match it. What the email wait helpers do not take is a caller-supplied since — each new call starts from the inbox's full history. Between separate waits on a long-lived inbox, that's exactly how yesterday's message gets matched — one more reason short-lived per-test inboxes beat clever cursor bookkeeping. (If you manage cursors yourself via messages({ since }), you're rebuilding what per-test isolation gives you for free.)

Deterministic teardown

Parallel suites multiply leftovers, so cleanup needs to be layered, not hopeful:

1. Delete in teardown. inbox.delete() in the fixture's teardown position runs whether the test passed or failed, and drops the inbox and its messages immediately. That's the deterministic path.
2. TTL as the backstop. Teardown doesn't run when CI kills the job, the runner OOMs, or someone hits cancel. ttlSeconds: 900 means an orphaned inbox stops accepting mail when the TTL passes, stops counting against your active-inbox cap, and is swept — with its messages — by an hourly cleanup. A TTL is not a substitute for teardown; it's the insurance for runs that never reach it.
3. clear() is not teardown. It drops messages but keeps the address alive and counting against your cap. It's the between-tests tool for the per-worker pattern, not an end-of-test tool.

Deletion is immediate and permanent — no retention grace. For a test inbox that's the point: no state survives into the next run.

What parallelism costs

Inbox-per-test sounds expensive. Run the numbers; mostly it isn't.

Active-inbox caps count concurrently existing inboxes — 25 on Free, 250 on Solo, 1,000 on Team, 5,000 on Scale. Creation past the cap answers 403. But with delete-on-teardown plus a TTL, a parallel suite's concurrent count is roughly its worker count: 16 workers ≈ 16 live inboxes at any instant, comfortably inside even the free cap. Suites hit the cap by leaking — skipping teardown and setting no TTL — not by parallelism.

Creation velocity is rate-limited per API key: bursts to 10, refilling at 10 per minute — it's an abuse control, not a throughput knob. The arithmetic: sustained, that's one inbox every six seconds, so a suite stays under it while its average test outlasts worker count × 6 seconds. Eight workers running minute-long browser tests sit comfortably below the line; sixteen workers running fast API-only tests will cross it. When a create does trip the limit it answers 429 with a Retry-After header — the SDKs absorb 429s inside wait helpers, but a 429 on create surfaces as a MailFixtureError carrying retryAfter, worth honoring in the fixture rather than retrying blind. If your parallelism genuinely outruns the refill rate, that's the one honest case for the per-worker pattern above.

Message volume is the real budget line. Parallel suites don't receive more email per test, but they make running the whole suite cheap enough to do constantly — and every received message counts against daily and monthly plan quotas. A 200-test suite that triggers one email each, run five times a day, is a thousand messages a day. Size the plan for suite-runs-per-day, not tests-per-suite.

Failure modes

What this doesn't fix

Honesty about the boundary: per-test inboxes remove the inbox as shared state. They do not parallelize the rest of your stack.

• Your app can still race itself. Per-IP or per-account send throttles, "one active OTP at a time" logic, global signup rate limits — forty parallel signups will find them. Those failures are real findings about the system under test, not test flakiness; don't paper over them with serialization until you know which they are.
• Receive-only. MailFixture receives mail; it never sends any. Your application (or its email provider) must actually deliver — a test can't fake the inbound message it then asserts on, and that's the point.
• Delivery ≠ rendering. These tests prove content, codes, and links arrived. They prove nothing about how a mail client renders the message.

Where to go next

• The runner-specific architecture, including the rules this guide compressed: Playwright's test.extend fixture, pytest's conftest version (with the xdist and rerun specifics), and the cy.task bridge for Cypress.
• The anatomy of OTP flakes beyond parallelism — sleeps, regexes, and typed extraction: the OTP guide.
• The whole mock / capture / real-inbox decision, before any framework specifics: the automated email testing guide.

If your suite is already parallel and already flaky, the fifteen-line fixture above is usually the whole migration: delete the shared-inbox helper, and let the compiler or the import error walk you to every spec that needs the fixture instead.