GUIDES / LOGIN ALERTS

Testing login-alert emails, end to end.

“New sign-in to your account from Chrome on Linux near Amsterdam, NL.” Every serious product sends this email; almost no test suite covers it. The excuse is real: the alert's whole payload — city, country, IP — is derived from where the login came from, and in CI that's a moving target. Here's how to pin it down, and the one assertion that catches the bug every alert pipeline eventually ships.

Why login-alert tests flake

1. Fixed sleeps. The baseline disease of every email flow: waitForTimeout(5000) is red when the alert takes six seconds and wasted time when it takes one. Clear this first; nothing below works on top of a sleep.

2. The hardcoded city. The location in the alert is computed from the sign-in request's source IP — which, in CI, is your runner's egress IP. Write expect(body).toContain('Ashburn') and you're asserting where your CI vendor bought servers this quarter. The vendor shifts a region, the test goes red, someone deletes the assertion — and now the most security-relevant line of the email is untested.

3. One shared inbox, many alerts. Every test that logs in fires an alert at the same mailbox, so “wait for the alert” happily grabs one from a parallel worker. And because the alert describes an account event, inbox-per-test only helps if the account is fresh too — the same account-per-test move password-reset flows need, with a private inbox making the throwaway registration cheap.

The happy path, properly

A fresh inbox, a throwaway user, a long-poll instead of a sleep, and structural assertions that hold no matter where the runner is: the device line rendered, and none of the null-shaped garbage — undefined, None, an empty “near , ” — that geolocation lookups love to leave behind.

test('a fresh sign-in triggers a truthful alert', async ({ page }) => {
  const inbox = await mail.createInbox({ ttlSeconds: 900 });
  await seedUser({ email: inbox.address, password: 'pw-1' }); // your app's test hook

  await login(page, inbox.address, 'pw-1');

  const alert = await inbox.waitForMessage({ match: 'subject:New sign-in', timeout: 30_000 });
  expect(alert.textBody).toMatch(/Chrome|Firefox|WebKit/);      // the device line rendered
  expect(alert.textBody).not.toMatch(/undefined|None|near\s*,/); // the empty-geo bug
});

The timeout is a ceiling, not a duration — the wait resolves the moment the alert lands. The same shape works from Cypress through a task bridge and pytest through a fixture.

Asserting the location without hardcoding it

The structural test above proves a location rendered. It doesn't prove it's the right one — and “right” is the entire point of this email. The trick is that your test already knows the ground truth: the sign-in came from the runner itself, so the runner's public egress IP is exactly what the alert should describe. Learn that IP, ask a geolocation API where it actually is, and assert the email agrees on both counts:

// 1. Ground truth: what the world sees when this runner connects.
const ip  = (await (await fetch('https://api.ipify.org')).text()).trim();
const geo = await (await fetch(
  `https://api.theipapi.com/v1/ip/${ip}?api_key=${process.env.THEIPAPI_KEY}`
)).json();

// 2. Sign in and catch the alert as before.
await login(page, inbox.address, 'pw-1');
const alert = await inbox.waitForMessage({ match: 'subject:New sign-in', timeout: 30_000 });

// 3. The alert must name the real source — not the load balancer.
expect(alert.textBody).toContain(ip);
expect(alert.textBody).toContain(geo.body.location.city);
expect(alert.textBody).toContain(geo.body.location.country_code);

The IP assertion is the load-bearing one. The classic production bug in this flow is geolocating the wrong address: behind a load balancer or CDN, request.ip is your own infrastructure, and a mis-read X-Forwarded-For means every alert your users ever receive says “new sign-in from your datacenter's city” — plausible-looking, structurally perfect, and useless as a security signal. Hardcoded-city tests can't see this bug (they were written to expect whatever the broken pipeline produced); the egress-IP check fails on it immediately.

One honesty note on the city line: IP geolocation is an estimate, and two databases can genuinely disagree on the city of a datacenter range even when both are “right.” If your app uses a different geolocation provider than your test, assert country_code and treat the city as informational — or point both sides at the same provider, and the disagreement disappears by construction.

DISCLOSURE The IP API (theipapi.com) is built by the same maker as MailFixture. Any geolocation API with a city field fits this pattern — we use ours because the free tier (1,000 lookups/day, no card) covers a CI suite many times over. The pattern is the point, not the provider.

The tests around the happy path

A known device stays quiet. Alert fatigue is how users learn to ignore security email. If your product suppresses alerts for recognized sessions, prove it: log in twice from the same context and assert the second sign-in sends nothing — a bounded negative wait, where the timeout is the pass condition.
The unroutable IP renders honestly. Staging traffic often arrives from private ranges that have no location at all. The alert must degrade to an explicit “Unknown location,” never to templating debris — this is the single most common rendering bug in the flow, and the not.toMatch line in the first test is what catches it.
Decide what CI sign-ins should look like. Every CI runner is a datacenter IP — the exact signal alert pipelines treat as suspicious (a geolocation response says so outright: is_datacenter: true). If your product has a “suspicious sign-in” variant, your suite is the perfect trigger for it: assert that the stronger alert fires for a datacenter source, rather than letting CI noise train you to ignore the distinction.

test_login_alert.py · pytest — the quiet path

app.login(session, user.email, "pw-1")   # same device, second sign-in

with pytest.raises(MailFixtureTimeout):
    inbox.wait_for_message(match="subject:New sign-in", timeout=5)  # silence IS the assertion

Same honesty rule as the enumeration test: five seconds of silence proves “no alert was sent promptly,” not “no alert will ever exist.” That bounded claim is still exactly enough to catch the refactor that wires recognized devices back into the alert path.

Local dev vs CI

This is one flow where localhost capture tools structurally can't help: a sign-in from 127.0.0.1 has no geography, so the most important content in the email never renders locally. You'll see the template; you'll never see the bug. Receiving in CI on a real address, from a runner with a real egress IP, is the only place the location pipeline actually executes end to end.

Related flows, same three mistakes: codes instead of alerts, login links, and the reset flow this alert usually guards.