SimpleCold can push events to your server: webhooks send bulk.finished when a bulk job ends and result.ready when a slow mailbox check settles. If you can take them, they are the simplest option. Polling is still worth knowing. It needs no public endpoint and no signature checks, it works from a laptop or a locked-down network, and a missed event can't leave you stuck, because the next poll picks it up. It is also the right fallback behind a webhook handler.
The trick is to poll from stored state rather than from a long-running loop. This guide builds that: a small table of outstanding work, a sweeper that runs every minute, and scheduling that respects both the API's hints and its rate limits.
What can be outstanding
There are two kinds of unfinished work:
- Bulk jobs. You uploaded a CSV with
POST /api-validation/bulkand got a jobid. You need to know when its status becomesCOMPLETED, then download the result. - Pending single checks. A single or batch call returned
status: "processing". You need the final result fromGET /api-validation/single/result.
Both fit in one table.
The table
CREATE TABLE verification_work (
id BIGSERIAL PRIMARY KEY,
kind TEXT NOT NULL CHECK (kind IN ('bulk_job', 'pending_email')),
ref TEXT NOT NULL, -- job id, or the email address
state TEXT NOT NULL DEFAULT 'waiting', -- waiting | done | failed
next_check_at TIMESTAMPTZ NOT NULL DEFAULT now(),
attempts INT NOT NULL DEFAULT 0,
last_payload JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
UNIQUE (kind, ref)
);
CREATE INDEX ON verification_work (state, next_check_at);
The unique constraint matters. If the code that uploads a file runs twice, the second insert conflicts instead of creating a duplicate row, and you can see that something went wrong.
Write the row in the same place you call the API:
// after a successful bulk upload
await db.query(
`INSERT INTO verification_work (kind, ref) VALUES ('bulk_job', $1)
ON CONFLICT (kind, ref) DO NOTHING`,
[job.id],
);
// after a single/batch result with status "processing"
await db.query(
`INSERT INTO verification_work (kind, ref, next_check_at)
VALUES ('pending_email', $1, now() + interval '30 seconds')
ON CONFLICT (kind, ref) DO UPDATE SET state = 'waiting'`,
[result.email],
);
The sweeper
Run this every minute from cron, a scheduled container, or a queue's repeat job. It picks rows that are due, checks each one, and writes back when to look next.
// sweeper.mjs (Node 18+, uses node-postgres)
import pg from "pg";
const db = new pg.Pool();
const API = "https://simplecold.com/api/v1";
const auth = { Authorization: `Bearer ${process.env.SIMPLECOLD_API_KEY}` };
// Stay under the per-key API limits: bulk status 120/min, single + single/result 100/min.
const BUDGET = { bulk_job: 100, pending_email: 80 };
async function due(client, kind, limit) {
const { rows } = await client.query(
`SELECT id, ref, attempts FROM verification_work
WHERE state = 'waiting' AND kind = $1 AND next_check_at <= now()
ORDER BY next_check_at LIMIT $2
FOR UPDATE SKIP LOCKED`,
[kind, limit],
);
return rows;
}
function delayFromStatus(p, attempts) {
if (p.nextUpdateAt) {
const ms = new Date(p.nextUpdateAt).getTime() - Date.now() + 1_000;
return Math.min(Math.max(ms, 15_000), 10 * 60_000);
}
if (p.etaSeconds != null) return Math.min(Math.max(p.etaSeconds * 1000, 15_000), 10 * 60_000);
return Math.min(30_000 * 2 ** attempts, 10 * 60_000);
}
// A 429 says how long to wait in Retry-After (seconds).
function retryAfterMs(res) {
const seconds = Number(res.headers.get("retry-after"));
return seconds > 0 ? seconds * 1000 : 60_000;
}
async function checkBulk(row) {
const res = await fetch(`${API}/api-validation/bulk/${row.ref}/status`, { headers: auth });
if (res.status === 429) return { retryInMs: retryAfterMs(res) };
const body = await res.json();
if (!res.ok) return { failed: true, payload: body };
const p = body.payload;
if (p.status === "COMPLETED" || p.status === "CANCELED") return { done: true, payload: p };
if (p.status === "FAILED") return { failed: true, payload: p };
return { retryInMs: delayFromStatus(p, row.attempts), payload: p };
}
async function checkPending(row) {
const q = `?email=${encodeURIComponent(row.ref)}`;
const res = await fetch(`${API}/api-validation/single/result${q}`, { headers: auth });
if (res.status === 429) return { retryInMs: retryAfterMs(res) };
if (res.status === 404) return { done: true, payload: null }; // already collected elsewhere
const body = await res.json();
if (!res.ok) return { failed: true, payload: body };
const r = body.payload;
if (r.status === "processing") {
return { retryInMs: Math.min(30_000 * 2 ** row.attempts, 5 * 60_000), payload: r };
}
return { done: true, payload: r };
}
async function sweep(kind, check) {
const client = await db.connect();
try {
await client.query("BEGIN");
const rows = await due(client, kind, BUDGET[kind]);
for (const row of rows) {
const out = await check(row).catch(() => ({ retryInMs: 60_000 }));
const state = out.done ? "done" : out.failed ? "failed" : "waiting";
await client.query(
`UPDATE verification_work
SET state = $2, attempts = attempts + 1, last_payload = $3,
next_check_at = now() + ($4 || ' milliseconds')::interval
WHERE id = $1`,
[row.id, state, JSON.stringify(out.payload ?? null), String(out.retryInMs ?? 0)],
);
if (out.done) await onDone(kind, row.ref, out.payload);
}
await client.query("COMMIT");
} catch (e) {
await client.query("ROLLBACK");
throw e;
} finally {
client.release();
}
}
async function onDone(kind, ref, payload) {
// Bulk: download the result file and import it. Pending: store the final result.
// Keep this idempotent; a crash after onDone but before COMMIT means it runs again.
}
await sweep("bulk_job", checkBulk);
await sweep("pending_email", checkPending);
await db.end();
due runs on the transaction's client, so FOR UPDATE SKIP LOCKED holds its locks until COMMIT. Two sweepers running at once, say during a deploy, won't pick the same rows.
Why this shape works
The API tells you when to come back. Bulk status includes nextUpdateAt, the time the progress numbers will next change. Polling before then returns identical data and spends a status call for nothing. The sweeper schedules the next check for just after that time, with a 15-second floor and a 10-minute ceiling.
Budgets keep you under the rate limits. Bulk status has its own limit of 120 calls per minute per key, separate from uploads (10 per minute) and downloads (30 per minute), so polling never eats into your uploads. The sweeper takes at most 100 status checks per run. Pending checks share the single limit of 100 per minute with new single checks; the sweeper takes 80, leaving room for live traffic. Your organization also has a cap of three keys' worth on each endpoint, so if you run several services, split these budgets between them. When a 429 does come back, the sweeper waits for its Retry-After. The rate limit guide covers how limits are counted.
Restarts don't lose work. State lives in the database, not in a while loop. Kill the process, deploy, scale to zero: the next run picks up whatever is due.
404 on a pending check is not an error. The result endpoint returns 404 once a finished result has been collected. If another process got there first, or the user retried through the single endpoint, the work is done. Mark the row done and move on.
Python version of the core loop
If your stack is Python, the same logic fits in a short function. This version uses SQLite to stay self-contained, with a work table that has the same columns as above and stores next_check_at as Unix seconds.
import os, time, json, sqlite3
from datetime import datetime, timezone
import requests
API = "https://simplecold.com/api/v1"
s = requests.Session()
s.headers["Authorization"] = f"Bearer {os.environ['SIMPLECOLD_API_KEY']}"
db = sqlite3.connect("work.db")
def next_check(p, attempts):
if p.get("nextUpdateAt"):
t = datetime.fromisoformat(p["nextUpdateAt"].replace("Z", "+00:00")).timestamp()
return max(time.time() + 15, min(t + 1, time.time() + 600))
return time.time() + min(30 * 2 ** attempts, 600)
def sweep_bulk(limit=100):
rows = db.execute(
"SELECT id, ref, attempts FROM work WHERE kind='bulk_job' AND state='waiting' "
"AND next_check_at <= ? ORDER BY next_check_at LIMIT ?", (time.time(), limit)).fetchall()
for id_, ref, attempts in rows:
res = s.get(f"{API}/api-validation/bulk/{ref}/status", timeout=30)
if res.status_code == 429:
break # out of budget; Retry-After says for how long
p = res.json().get("payload") or {}
state = {"COMPLETED": "done", "CANCELED": "done", "FAILED": "failed"}.get(p.get("status"), "waiting")
db.execute("UPDATE work SET state=?, attempts=attempts+1, last_payload=?, next_check_at=? WHERE id=?",
(state, json.dumps(p), next_check(p, attempts), id_))
db.commit()
When polling stops
Set an upper bound. A bulk job that is still IN PROGRESS after a day, or a pending address that hasn't settled after an hour, deserves a human look rather than another thousand polls. Add a check on attempts or created_at that moves the row to failed and sends an alert. For pending addresses, the pending marker on SimpleCold's side expires after about 30 minutes; after that the result endpoint returns 404, and a fresh single check is a new check that uses a credit. Most slow checks settle within about two minutes, so an address still pending after half an hour is rare.
Adding webhooks on top
With webhooks this design still holds. Subscribe an endpoint to bulk.finished and result.ready, check the SimpleCold-Signature header, and have the handler mark the matching row as due now instead of waiting for its scheduled check. Keep the sweeper running with longer intervals as a fallback: it covers any delivery your server missed while it was down, and SimpleCold also lets you resend a delivery from the dashboard.
For the individual endpoints used here, see run a bulk job end to end and handle processing results. The API docs have the full reference.
