devto 2026-10-11 원문 보기 ↗
A small property-management app should start with externally hosted uptime checks for its public health endpoint and heartbeat checks for delivery jobs. Self-host the monitor only when control over data location or the monitoring path matters enough to justify operating another failure domain.
TL;DR: choose the setup that produces the clearest evidence with the least configuration, then send four signals: reachability, dependency readiness, job completion, and delivery outcome. A monitor that pages on every transient error will be ignored. A monitor inside the same broken network may stay quiet when it matters.
| Choice | Strongest evidence | Main blind spot | Operational load | Best fit |
|---|---|---|---|---|
| Externally hosted monitor | What an outside client can reach | Internal-only paths | Low | Small team with a public endpoint |
| Self-hosted monitor | What the chosen network can reach | Shared infrastructure failures | Higher | Strict placement or private-network constraints |
| Split monitor | Outside reachability plus internal job evidence | More correlation work | Medium | Delivery paths that cross trust boundaries |
The default recommendation is the first row plus a separate cron heartbeat. It gets to the first useful alert quickly, with little glue. Move to the split model when delivery workers or queues cannot be assessed from the public endpoint. Do not expose private dependency details merely to make an external checker happy.
An uptime probe and a notification-delivery check answer different questions. GET /health can prove that an HTTP process responds. It cannot prove that the 09:00 rent reminder was dequeued, handed to a delivery channel, or rejected. Treating one green response as proof of the whole workflow creates a comforting dashboard and weak evidence.
For this system, four signals are enough to begin:
OpenTelemetry describes a metric as a measurement captured at runtime, including values such as request counts and active requests. That model fits delivery outcomes: use counters for attempts and failures, then calculate rates over an explicit window. Keep attributes bounded. A tenant ID, lease ID, phone number, or raw error message turns a useful metric into a cardinality problem and may leak customer data.
The health endpoint should stay boring. Return success only when the process is ready for the contract the endpoint represents. Put deep diagnostics behind authenticated operational access, because a public response does not need queue names, database errors, or resident information.
Property notifications have uneven urgency. A failed maintenance-entry notice can be time-sensitive; a delayed monthly summary usually is not. One global threshold erases that distinction. Define alert policy from user impact, not from whichever counter was easiest to export.
Noise usually enters through retries. Picture a rent reminder batch starting at 09:00. One delivery attempt fails, a retry succeeds, and both events feed a raw failure alert. The operator gets paged for an outcome the system already recovered, opens the delivery record, finds the successful retry, and learns that this alert class wastes attention. Repeat that pattern and a real terminal failure has to compete with yesterday's harmless retries. Count both attempts for diagnosis, but alert on the terminal result or on an error-rate window that survives the retry policy. Preserve the first failure as evidence; don't promote it to an incident before the workflow reaches its recovery boundary. Fast feedback is useful. False urgency isn't.
I benchmark monitoring designs by time-to-first-trustworthy-signal, not time-to-first-dashboard. The stopwatch should stop only after a forced failed delivery produces one actionable notification and a recovered retry does not produce another. The trade-off is blunt: a five-minute synthetic probe may detect a dead endpoint later than a one-minute probe, but the tighter interval also increases requests and makes transient network failures more visible. Pick the interval from the response objective and require consecutive failures when a single miss is not actionable.
Trust comes first.
Short alerts need context: environment, affected workflow, observation window, and a correlation key that leads to logs or traces. They do not need a configuration dump.
Keep the application contract generic. The probe below checks reachability and response time without binding the service to a monitoring vendor. It also uses a timeout, because an indefinitely pending check is not a health result.
type ProbeResult = {
ok: boolean;
status?: number;
durationMs: number;
error?: "timeout" | "network";
};
export async function probeHealth(
url: string,
timeoutMs = 3_000,
): Promise<ProbeResult> {
const startedAt = performance.now();
try {
const response = await fetch(url, {
method: "GET",
signal: AbortSignal.timeout(timeoutMs),
headers: { accept: "application/json" },
});
return {
ok: response.ok,
status: response.status,
durationMs: Math.round(performance.now() - startedAt),
};
} catch (error) {
const timedOut = error instanceof DOMException && error.name === "TimeoutError";
return {
ok: false,
durationMs: Math.round(performance.now() - startedAt),
error: timedOut ? "timeout" : "network",
};
}
}
The scheduled dispatcher needs the inverse contract: it sends a heartbeat after successful completion. Heartbeat monitoring documentation commonly models this with a unique ping URL and a grace period. Send the success signal after the batch reaches its defined terminal state, not when the job starts. Otherwise a worker can hang halfway through and still look healthy. The example's 3,000 ms deadline is a deliberate bound, not a universal threshold; set it from the probe's own response objective.
Do not place a secret heartbeat URL in logs. Inject it at runtime, validate that it exists during deployment, and make the call bounded. Decide explicitly whether heartbeat failure should fail the business job; in most notification paths, recording the delivery result is primary and reporting monitoring state is secondary.
Deployment testing should force each boundary separately: make the public handler unavailable, block a dependency used by readiness, prevent the scheduler from completing, and inject a terminal delivery failure. Verify routing and deduplication as well as detection. A rule that fires into an unattended channel is configuration theater.
Self-hosting is the runner-up for a small public app because it adds patching, backups, storage limits, alert transport, certificate renewal, and its own monitoring. That work may still be correct. Choose it when policy requires monitoring data to remain in a controlled EU environment, when the relevant endpoints are private, or when the team needs to own retention and network placement.
Placement is the hard part. If the monitor shares the application's host, cluster, DNS path, or outbound notification path, one incident can disable both the service and its witness. Run at least one check from an independent failure domain. For an EU deployment, document where probe results, contact details, and alert payloads travel; the label on the application region alone does not answer that data-flow question.
A split design often earns its complexity. An external probe observes public reachability, while an internal collector records queue and delivery outcomes. Correlate them at the alert-policy layer. This keeps sensitive internals private without pretending that an HTTP check proves delivery.
Cost belongs in the decision, but price is a weak primary filter. Compare operator hours, retained metric volume, notification delivery, upgrades, and recovery tests. Config bloat has a cost too: every special label and exception is another branch someone must understand during an incident.
Start externally hosted when the endpoint is public, the team is small, and the main need is independent reachability plus cron completion. Start self-hosted when private topology or verified data-placement requirements dominate. Use both when no single vantage point can prove the delivery workflow.
Then run a failure drill before calling the setup done. Four signals, one owner per alert, bounded labels, and an explicit recovery condition beat a wall of green charts. The goal is credible evidence with low noise.