There’s no hosted APM, no tracing in production and no dashboard I have to remember to open, and the only part that costs extra is the log archive, at about US$2 a month. Here’s the whole list, and the things I dropped or stopped trusting, one of which had me chasing an outage no user ever saw.

What I mean by monitoring, and how small this is

#

By DevOps monitoring I mean finding out that something’s broken before somebody tells me, and being able to work out why without guessing. Every tool below is judged on those two things.

Last year I wrote about setting up monitoring that developers actually use, mostly with teams in mind: a cloud provider’s built-in monitoring for a small team, a hosted APM once there are hundreds of services. This is the small end of the same idea. Both products are Django on Fly’s shared-cpu-1x machines, as described in My Current Stack, and I’m the only developer on either. If monitoring is a product and developers are its users, mine has a user base of one. A baseline I pulled from Fly’s metrics in September had the marketplace’s API averaging 1% CPU. This isn’t an enterprise stack, and I won’t dress it up as one.

Of the three pillars, metrics, logs and traces, I have two. Neither product has distributed tracing: each is one Django codebase and one database, with no service boundary for a trace to cross. The only traces I read are the reverse proxy’s, on my workstation.

The list

#

Everything in the table is the marketplace unless it says both apps. The estimating app has less: Fly’s health check, logs and metrics, app analytics on iOS, and a daily Parquet export of its database that I query with DuckDB. It has no crash reporting and no alerts channel, which is the biggest gap here. (A disclosure for the update-service row: the company behind it pays me to write for their blog.)

Errors: a logging handler posting to chat

#

Until September, flyctl logs was the only place an unhandled 500, a failed push notification or a broken WebSocket frame showed up, and nobody reads that stream unprompted. Now a logging handler sends every record at ERROR or above to a chat channel, from the API, the WebSocket server and the background worker. Warnings deliberately don’t alert.

The handler posts on a background thread, so it never slows a request. Nothing in it logs, so it can’t recurse. And the same error goes out at most once every ten minutes, with a cap of 30 an hour, so an outage reads as one message rather than a thousand.

The channel also carries what I want to read errors against: deploys post when they start, when they’re up and where they failed, with the commit and who ran it, and so do over-the-air updates and web releases. A burst of wrong sign-in codes on one account logs at ERROR too, so someone guessing shows up next to the outages.

In October an alert arrived that matched no line in any log. Now each alert carries a short random id, the hostname and the process’s start command, and writes the same line to that process’s log, so an id I can’t find means a process that doesn’t log there, like an SSH shell. Everything in the channel also goes into a feed that a coding agent I work with can read, so the agent working on the code sees the errors when I do.

Logs: Fly’s shipper to S3, queried with DuckDB

#

Fly’s own buffer holds a few minutes of output, roughly the last hundred lines, which is fine for watching a deploy and no use for yesterday. So a third Fly app runs Fly’s log shipper, which is Vector subscribed to the organisation’s log stream, with one change to its S3 sink:

[sinks.aws_s3]
  type = "aws_s3"
  bucket = "myapp-logs"
  compression = "gzip"
  key_prefix = "logs/{{fly.app.name}}/%F/"   # one folder per app per day
  batch.timeout_secs = 300                  # a quiet app still lands its lines

Every line from the marketplace’s apps lands in an S3-compatible bucket as gzipped JSON. The shipper is one shared-cpu-1x machine with 256 MB, about US$2 a month plus pennies of storage. DuckDB reads the files where they sit through one macro, and a day comes back in about a second:

CREATE MACRO logs(app, day := '*') AS TABLE
  SELECT * FROM read_json_auto('s3://myapp-logs/logs/' || app || '/' || day || '/*.log.gz',
                               union_by_name := true);

SELECT timestamp, message FROM logs('myapp-api', '2026-09-06') WHERE message LIKE '% 500 %';

That macro is how I measured the cold starts in the scale-to-zero post: the startup script announces each phase, every line carries the machine’s id, and the rest is a GROUP BY.

So it’s three levels of one stream, by who reads them: the archive keeps INFO and up, which is why the apps log at INFO in production, a live-tail script shows warnings and errors, and the chat channel gets ERROR. None of them knows the caller, though. The access log has no client IP or user agent, and for that I go to Cloudflare.

Metrics: Fly’s built-in Prometheus

#

Fly runs a managed Prometheus for each organisation, so I don’t run a metrics stack of my own. I query its API when I have a question: memory, CPU and burst credit per machine (fly_instance_cpu_balance matters on a shared CPU), responses and concurrency per app. Two things I wish I’d known first. A token from fly tokens create goes in as Authorization: FlyV1 <token>; Bearer only works with the personal token from flyctl auth token, and a read-only token gets “something went wrong resolving organization”. And it keeps about two weeks (Fly says roughly 15 days).

It earned its place in September, when the logs had “Could not open a database connection” several times a day, which reads exactly like an app bug. The managed Postgres cluster is in the same Prometheus, and at every one of those minutes the primary’s scrape time jumped from about 80 milliseconds to between one and ten seconds, while the replica was fine and Postgres’s start time never moved. Postgres hadn’t restarted; the host under it was stalling. I sent Fly’s support the timestamps, and after a maintenance restart in early October moved the primary to another host, the daily stalls stopped.

Crash reports: Sentry’s SDK, pointed at my own API

#

A crash on a phone used to leave no trace anywhere I could see. The obvious answer was Sentry, and I took half of it: the app carries Sentry’s open-source SDK, but its DSN points at an endpoint in my own Django API that accepts Sentry’s envelope format, groups the crashes, and alerts through the same chat handler. Tracing, sessions and replays are off. It’s errors only.

I didn’t sign up for sentry.io because, at a few reports a day, it would have been another vendor holding user data, another line in the privacy policy and another bill. The price is symbolication, which I haven’t replaced. Stacks are minified JavaScript or raw native addresses, so an issue tells me what crashed, on which release and device, and what the person did just before, but not where in the source.

The only dashboard is a message

#

I haven’t built a dashboard for either app. Last year’s post had a test for them: if nobody opens one unprompted, it isn’t earning its place. A message that arrives on its own doesn’t need opening, so the closest thing I have is a post in the team chat every morning.

It starts with the last 24 hours of production errors, read from the log archive rather than from the chat channel, because throttled alerts can’t say how often something happened. Numbers, ids and addresses are masked before grouping, so a hundred copies of one failure are one row with a count and no email reaches the channel.

Then failed health checks, yesterday’s usage against the same weekday a week earlier (a Sunday is never like a Monday), the last week of search traffic, chats waiting on a reply, the task board and what I’ve spent. A section that can’t load says so, and the post still goes out.

Reading app analytics through an API teaches you to distrust them a little. In August, 121 Android first-opens from Azerbaijan were followed by 119 uninstalls, and I read that month’s Android totals with them taken out.

Releases have their own screen. When over-the-air updates came from my own server, my view of a release was an admin list with an “is active” checkbox, and I had no idea how many phones had taken one. The hosted service’s dashboard shows which release devices run, how installs went, and a crash rollback when a bad bundle crashes and the SDK restores the previous one. I wrote up the move to a hosted update service separately.

What I dropped, or stopped believing

#

None of this was a paid tool. Mostly it was things I’d been treating as signals that weren’t, and one job running in the wrong place.

Cloudflare’s 504 rate, until I grouped it by user agent

#

In September the Cloudflare dashboard showed what looked like 22% of a day’s requests failing with 504s, concentrated on the endpoints the app polls every 30 seconds, while every health check was green and the app’s own logs said those requests were fast. I spent days on it, found that machines took 25 to 33 seconds to cold-start and did it 92 times a day, fixed that, and wrote it up in the scale-to-zero post as the cause.

The next morning I grouped the same 504s by user agent. All 7,344 of them between 5 and 9 September came from Cloudflare’s own Early Hints cache lookups (user agents bastion early hints and nginx-ssl early hints), which never reach the origin: a 504 on one of those lines is a cache miss, not a timeout. They copy real request paths, so they landed on the busiest endpoints and looked exactly like users failing. The app, browsers, curl and search crawlers had none. Early Hints wasn’t doing anything for me because the origin sent no Link header, so I turned it off, and later made it work.

The cold-start work still paid off, since deploys and wakes are faster for it, but the outage I thought I was fixing never reached a user.

So this is the rule I’d most like to pass on: before you believe a CDN’s error rate, group it by user agent and check it isn’t the CDN’s own machinery. On Cloudflare’s free plan that’s a GraphQL query on httpRequestsAdaptiveGroups, a day at a time. The same query settled the next scare, when one polling endpoint went from 70–260 requests a day to 2,880. It wasn’t a scraper. It was my own Mac, with the app left running in the iOS simulator after testing.

A green health check

#

Fly checks /api/health/ on both apps every 30 seconds and uses the answer to decide which machines get traffic, which it’s good at. But a health check runs against a machine that’s already up, so it can’t see what happens before one is: a cold start, a queue at the proxy, a request that never arrives. Routing is all I ask of it now.

Deploys get the probe script instead, which hits the origin throughout and exits 1 if anything failed. It caught a plain flyctl deploy leaving a 7.3-second gap, then counted no failures in 2,514 requests through the deploy script, which wakes a spare machine first.

The health check has also caused two outages of its own. In July, Django’s SECURE_SSL_REDIRECT answered Fly’s plain-HTTP probe with a 301, every machine failed its check, and everyone got a 503. In August, narrowing an except Exception in the health view let a “connection already closed” error escape as a 500, and Fly pulled the machine in exactly the case the view’s retry was there for. Now the check only asks whether the process can serve, and a database stall no longer takes both apps out of routing.

The morning summary had to learn the same lesson about counting. The marketplace suspends idle machines, so a machine woken by a request fails its first check a few hundred milliseconds after starting and passes the next. Counting those made every morning read like an outage, so a failure that passes again within a minute is now listed separately, as machines waking up and deploys.

A scheduled job in CI

#

The morning summary started as a CI workflow on a cron schedule and moved within a day. Once it read the analytics and search data, the key it needed was the analytics project’s service-account key, which can do anything in that project, and I didn’t want that sitting in CI secrets. Now a scheduler on my Mac runs it every 20 minutes, and it posts once it’s past eight in Auckland and today’s post hasn’t gone out, so a Mac asleep at eight posts when it wakes. The workflow stays as a manual fallback, minus the analytics and search sections.

If I were starting again

#

I’d set up three things before the first user rather than after the first incident: errors to a chat channel the team already reads, with a rate limit; a log archive in object storage I can query with SQL; and whatever metrics the platform already collects. That’s a couple of dollars a month. Crash reporting comes the week an app goes onto phones, because otherwise a crash on a phone tells nobody anything. A morning summary comes once there are enough places to check that I’d stop checking them.

I’d leave tracing until there’s a second service for a request to cross, and a hosted APM until there’s a team that needs shared dashboards, an on-call rota and a budget for someone else’s bill. That’s when the tools in last year’s post start paying for themselves. Before then the most useful habit is free: when a number looks alarming, find out what produced it before deciding what broke. Several of the scares in this post came from the monitoring, not the product.

There’s a gap between “works on my machine” and “works in production”, and monitoring is supposed to bridge it. Most of the time it doesn’t, because it gets built as an ops tool rather than something a developer reaches for when things break.

I’ve just gone freelance doing DevOps and infrastructure work, and the monitoring setup is one of the first things I look at anywhere I land — it tells you more about a team than the README does. The failure mode is nearly always the same: the tools exist, but they don’t connect to how developers actually work. Someone burns three hours on a bug that production logs would’ve solved in five minutes. Failures arrive as support tickets instead of alerts. Or the opposite: 200 alerts a day, most of them noise, so everyone ignores all of them, and the first half hour of every incident is spent working out what broke rather than fixing it.

This is the general version. For the specifics, I’ve since written up what I actually run today on two small products.

Metrics, logs, and traces — linked, or useless

#

Metrics tell you something is wrong (error rate spiked, latency jumped). Logs tell you what happened (stack traces, request payloads, error messages). Traces tell you where (which service, which endpoint, which database call). None of that is news. What matters is whether they’re linked: when an alert fires, you should be able to click from the metric to the relevant logs to the trace. If developers are manually correlating timestamps across three tools, the setup has failed, no matter how good each tool looks on paper.

Dashboards deserve the same test. CPU, memory and disk I/O matter for capacity planning, but they don’t help anyone debug a 500. The dashboards people open voluntarily show business metrics next to technical ones (signups per hour beside error rate), recent deployments overlaid on the timeline, the top five errors in the last hour with links to logs, and latency percentiles — p50, p95, p99 — rather than averages. If nobody opens a dashboard unprompted, it isn’t earning its place.

Alerts, same idea. Every alert should answer two questions: what’s broken, and where do I start looking?

Bad alert: “High CPU on web-server-3”

Good alert: “Error rate > 5% on /api/payments since 14:32. Last deploy: 14:15 by @sarah. [View logs] [View trace]”

A few things push alerts toward the second kind: alert on deviation from a baseline rather than an arbitrary threshold, route by ownership (payment errors go to the payments team, not the whole org), and group related errors into one Slack message instead of 50. Keep the loop tight, too — the time between “something broke” and “a developer knows” should be under a minute, which means real-time log streaming rather than five-minute batches, deploy markers on every dashboard, and tracing that survives service boundaries.

Build it for the people who’ll use it

#

The biggest mistake ops teams make is building monitoring for themselves. The fix isn’t sophisticated. Sit with a developer during a debugging session and watch where they get stuck. Ask what questions come up mid-incident — “which service?”, “what changed?”, “what did the request look like?”. Find out which metrics actually matter for the product; it’s rarely CPU, and usually things like checkout completion rate or upload success rate.

Then remove friction. If instrumenting a new service takes a day, it won’t happen. Provide libraries that auto-instrument the frameworks you already use (Django, Express, Spring), copy-paste templates for dashboards and alerts, and self-service metrics so nobody has to file a ticket. And treat the whole thing as code: config in version control, alert rules tested so you know they fire when they should, and multi-region considered early if that’s where you’re headed.

The same habits keep it alive afterwards. Instrumentation goes through code review like any other production code, and a custom metric gets a test. Every postmortem asks what monitoring would have caught this sooner, and the answer goes into the dashboards and alerts that week rather than into the write-up. And a few times a year I go through what’s there and delete what nobody opens, because an alert nobody acts on is training people to ignore the next one.

Small team on AWS? Start with CloudWatch

#

If you’re under 50 engineers and on AWS, CloudWatch is the right starting point, and I’d resist buying anything fancier on day one. EC2, Lambda, RDS and ECS all report to it automatically, so you get visibility without writing instrumentation. It runs $10–50 a month for a small team with no platform fee, and metrics, logs, traces (via X-Ray) and alarms all live in one place. You can have meaningful alerts and dashboards up in an afternoon, not weeks.

CloudWatch Logs Insights is the underrated bit — you can query logs without standing up Elasticsearch:

fields @timestamp, @message
| filter @message like /ERROR/
| stats count() by bin(5m)

Beyond that:

• Composite alarms cut noise — alert when error rate is high and response time is degraded, not on each condition separately.
• Custom metrics are where the real value is. Push business metrics (signups, transactions, feature usage) through the CloudWatch SDK alongside the infrastructure ones.
• CloudWatch Synthetics runs scheduled canaries through real user journeys, so you learn the checkout is broken before your users do.
• X-Ray gives you distributed tracing with minimal setup — good enough for most microservice architectures.

You’ll know when you’ve outgrown it: you go multi-cloud, you want proper anomaly detection, dashboard customisation starts to hurt, or the team passes 50 engineers and needs real collaboration features.

Bigger teams: Datadog

#

Datadog is expensive — $20–100K+ a year for a large org is normal — but at scale it earns its keep. One view across AWS, Azure, GCP, on-prem, containers, serverless, databases and frontend. Watchdog flags anomalies without hand-tuned thresholds, which matters because nobody can manually watch thousands of services. And you get the collaboration layer CloudWatch lacks: team dashboards, RBAC, shared investigation notebooks, PagerDuty/Opsgenie integration, multi-condition alerts, threshold forecasting, maintenance windows, and an APM that goes down to code-level profiling with auto-generated service maps.

Rolling it out, in roughly the order I’d do it: instrument critical services first and tag everything by team and environment from day one. Agree naming conventions, dashboard templates, severity levels and SLO definitions early — retrofitting standards is miserable. Wire it into CI/CD deploy markers, incident management and Slack. Budget time for training, because Datadog is powerful but not self-explanatory. And watch the bill: filter noisy metrics, sample traces on high-volume services, and audit which features you’re actually paying for.

Worth a look before committing: New Relic (similar, sometimes cheaper for high-volume tracing), Dynatrace (strong AIOps, popular in financial services), Splunk (best-in-class log analysis, especially if security already runs it), and Grafana Cloud (the natural home if you’re already on Prometheus/Loki).

In practice a lot of teams land on a hybrid: CloudWatch for AWS-native services because it’s automatic and cheap, Datadog for the application layer, with CloudWatch metrics ingested into Datadog for one unified view. It’s unglamorous, and it works.

CloudWatch vs Datadog at a glance

#

Starting from scratch: the build order

#

The order I’d do things in:

1. Talk to developers about what hurts during incidents.
2. Define SLOs for the flows that matter. What does “working” mean for login, checkout, search?
3. Instrument that critical path first.
4. Add distributed tracing next; it has the best debugging ROI in a microservice setup.
5. Alert on those flows rather than on CPU, and send each alert to the team that owns it.
6. Write a runbook for every alert so 3am-you knows what to check.
7. Put a quarterly review in the calendar to delete stale alerts, fix drifted thresholds, and confirm the dashboards still match the architecture.

And the traps I keep pulling teams out of: too many tools (three that work together beat six that don’t), graphs with no baseline (is 500 rps normal or a 10x spike?), monitoring things users don’t feel (disk I/O on a stateless container), and no redundancy for the monitoring itself — if your alerting dies during an incident, you’re blind exactly when you can least afford it.

None of this needs the fanciest tooling. It needs developers to get the information they need, in the place they’ll actually look, fast.