One Elixir monolith serving a dashboard, a public API, a marketing site, and every background job behind eSIM fulfilment — with carrier webhooks that cannot be allowed to drop.

The problem

In connectivity, the provider is the product. When a carrier reports that an eSIM has been installed, that a package is nearly exhausted, or that a device's IMEI has changed, that event has to land — every time, in order, and idempotently. A dropped webhook is a customer with no service and no explanation.

The platform also had the shape that catches teams out: a single Phoenix application serving an internal dashboard, a public API, a marketing surface, and all the background processing. Convenient to ship, unforgiving when one part gets loud.

Our approach

We treated the provider integration as a first-class domain rather than a client wrapper. Template sync, product and channel mapping, and dashboard-driven resync were built as product surfaces the operations team could drive, so that recovering from a provider-side change did not require a deploy.

For the webhooks, the priority was the unglamorous half: handling the edge cases that only appear in production — resolving an ICCID when the payload does not carry the one you expected, reconciling usage against plan state, deciding what a status change means for a plan that is paused, legacy, or unlimited.

Analytics moved to ClickHouse rather than being bolted onto the transactional database, which keeps reporting load off the path that fulfils orders.

What we built

  • Telna implemented as a first-class provider: template sync, product and channel mapping, a channel products manager, and operator-driven resync from the dashboard
  • Provider webhooks owned end to end — eSIM status, package usage and alerts, package status, IMEI changes — wired into plan updates, including ICCID resolution edge cases
  • eSIM subscription sync across Webbing and Telna flows, with the eUICC profile API, scheduled workers, and rate-limited batch sync
  • Operational flows hardened: cancel and suspend, balance-line activation rules, pause and resume, and correct handling of legacy versus unlimited plans
  • ClickHouse event and analytics infrastructure, including schema and migrations, local cluster setup, and replication fixes
  • Background job reliability with Oban: dedicated workers, throttling for plan updates, a job cleaner, and Oban Web for visibility
  • PostgreSQL trigram fuzzy search with GiST indexes and tests, backing coverage-style endpoints
  • V2 API surface expanded under OpenAPI, with rate limiting on the sensitive paths — login, email checks, signup
  • Security posture work: IP allowlisting driven from the database, reduced password leakage in OAuth error paths, log sensitivity filtering, and SQL-injection hardening

The outcome

Provider events now flow into plan state as a matter of course rather than as a recurring incident, and the operations team can resync from the dashboard instead of filing a ticket.

The security and rate-limiting work is the kind that produces no visible result when it goes well, which is the point. Login and signup endpoints are no longer the cheapest thing on the platform to abuse.