Fail-Closed by Default: Tenant Isolation and Retry Safety

A smart lock platform is judged on the boring parts: does it refuse the right requests, does it retry the right failures, and can it tell those two cases apart. A recent batch of work across the UnlockOS SDK clustered around exactly those questions — cross-tenant authorization, row-level security, error classification, idempotent money operations, and validity-window gating.

This article pulls out the generalizable patterns. None of it is exotic. All of it is the difference between a system you can put in front of a physical door and one you cannot.

1. Tenant isolation is a stack, not a check

A single authorization check is a single point of failure. The work in this cycle layered four independent mechanisms, each of which must be bypassed for a cross-tenant read to succeed.

Layer 1: request-scoped authorization on resources without an owner

The interesting case is not "user A reads user B's reservation". It is a reservation with no owner — a walk-in, an imported booking, an unclaimed record. Code that checks resource.user_id === caller.id silently degrades to "allow" when resource.user_id is null.

The fix is to never authorize on ownership alone. Fall back to the containing scope:

type AuthzResult =
  | { allowed: true; via: 'owner' | 'facility_staff' }
  | { allowed: false; reason: 'not_owner_no_facility_role' | 'wrong_facility' };

export function authorizeReservation(
  caller: Caller,
  reservation: { id: string; userId: string | null; facilityId: string },
): AuthzResult {
  if (reservation.userId !== null && reservation.userId === caller.userId) {
    return { allowed: true, via: 'owner' };
  }
  const role = caller.facilityRoles[reservation.facilityId];
  if (!role) {
    return { allowed: false, reason: 'not_owner_no_facility_role' };
  }
  return { allowed: true, via: 'facility_staff' };
}

The shape matters. Returning a discriminated union instead of a boolean forces the caller to handle denial explicitly, and gives the audit log a machine-readable reason. reason strings end up in dashboards; false ends up nowhere.

Layer 2: bind machine credentials to a scope, not to a tenant root

API keys are the most common tenant-isolation hole, because they are usually minted at the organization level and then used against facility-level resources. Binding the key's tenant to the resource's facility — not to whatever the key claims — closes it:

export async function resolveApiKeyScope(
  apiKey: ApiKey,
  reservationId: string,
): Promise<{ tenantId: string; facilityId: string }> {
  const reservation = await loadReservationUnsafe(reservationId);
  if (!apiKey.facilityIds.includes(reservation.facilityId)) {
    throw new ForbiddenError('api_key_facility_scope_mismatch', {
      apiKeyId: apiKey.id,
      requestedFacilityId: reservation.facilityId,
    });
  }
  return { tenantId: reservation.tenantId, facilityId: reservation.facilityId };
}

Note that loadReservationUnsafe is named to be unpleasant. A loader that deliberately skips authorization should be hard to use by accident.

Layer 3: row-level security as the backstop

Application checks get refactored away. The database does not. Two separate RLS defects were closed in this cycle, and both are instructive.

The first: a policy that let any authenticated user read the cross-tenant admin graph. The pattern to avoid is a policy whose USING clause is satisfied by mere authentication.

-- Anti-pattern: "is logged in" is not an authorization decision
CREATE POLICY admin_graph_read ON admin_memberships
  FOR SELECT TO authenticated
  USING (auth.uid() IS NOT NULL);
-- Correct: the row must be reachable from the caller's own membership
CREATE POLICY admin_graph_read ON admin_memberships
  FOR SELECT TO authenticated
  USING (
    organization_id IN (
      SELECT organization_id FROM admin_memberships
      WHERE user_id = auth.uid()
    )
    OR is_platform_admin(auth.uid())
  );

The second: guests could write check-in events, which let them clear their own postpaid balance. Read policies get reviewed; write policies get copied. Any table whose rows feed a financial or access calculation needs its INSERT/UPDATE policies audited as carefully as a payments endpoint. The rule of thumb: if a row is an input to a decision the system makes about the user, the user must not be able to write it.

A related migration bug is worth noting because it is a correctness-of-security issue, not a style issue: calling an overloaded helper without an explicit cast.

-- Ambiguous when the helper is overloaded; fails or resolves surprisingly
USING (is_platform_admin(auth.uid()))
-- Explicit, and resolves to exactly one function
USING (is_platform_admin(auth.uid()::uuid))

A policy that errors at runtime is a denial-of-service. A policy that resolves to the wrong overload is a breach. Both are avoided by being explicit at the boundary.

Layer 4: verify edge-function callers with a privileged client

When a serverless function verifies its caller using an anonymous client, the verification itself is subject to RLS — which means a correctly-restricted table can make a legitimate caller look unauthenticated, and a mis-restricted one can make an attacker look fine. Caller verification must run with a client whose visibility is not a function of the caller:

const verifier = createClient(url, SERVICE_ROLE_KEY, {
  auth: { persistSession: false, autoRefreshToken: false },
});
const { data, error } = await verifier.auth.getUser(bearerToken);
if (error || !data.user) {
  return deny('invalid_token');
}
// Everything AFTER verification uses a caller-scoped client.
const scoped = createClient(url, ANON_KEY, {
  global: { headers: { Authorization: `Bearer ${bearerToken}` } },
});

The discipline: privileged client for identity, scoped client for data. Mixing them in either direction is a vulnerability.

Bonus: scope client-side session storage too

A subtle multi-tenant bug is session storage keyed globally. Open two facilities in two tabs and the second login overwrites the first, silently acting on the wrong facility. Namespacing the storage key fixes it:

export function facilityScopedStorageKey(facilityId: string): string {
  return `unlockos.auth.${facilityId}`;
}

This is not a server-side vulnerability, but it produces exactly the class of incident — an action taken against the wrong tenant — that a tenant-isolation model is supposed to prevent.

2. Transient vs. terminal: the most under-modeled distinction

Several fixes in this cycle were all the same bug wearing different clothes: a transient failure was treated as a definitive negative answer.

• A flaky read of payment settings was treated as "settings are unusable" and used to skip a member check.
• A failure to acquire a check-in lock was treated as "there is no session".
• A resource_missing from an upstream cancel was treated as "the subscription does not exist".
• Repair sweeps retried key issuance that could never succeed, forever.

The cure is to model failure as a type, not a catch:

export type Outcome<T> =
  | { kind: 'ok'; value: T }
  | { kind: 'absent' }                              // verified negative
  | { kind: 'rejected'; code: string; message: string } // terminal, do not retry
  | { kind: 'unavailable'; cause: unknown };        // transient, retryable

export function toHttpStatus(outcome: Outcome<unknown>): number {
  switch (outcome.kind) {
    case 'ok': return 200;
    case 'absent': return 404;
    case 'rejected': return 409;
    case 'unavailable': return 500;
  }
}

The status mapping is the contract. 409 says the request was understood and refused for a stable reason; retrying unchanged will fail again, and the message is safe to show a human. 500 says we do not know; retry is appropriate. Collapsing unavailable into rejected is the dangerous direction: it converts "we could not read your settings" into "your settings are broken", and the client stops retrying something that would have succeeded.

The lock case deserves emphasis because it is access-control-adjacent:

const lock = await tryAcquireCheckinLock(stayId);
if (lock.kind === 'unavailable') {
  // Someone else holds the lock, or the lock store is down.
  // This is NOT evidence that the session is absent.
  return { kind: 'unavailable', cause: lock.cause };
}

If a lock timeout had been allowed to mean "no active session", a concurrent request could have driven the stay into the wrong state. Ambiguity must never resolve toward the permissive answer.

The inverse failure — infinite retry of a terminal error — is equally real. A repair sweep needs a terminality predicate and a persisted marker:

const TERMINAL_ISSUANCE_CODES = new Set([
  'plan_not_found',
  'window_already_past',
  'device_decommissioned',
]);
if (TERMINAL_ISSUANCE_CODES.has(result.code)) {
  await markIssuanceUnrecoverable(jobId, result.code);
  return;
}
await scheduleRetry(jobId, backoff(attempt));

Without this, a dead job consumes a worker slot on every sweep forever, and real failures get lost in the noise of a permanently red queue.

3. Idempotency for anything that moves money or opens a door

Two defects here: a plan-change charge key that could replay a charge already made, and a settlement that queued a second receipt when the balance was already zero.

An idempotency key must be derived from the full semantic content of the operation, not just the entity it targets. A key of plan-change:{membershipId} collides across two legitimate different charges and, worse, lets a stale retry re-apply an amount that has since changed.

export function chargeIdempotencyKey(input: {
  membershipId: string;
  fromPlanId: string;
  toPlanId: string;
  amountMinor: number;
  periodStart: string;
}): string {
  return [
    'plan-change',
    input.membershipId,
    input.fromPlanId,
    input.toPlanId,
    String(input.amountMinor),
    input.periodStart,
  ].join(':');
}

And the guard before emitting a side effect should be a state assertion, not a flag:

if (settlement.balanceMinor === 0 && settlement.receiptIssuedAt !== null) {
  return { kind: 'ok', value: { receipt: 'already_issued' } };
}

The general rule: side-effecting handlers should be written so that calling them twice with the same input is indistinguishable from calling them once. Webhook deliveries, payment returns, and repair sweeps all guarantee you will be called twice.

4. Orthogonal lifecycles need orthogonal state

A reservation was displaying as "not completed" because the stay it pointed at was still checked_in. That is a modelling error, not a display error: two independent lifecycles had been squeezed into one derived status.

A reservation is a commercial record. A stay is an occupancy record. They advance on different clocks and can legitimately disagree — a completed reservation whose guest has not yet physically left is a normal state, not a contradiction.

type ReservationState = 'draft' | 'confirmed' | 'cancelled' | 'completed';
type StayState = 'pending' | 'checked_in' | 'checking_out' | 'checked_out';
export function displayStatus(r: ReservationState, s: StayState | null) {
  if (r === 'cancelled') return 'cancelled';
  if (r === 'completed') return 'completed';   // independent of stay
  if (s === 'checked_in') return 'in_stay';
  if (s === 'checking_out') return 'checking_out';
  return r;
}

The checking_out state is itself a lesson: auto-checkout is asynchronous, so the UI needs a real in-progress state rather than flipping optimistically between two terminal states. Any transition that involves an external system — a payment, a lock command — needs its intermediate state named, or users will be shown a lie for the duration.

The same logic applies to deletion guards. When one record is referenced by audit-style logs (delivery logs referencing an entrance, say), the correct move is usually to allow reclassification while preserving the historical reference, rather than blocking the operation. Audit records should be immutable history, not foreign-key handcuffs on present-day configuration.

5. Validity windows: validate, and fail closed on absence

A quick-issue access URL is a bearer credential. Two gates were added:

1. Refuse a URL used outside its [valid_from, valid_until] window.
2. Refuse a plan that has no valid_until at all.

The second is the one teams skip. A missing expiry is not "no restriction" — it is a missing input, and for a credential that opens a door, a missing input must deny.

export function checkIssuanceWindow(
  plan: { validFrom: string | null; validUntil: string | null },
  now: Date,
): Outcome<void> {
  if (!plan.validUntil) {
    return { kind: 'rejected', code: 'plan_missing_valid_until', message: 'Plan has no expiry.' };
  }
  const from = plan.validFrom ? new Date(plan.validFrom) : null;
  const until = new Date(plan.validUntil);
  if (Number.isNaN(until.getTime())) {
    return { kind: 'rejected', code: 'plan_invalid_valid_until', message: 'Plan expiry is unparseable.' };
  }
  if (from && now < from) return { kind: 'rejected', code: 'window_not_started', message: 'Not yet valid.' };
  if (now >= until) return { kind: 'rejected', code: 'window_expired', message: 'Expired.' };
  return { kind: 'ok', value: undefined };
}

Related: a date-range bug was fixed by computing ranges on the facility's clock rather than the server's. Timezone handling in an access-control system is a security concern, not a formatting concern — a window computed in the wrong zone is a window that is open for nine hours it should not be.

6. Make drift a build failure

One CI change guards an offset list against TypeScript/SQL drift. This is the cheapest high-value test a team can write: whenever the same enumeration exists in two places — a TS union and a Postgres enum, a constant list and a CHECK constraint — assert equality in CI.

it('TS anchors match the SQL enum', async () => {
  const sql = await readFile('supabase/migrations/anchors.sql', 'utf8');
  const fromSql = [...sql.matchAll(/'([a-z_]+)'::anchor_kind/g)].map((m) => m[1]);
  expect(new Set(fromSql)).toEqual(new Set(EVENT_ANCHORS));
});

The failure mode this prevents is nasty: code that compiles, deploys, and then silently skips a branch at runtime because a value only exists on one side of the boundary.

7. Tests that fail for the right reason

Three test fixes in this batch were all about determinism, and all three are generalizable:

• Pin the clock. Any test touching date ranges or expiry must inject now. A test that passes until the first of the month is not a test.
• Do not depend on lazy-loaded UI. A read-only assertion was coupled to a debug panel's lazy load. Assert against the component under test, not against whatever else happens to render.
• Treat unhandled rejections as failures. A test suite that swallows an unhandled promise rejection will happily pass while the production code path throws. Fix the floating promise; do not silence the warning.

const now = new Date('2026-03-01T00:00:00Z');
expect(checkIssuanceWindow({ validFrom: null, validUntil: '2026-02-28T00:00:00Z' }, now))
  .toEqual({ kind: 'rejected', code: 'window_expired', message: 'Expired.' });

Type-level hygiene belongs in the same bucket. Typing a Supabase lookup as SupabaseClient<Database> rather than any in a guard that performs state writes turns a class of runtime mistakes into compile errors — in a guard, specifically, where a silently-undefined field means the guard does nothing.

Summary

The pattern across all of this work:

1. Layer isolation. App-level authz, credential scope binding, RLS on reads and writes, and privileged-client identity verification. Assume any one layer will be bypassed.
2. Never authorize on ownership alone — unowned resources must fall back to a containing scope, never to "allow".
3. Model failure as a type. absent, rejected, and unavailable are three different answers with three different HTTP statuses and three different retry behaviours.
4. Resolve ambiguity toward denial. A lock timeout is not "no session". A flaky read is not "invalid config". A missing expiry is not "never expires".
5. Terminate terminal failures. Retry forever is its own outage.
6. Derive idempotency keys from full semantics, and assert state before every side effect.
7. Keep orthogonal lifecycles orthogonal, and name your in-progress states.
8. Make cross-boundary drift a CI failure, and make tests deterministic enough that a failure means something.

None of these require a framework. They require deciding, for every branch, what the system should do when it does not know — and writing that decision down in a type.