Alpha → Production QA Matrix (2026-05-12)

Scope

This matrix covers QA for promoting alpha → production for the API and service web app.

Commit window on alpha (not on main): 2026-03-16 → 2026-05-12

Primary feature groups:

• Dynamic dashboards (API + Dashboard V2 UI)
• Super-admin leads inbox and public lead capture
• MFA, refresh-token lifecycle, and auth hardening
• Stripe billing checkout, webhooks, and super-admin billing UI
• Tier / feature-flag gating (Starter, Growth, Pro)
• Trucking dispatch scheduler (timing alerts, resource day view, timeline DnD)
• Tags, self-serve onboarding, tutorial progress, settings tenant isolation
• Jobs: rates, leg create/show, leg audit on reports

Release report (dev + deploy detail): Alpha → Production Release Report (2026-05-12)

Not in scope: feature/tenant-roles-permissions (not on alpha). Mobile native/OTA unless separately released.

Environments

• API branch: alpha → promote to main via PR #180 (refresh before merge)
• Service branch: alpha → promote to main via PR #177 (refresh before merge)
• API version (alpha): 2.10.13 (prod main currently 2.10.14 until next promote bump)
• Service version (alpha): 2.31.0 (prod main currently 2.28.21)
• App type: Trucking primary; service/repair secondary; super-admin backoffice
• Test tenants: one Starter-tier tenant, one Growth-tier tenant, one super-admin user; optional VIP tenant with grandfatherFullAccess if testing bypass

Pre-Test Setup

1. Deploy or use the latest alpha API and service builds (paired hosts).
2. Prepare users: tenant admin, standard user, driver (mobile if testing API contract), super admin.
3. Configure Stripe test keys and webhook endpoint pointing at alpha API.
4. Configure Turnstile for public lead forms on alpha (or confirm test bypass behavior).
5. Have at least one trucking job with legs, rates, and dispatch timing alert candidates.
6. Have one incomplete lead in super-admin inbox and one public lead submission path ready.
7. For Dashboard V2: confirm tenant planTier and dashboards.* feature flags for on/off cases.
8. Refresh open alpha→main PRs so they include the 2026-05-12 beta→alpha merge before production sign-off.

Matrix

ID	Area	Scenario	Steps	Expected Result
Q1	Auth	Login and session refresh	Sign in on service; use app normally; trigger token refresh (wait or reload)	Session remains valid; no unexpected logout loops
Q2	Auth	MFA challenge on login	Sign in as user with MFA enabled	MFA challenge appears; successful verify completes login
Q3	Auth	MFA recovery	Use forgot-password / MFA recovery path per product flow	User can recover access with clear errors on invalid input
Q4	Auth	Refresh token versioning edge case	Sign in on two browsers; sign out one; refresh the other	Behavior matches policy; no silent auth corruption
Q5	Tier gating	Starter tenant blocked features	As Starter admin, open dashboards, route optimization, or tier-gated settings	Gated UI hidden or upgrade messaging shown; API returns consistent errors if forced
Q6	Tier gating	Growth tenant unlocked features	As Growth admin, repeat Q5 scenarios allowed for tier	Allowed features work; caps (e.g. saved reports) enforced per tier
Q7	Dashboard V2	Create and save dashboard	Open Dashboard V2; add widgets; save layout	Layout persists on reload; widgets load data
Q8	Dashboard V2	Role / visibility	View dashboard as admin vs non-admin roles	Widget visibility respects role settings
Q9	Dashboard V2	Tier flag off	Tenant with dashboards disabled in flags/tier	Safe empty state or redirect; no broken API errors
Q10	Dispatch	Timing alerts	Open dispatch with timing review / attention leg	Alert state renders; workflow not incorrectly blocked
Q11	Dispatch	Resource day view	Switch to resource day view; scroll time grid	Sticky headers and events readable; no layout break
Q12	Dispatch	Timeline drag (optimistic)	Drag event on timeline where supported	UI updates optimistically; server state syncs or rolls back cleanly
Q13	Jobs	Job show rates	Open job; edit rates	Rates save; dashboard/job views stay consistent
Q14	Jobs	Leg create	Create leg from job show flow	Leg created; appears in job and dispatch contexts
Q15	Jobs	Leg audit on report	Open report path with leg audit	Audit data visible; leg updates do not corrupt report
Q16	Settings	User tenant isolation	As tenant A admin, attempt to access tenant B user via settings/API	Access denied or 404; no cross-tenant data leak
Q17	Tags	Create and assign tags	Create tag; assign to job/customer	Tag persists; list/filter works
Q18	Super admin	Leads inbox	Open leads list; filter and open detail	List loads; detail matches record
Q19	Super admin	Lead note and status	Add note; change status and assignee	Updates persist and appear for other super admins
Q20	Super admin	Create customer from lead	Convert lead to customer	Customer created; lead state updated appropriately
Q21	Public leads	Submit lead form	Submit valid public lead with Turnstile	201/success; lead appears in admin pipeline
Q22	Public leads	Rate limit / bot protection	Submit repeatedly from same IP	Rate limit or Turnstile blocks abuse with clear response
Q23	Billing	Tenant billing summary	Tenant admin opens billing in settings	Summary loads or safe empty state
Q24	Billing	Checkout session	Start checkout from billing UI	Redirect to Stripe test checkout without broken navigation
Q25	Billing	Webhook sync	Complete or simulate subscription event; refresh billing	Customer billing state updates; no duplicate webhook processing
Q26	Billing	Super-admin billing view	Super admin opens customer billing tab	Billing data renders for Stripe-backed test customer
Q27	Onboarding	Self-serve signup	Complete self-serve onboarding on alpha	Tenant/user created; can sign in
Q28	Onboarding	Tutorial progress	Complete or skip tutorial steps; reload	tutorialProgress persists on user
Q29	Onboarding	CSV import (if enabled)	Upload valid CSV via onboarding path	Import succeeds or shows row-level errors
Q30	Search / approvals	Scoped search	Search jobs/approvals as tenant user	Results scoped to tenant only
Q31	Chat	Thread and read state	Open messages; send message; return to list	Send works; unread state updates
Q32	Reports	Saved report tier cap	Create reports up to tier limit	At cap, enforcement blocks or warns per enforcementMode
Q33	Mobile API	Legs mobile dashboard (if mobile tested)	Driver opens mobile dashboard for legs	Triptych sections (active/today/upcoming) render without client errors
Q34	Regression	Core job workflow	Create job → assign → complete leg path	End-to-end trucking flow still works
Q35	Regression	Manifest / invoice	Open manifest and invoice flows touched by release	No regression in display or edit

API Verification Checklist

• Health check passes on alpha after deploy.
• GET/POST dashboard routes under /api/v1/dashboards with tenant scoping.
• Admin leads routes: list, get, notes, status, assignee, create customer from lead.
• POST public leads with Turnstile and rate limiting.
• Billing: summary, checkout-session, portal; webhook handler updates customer.billing.
• MFA challenge and refresh endpoints; refreshTokenVersion increments on login/refresh.
• Legs getById / update return 404 for cross-tenant IDs (not 403).
• Legs mobileDashboard response shape (triptych) for mobile consumers.
• Feature flag resolution per planTier and grandfatherFullAccess.
• API starts without Tag duplicate model error (post PR #185 fix).
• Stripe env vars present in production before merge.

Service Verification Checklist

• Login, MFA challenge, forgot password, and session flows on alpha service host.
• Dashboard V2 builder CRUD and widget data loading.
• Dispatch scheduler: timing alerts, resource day view, timeline interactions.
• Job show: rates, leg create, tags UI.
• Settings → Users: no cross-tenant leakage.
• Super-admin: customers, leads inbox, billing tabs.
• Billing UI navigation from tenant settings.
• Tier-gated menu items and routes hidden or blocked appropriately.
• Hosted app-link / association files if testing deep links from email.

Mobile Verification Checklist

(Only if mobile build is promoted with or soon after this API release.)

• Driver login and session against alpha API.
• Legs mobile dashboard sections match API contract.
• Push/token behavior unchanged for auth changes.

Regression Checklist

• Existing login/logout for web users without MFA.
• Existing job create/edit/complete for trucking tenants.
• Existing manifest and invoice flows.
• Existing chat threads and notifications.
• Manually billed tenants (no Stripe) still see safe billing empty states.
• Super-admin routes block non-super-admin users.
• Search and approval flows remain tenant-scoped.

Exit Criteria

• All P0 flows in the release report pass on alpha (auth, MFA, tier gates, Dashboard V2, dispatch, jobs, settings isolation, leads, billing, onboarding, tags).
• No blocker: API startup failure, auth outage, cross-tenant data leak, or Stripe webhook misconfiguration.
• Promote PRs updated to include 2026-05-12 alpha tip.
• API merged to main and health-checked before or simultaneous with service merge.
• Production smoke: login, one job flow, Dashboard V2 or gated fallback, one billing read path.
• Known risks documented: refresh-token churn, mobile/API contract skew if mobile not released, seat-cap mode in production env.

Metadata

• Paired doc: alpha-to-production-release-2026-05-12.md
• Last updated: 2026-05-16