Skip to content

Claude/flight monitoring app bm8h c#392

Open
ctkubik wants to merge 57 commits intojdholtz:masterfrom
ctkubik:claude/flight-monitoring-app-Bm8hC
Open

Claude/flight monitoring app bm8h c#392
ctkubik wants to merge 57 commits intojdholtz:masterfrom
ctkubik:claude/flight-monitoring-app-Bm8hC

Conversation

@ctkubik
Copy link
Copy Markdown

@ctkubik ctkubik commented Mar 26, 2026

No description provided.

claude added 30 commits March 26, 2026 23:07
Adds a Next.js frontend with dashboard, accounts, reservations, flights,
and settings pages. Includes API routes backed by SQLite for persistent
storage. Adapts the existing Python check-in engine as a DB-driven worker
process. Both run together via supervisord in a single Docker container
(Dockerfile.web) for deployment on Railway or similar platforms.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
- Add railway.json to point Railway at Dockerfile.web
- Copy static files and better-sqlite3 native module into standalone build
- Use absolute /app/data path for SQLite DB (works in both dev and Docker)
- Fix supervisord working directory for standalone server

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Rename original CLI Dockerfile to Dockerfile.cli and make the web
version the default Dockerfile. Simplify railway.json since Railway
auto-detects the default Dockerfile.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
webdriver.py imports IS_DOCKER from lib.config. Created a minimal
config.py in worker/lib/ with just the values needed at runtime.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
- Add login page with username/password auth (set via AUTH_USERNAME
  and AUTH_PASSWORD env vars, defaults to admin/admin)
- Add middleware to protect all routes (redirects to /login)
- Add auth cookie with HMAC-signed token (7-day expiry)
- Add logout button to sidebar
- Move pages into (app) route group so login page has no sidebar
- Fix FakeScheduler to provide reservation_monitor.config.browser_path
  chain that WebDriver._get_driver() expects
- Add minimal lib/config.py stub for worker

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Next.js middleware runs in the Edge Runtime which doesn't support
Node.js crypto module. Rewrote token verification to use the
Web Crypto API (crypto.subtle) with constant-time comparison.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…vations

The worker's process_accounts() previously only refreshed generic headers
and queried an always-empty reservations table. Now it:

- Creates an AccountMonitorStub with account credentials
- Calls WebDriver.get_reservations() to log in and fetch reservations
- Upserts returned reservations into the database
- Deactivates stale/cancelled reservations
- Fetches flight details for each reservation via the SW API
- Handles login errors (deactivates account on bad credentials)

Also adds upsert_reservation() and deactivate_stale_reservations() to
worker/db.py for managing reservation records.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Dashboard fixes:
- Add export const dynamic = "force-dynamic" to all API routes to prevent
  static pre-rendering (was causing stale/empty data)
- Fix datetime comparison: use JS-generated ISO string instead of SQLite
  datetime('now') which doesn't match stored ISO 8601 timestamps
- Increase recent logs from 10 to 20

Activity viewer:
- New /activity page with filterable, paginated worker log viewer
- New /api/activity endpoint with level filter and pagination
- Added Activity link to sidebar navigation

Enriched reservations:
- Reservations page now shows check-in countdown timer per flight
- Better formatted departure date/time (weekday, date, time)
- Added route emphasis with bold airport codes

Fare checking:
- Worker now checks fares every 4 hours for upcoming flights
- Stores reservation_info_json on flights for fare checker access
- Records fare history in new fare_history table
- New /api/flights/[id]/fares endpoint for fare history
- Logs fare drops and fare check results to activity log

Seat preferences:
- New seat_preferences table and /api/preferences endpoint
- Settings page now includes seat preference configuration
- Preferred letters (A-F toggle buttons), preferred rows, fallback letters
- Preferences saved to DB for future seat selection feature

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…rades

Southwest now assigns seats (effective Jan 27, 2026). This update adapts
the app for the new seating model:

Accounts:
- Add is_alist and auto_upgrade_seats flags to accounts table
- Accounts page shows A-List badge (clickable toggle) and auto-upgrade toggle
- Add Account dialog includes A-List and auto-upgrade options
- API supports PATCH for both new fields

Seat discovery (after check-in):
- CheckInHandler now logs the full check-in response structure
- Discovers _links entries containing "seat" for seat selection endpoints
- Extracts seat assignments from passenger data (multiple field names)
- Stores assigned_seat in flights table
- If seat selection links found, attempts to select preferred seat

48-hour seat upgrades:
- Worker checks for flights departing in 47-49 hours
- For A-List accounts with auto_upgrade enabled, attempts seat upgrade
- Queries reservation for seat-related _links
- Logs all discovered links for progressive API discovery
- Uses seat preferences from settings to determine target seats

Frontend:
- Flights table shows assigned seat column
- Settings page text updated for assigned seating model
- Accounts table shows Tier (A-List/Standard) and Seat Upgrade (Auto/Off)

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Complete rewrite covering:
- Feature overview (dashboard, fare monitoring, seat preferences, A-List)
- Architecture diagram showing Next.js + Python worker + SQLite
- Installation instructions for Railway, Docker self-hosted, and CLI
- Web app usage guide for all pages (dashboard, accounts, reservations,
  flights, activity, settings)
- Environment variable reference
- Seat preference configuration guide
- Notification service examples (Telegram, Discord, Slack, etc.)
- Updated FAQ for assigned seating, web vs CLI differences
- Troubleshooting for both web app and CLI

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
403 Fix:
- After login, refresh headers via set_headers() before calling
  view-reservation API (mirrors CLI flow where refresh_headers()
  is called before process_reservations())
- If view-reservation returns 403, automatically refresh headers
  and retry once before giving up
- Reduce max_attempts from 20 to 3 for view-reservation (no point
  retrying auth failures with same stale headers)

Self-healing diagnostics:
- New diagnostics table storing structured failure data: category,
  endpoint, expected vs actual behavior, headers snapshot, response
- log_diagnostic() writes to both diagnostics table and worker_logs
- Diagnostic logging added at all API failure points: process_reservation,
  check-in handler, and will propagate to fare/seat checks
- Categories: auth_failure, api_error, api_change, checkin_failure,
  unexpected_response

Diagnostics frontend:
- New /api/diagnostics endpoint with category filter and pagination
- Activity page now has two tabs: Activity Log and Diagnostics
- Diagnostics tab shows structured entries with expandable detail
  (expected/actual, headers sent, response body)
- Category filter buttons auto-populated from data
- Color-coded by category for quick scanning

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Flights page:
- Redesigned as card-based layout (one card per flight)
- Shows fare change from latest fare check (green for drops, red for
  increases, gray for no change)
- Click to expand: fare history timeline + activity logs side by side
- Shows assigned seat badge, countdown timer, and check-in status
- Helpful empty state message when no flights tracked
- API now returns baseline_fare and latest_fare per flight

Accounts:
- New display_name field for friendly account identification
- Shown prominently with username as subtitle when set
- Inline editing: click pencil icon to rename, Enter to save
- Display Name field in Add Account dialog
- DB migration adds display_name column to accounts table

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Critical fix: RequestError.__init__ received response_body but never
stored it as an instance attribute. Diagnostics were logging empty
response snapshots, making it impossible to see what Southwest returns
for 403 errors.

Changes:
- Add self.response_body = response_body to RequestError class
- Update diagnostic logging to include header count and keys in the
  actual_behavior field for richer debugging
- Add route summary (departure → destination) to collapsed reservation
  view on reservations page

After this deploy, the Diagnostics tab will show the actual 403 response
body from Southwest, which is the key to understanding why the
view-reservation API is rejecting requests.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Southwest's WAF blocks raw HTTP requests from datacenter IPs (Railway)
with empty-body 403 responses. The webdriver browser session passes WAF
because it goes through anti-bot challenges. Solution: route all API
calls through the browser using JavaScript fetch().

New BrowserSession class (worker/lib/browser_session.py):
- Manages a persistent headless Chrome instance
- Navigates to mobile.southwest.com to pass WAF and capture headers
- make_request() executes fetch() inside the browser via
  execute_async_script(), inheriting all cookies and WAF tokens
- Same-origin fetch (mobile.southwest.com -> mobile.southwest.com/api)
- Auto-restarts on crash, stale session (25 min), or 403 response
- Handles account login + navigation back to mobile site after login
- Thread-safe via internal lock

Worker changes:
- Replaced global headers/headers_lock with single BrowserSession instance
- main_loop() starts browser session once, keeps alive across poll cycles
- process_accounts() uses browser_session.login_and_get_reservations()
- process_reservation() uses browser_session.make_request()
- check_fares() and attempt_seat_upgrades() use browser_session
- Removed FakeScheduler/AccountMonitorStub/refresh_headers_via_webdriver

CheckInHandler changes:
- Accepts browser_session instead of raw headers + lock
- Routes check-in API calls through browser_session.make_request()
- Ensures browser session is alive before time-critical check-in

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Destination airport fix:
- Southwest API uses 'arrivalAirport' not 'destinationAirport' for the
  destination field. Changed process_reservation() to read from the
  correct field.
- Try 'code' first, fall back to 'name' for both departure and arrival
  airports (API response structure varies)
- upsert_flight() now updates empty departure/destination_airport fields
  on existing flights when re-processing, so existing records get fixed

SMS text notifications:
- New "SMS Text Notifications" card in Settings with Twilio quick setup
- Form fields for Account SID, Auth Token, From Number, To Number
- Auto-generates the Apprise Twilio URL and adds it as a notification
- Works with Twilio free trial

Test notifications:
- New "Send Test Notification" button in the Notifications section
- POST /api/notifications/test endpoint queues a test message
- Shows success/error feedback inline

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Previously, notification URLs were saved to the database but never
actually used to send messages. Now notifications are fully functional.

New worker/notifications.py:
- Simple Apprise wrapper that reads notification_configs from SQLite
- Functions: notify_checkin_success, notify_checkin_failed, notify_fare_drop,
  notify_test
- Sends to all configured services (Telegram, Twilio SMS, Discord, etc.)

Check-in notifications:
- CheckInHandler._send_notification() sends push/SMS on success or failure
- Includes confirmation number, route, and passenger name

Fare drop notifications:
- Worker sends notification when a lower fare is detected (price < -$1)
- Includes confirmation number, route, and price change

Test notifications:
- "Send Test Notification" button writes a marker to worker_logs
- Worker picks up __TEST_NOTIFICATION__ markers and calls notify_test()
- Sends "This is a test notification" to all configured services
- Fixed the endpoint to return proper JSON (was causing SyntaxError)

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…tall guide

Updated for ctkubik/auto-southwest-check-in fork with all changes:
- Correct repository URL and branch (claude/flight-monitoring-app-Bm8hC)
- All features documented: fare tracking, seat preferences, A-List support,
  display names, browser-routed API calls, diagnostics, SMS notifications
- Railway install: Command Palette volume creation, branch selection,
  RAILWAY_RUN_UID tip for permissions
- Docker install: correct clone URL with branch flag
- Full web app usage guide for all 6 pages including Diagnostics tab
- Notification setup: Telegram, Twilio SMS (built-in form), Discord, etc.
- Self-healing diagnostics section explaining the API change tracking system
- Comprehensive troubleshooting covering all known issues
- Updated FAQ: web vs CLI comparison table, WAF bypass explanation
- Architecture table with all tech stack components

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…PI recording

New capture engine (worker/lib/capture.py):
- CheckInCapture class captures maximum technical data during every check-in
- Screenshots: pre-checkin, post-checkin, and any seat selection pages
- Full API request/response bodies stored as JSON (not truncated)
- Network traffic: ALL request/response pairs captured via CDP listeners
  during the check-in window, with response bodies harvested after check-in
- DOM snapshots: Full page HTML captured at key moments
- Manifest file: JSON inventory of all captured files with timestamps and metadata
- Zero latency impact: All heavy I/O (screenshots, network harvesting, DOM)
  happens AFTER the time-critical check-in POST requests complete

Storage: /app/data/captures/{flight_id}/ on persistent volume

Check-in handler integration:
- _check_in() creates a CheckInCapture, calls start_capture() before and
  finish_capture() after the check-in attempt
- _check_in_to_flight() records both API calls (step1: initiate, step2: confirm)
  with full request bodies and complete response data
- Captures are saved even on failure (for debugging)

Database: New checkin_captures table linking capture directories to flights

API routes:
- GET /api/flights/{id}/captures - List captures for a flight
- GET /api/captures/{id} - Get capture manifest
- GET /api/captures/{id}/files?name=... - Serve individual files (images, JSON, HTML)
  with proper MIME types and path traversal protection

Frontend (flights page):
- Expanded flight detail now shows "Check-In Captures" section
- Screenshot thumbnails with click-to-view-full-size
- Clickable links for JSON response files and DOM snapshots
- Shows file count, total size, and network event count per capture
- Displays capture errors if any occurred

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Fare checking was broken for two reasons:
1. The FareChecker class internally calls make_request() from utils.py
   which uses raw HTTP - gets 403'd by Southwest's WAF on Railway.
   Rewrote check_fares() to route all fare check API calls through
   browser_session.make_request() directly, bypassing the FareChecker
   class entirely.
2. DB queries used datetime('now') which doesn't match ISO 8601
   timestamps stored in departure_time. Fixed get_flights_for_fare_check
   and get_pending_flights to use Python-generated UTC ISO strings.

Also:
- Removed reference to deleted global 'headers' variable in fare checker
- Added proper error logging when reservation_info_json storage fails
- Added activity log entries for fare check progress/failures
- Fare check now follows the full flow: get change page -> build search
  query -> get matching flights -> extract price difference
- Each API call goes through browser_session with proper WAF tokens

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Fare check 400 fix:
- Log full diagnostic entry when change page returns 400, including
  the complete change_link object and error response body
- This will reveal whether the query params, href, or request format
  is wrong for the current Southwest API

Destination airport debugging:
- Log the raw airport data structure from the first flight bound
  (departureAirport, arrivalAirport, destinationAirport fields)
- Shows exact field names and values Southwest returns
- Also try destinationAirport as fallback for arrivalAirport
- This will reveal the correct field name for destinations

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Fare check fix:
- browser_session.make_request() used urlencode() for GET query params,
  which fails on nested dicts (converts them to string repr). Southwest's
  change_link.query contains nested objects like {"key": {"sub": "val"}}.
- Added _flatten_params() helper that recursively flattens nested dicts
  into bracket-notation tuples: [("key[sub]", "val")] for proper encoding.
- Also handles boolean values (lowercase) and lists.

Destination airport fix:
- upsert_flight() previously only updated empty destinations (COALESCE/NULLIF).
  Changed to always overwrite with new value if non-empty, using CASE WHEN.
  This fixes flights that had wrong/empty destinations from the old
  destinationAirport field name bug.

Fare check interval:
- Temporarily reduced from 4 hours to 10 minutes for faster debugging
  iteration. Will restore to 4 hours once fare checks are confirmed working.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…hecking

The 400 error was: "Your session has expired due to inactivity. Please
start over." (VALIDATION__SEARCH_TOKEN__EXPIRED). The passenger-search-token
stored in reservation_info_json expires after a period of inactivity.

Fix: Before each fare check, re-fetch the reservation from the Southwest
API to get a fresh passenger-search-token. The fresh reservation_info is
also saved back to the DB, keeping the stored data current. This also
ensures destination airports get updated with fresh data on each cycle.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Notification deduplication:
- Fare check now compares current price with PREVIOUS check result
- Only sends notification when fare drops FURTHER (new amount < previous)
- Same fare drop on subsequent checks logs silently, no notification
- Prevents duplicate notifications for the same price change

Destination airports:
- Switched from manual field parsing to using the Flight class from
  lib/flight.py which knows the correct Southwest API field names
- Flight class uses flight_info["arrivalAirport"]["name"] which is the
  proven extraction path from the original CLI
- Falls back to manual extraction if Flight class fails

Original booking price:
- New original_price and original_currency columns on flights table
- Extracted from bound.fareProductDetails during reservation processing
- Displayed on flights page above fare change ("Paid $XXX USD")
- Stored only on first insert (not overwritten on updates)

Also:
- Restored fare check interval to 4 hours (was 10 min for debugging)
- Added get_last_fare_check() to db.py for dedup queries

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…ce extraction

Destination airports:
- Rewrote airport extraction to prioritize "code" field from raw bound
  data (3-letter codes like PHX, DEN) instead of relying on Flight class
  which uses "name" (full names like "Phoenix Sky Harbor International")
- Added length check: values > 4 chars are treated as full names, not codes
- Falls back through: raw.code -> raw.name (if short) -> Flight class
- Logs every flight's extracted airports with raw API data for diagnostics

Account deactivation:
- No longer deactivates on any non-429/500 LoginError. This was too
  aggressive - transient errors (502, 503, browser crashes, CAPTCHAs)
  caused permanent deactivation of legitimate accounts.
- Now ONLY deactivates for confirmed "Invalid credentials" errors
  (Southwest error code 400518024)
- Added failure counter: requires 3 consecutive credential failures
  before deactivating (prevents single glitch from killing an account)
- Resets failure counter to 0 on successful login
- All other errors logged as transient, retried next cycle

Original price:
- Removed broken fareProductDetails price extraction (the API field
  only contains fareProductId string, not price data)
- Removed "Paid $XXX" display from flights page
- Fare tracking via fare_history (baseline_fare / latest_fare) remains
  and is the correct way to track price changes

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Destination airports fix:
- Simplified upsert_flight() UPDATE to always set airports unconditionally
  instead of CASE WHEN logic. The diagnostic confirmed correct extraction
  (MKE -> PHX with code fields), but the conditional UPDATE wasn't
  applying to existing records. Now it always overwrites.

Original fare (editable):
- Since Southwest's API doesn't expose booking price, added manual
  input: click "$ Set fare" on any flight card to enter what you paid
- Shows "Paid $XXX" once set, click to edit
- New PATCH /api/flights/[id] endpoint updates original_price in DB
- Stored in the original_price column on the flights table

The fare change display now works alongside the original price:
- "Paid $600" (what you entered)
- "-$133 USD" (fare change from fare checker)
- "Lower fare available!" (status text)

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Comprehensive README update covering all changes since last revision:

New features documented:
- Check-in data capture system (screenshots, API recording, network
  traffic, DOM snapshots) with full section and file table
- Original fare tracking (manual input per flight)
- Smart fare notification deduplication
- Account smart deactivation (3 consecutive failures required)
- Fare check diagnostics category

Updated sections:
- Features reorganized into categories (Core, Account Management,
  Flight Tracking, Fare Monitoring, Notifications, Check-In Learning,
  System)
- Flights page: original fare editing, capture viewer, three-panel
  expanded detail
- Activity page: now has diagnostics tab description
- Architecture table: added Data Capture component
- Troubleshooting: added entries for repeated notifications, unexpected
  account deactivation, fare check 400 errors
- FAQ: added "How Does Fare Monitoring Work?" and "Why Was My Account
  Deactivated?" sections
- Web vs CLI comparison table: added original fare, check-in captures rows

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…tials

The error "Element {input[id='username']} was not present after 10
seconds" occurred because the browser navigated to Southwest's login
page but tried typing credentials after only a 1-3 second sleep.
The page needs more time to fully render, especially on cloud servers.

Fix: Added explicit wait_for_element_visible('input[id="username"]',
timeout=30) before typing. This waits up to 30 seconds for the login
form to appear. If it doesn't appear, takes a diagnostic screenshot
to /app/data/captures/login_form_failed.png and raises a
DriverTimeoutError (treated as transient, retried next cycle).

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…state

Southwest's WAF tokens are domain-specific. The BrowserSession initializes
on mobile.southwest.com to capture API headers, but login happens on
www.southwest.com. The reused browser's WAF tokens from mobile.southwest.com
get rejected by www.southwest.com's WAF, causing 403 on login POST.

Fix: Call _start_internal() at the beginning of login_and_get_reservations()
to restart the browser fresh before every login attempt. This matches the
original WebDriver.get_reservations() design which created a new browser
each time via _get_driver().

Flow after fix:
1. _start_internal() creates fresh browser, navigates to mobile.southwest.com
2. Browser gets clean WAF state and captures API headers
3. Navigate to www.southwest.com/loyalty/myaccount for login (clean WAF)
4. Login succeeds, reservations fetched
5. Navigate back to mobile.southwest.com, re-establish API session

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
New fare check modes (configurable in Settings):
- Same Flight Only: only check your exact booked flight (previous behavior)
- Same Day - Nonstop (default): check all nonstop flights on your route
  that day, find the cheapest option
- Same Day - All Flights: check all flights including connections

How it works:
- The Southwest changeShopping API already returns ALL flights for a
  route/date. Previously we only matched the booked flight number.
- Now the worker applies the selected filter (nonstop or all) and finds
  the lowest fare across matching flights.
- If a better alternative exists, it records the flight number and
  nonstop status in fare_history.

Settings UI:
- New "Fare Check Mode" radio selector with three options
- Visual radio cards with descriptions
- Saved alongside seat preferences

Fare history:
- New columns: best_flight_number, best_flight_nonstop
- Flights page shows "Better: WN 456 (Nonstop)" in fare history entries
- Notifications include alternative flight info when found

Database migrations:
- fare_history: best_flight_number TEXT, best_flight_nonstop INTEGER
- seat_preferences: fare_check_mode TEXT DEFAULT 'same_day_nonstop'

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
claude added 21 commits March 28, 2026 21:38
…ails

Alternative flight data:
- Worker now captures departure time, stop description, and your
  flight's fare from the changeShopping API response
- New DB columns: best_flight_stops, best_flight_depart_time, my_flight_fare
- Flights API now returns all alternative flight fields in latest_fare
- Notifications include departure time and stop info for alternatives

Flights page redesign:
- Two-row card layout: flight info on top, alternative flight below
- "Better Flight" card (green) shown prominently when a cheaper
  alternative exists, displaying:
  - Flight number (WN XXX)
  - Stop description (Nonstop, 1 Stop, etc.)
  - Departure time
  - Fare change amount
  - Savings vs your booked flight
- Seat number shown inline with passenger name
- Original fare (click to set) shown above fare change
- Cleaner layout with countdown and status on the right

Fare history (expanded):
- Each entry shows alternative flight details when available
- "Better: WN 456 (Nonstop)" with departure time

Seat display:
- Seat assignment shown next to passenger name on the card
- Blue text "Seat 12A" format

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
The system discovered Southwest's seat management API during a previous
upgrade attempt:
- POST /v1/mobile-seat-management/reservations
- body: {record_locator, first_name, last_name}

Now the worker actually follows through on the complete seat selection flow:

Step 1: Get reservation and find modifySeats link
Step 2: Call the seat management API to get the seat map
Step 3: Log the FULL response (keys, diagnostics, and file capture)
        for progressive discovery of the API structure
Step 4: Parse the seat map response using flexible recursive parsing
        that handles multiple possible response structures
Step 5: Score available seats based on user preferences:
        - Preferred letter + preferred row = 100 points
        - Preferred letter + any row = 80 points
        - Fallback letter + preferred row = 60 points
        - Fallback letter + any row = 40 points
        - Any other seat = 20 points
Step 6: Select the best seat via the selection endpoint
Step 7: Update the flight's assigned_seat in the DB
Step 8: Send notification confirming seat selection

Full seat map response is saved to /app/data/captures/{flight_id}/
for detailed analysis. All steps logged to activity and diagnostics.

_find_best_seat() helper recursively searches the response for seat
data using multiple common field names (seatNumber, seat, seatLabel,
available, isAvailable, occupied) since the exact API structure is
being discovered progressively.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…, capture screenshots

The seat upgrade was failing with:
- Status 0: browser session not alive when seat upgrade runs
- 403 Forbidden: WAF blocking the seat management API call

Fixes:
1. Call browser_session.ensure_alive() before seat upgrades
2. Navigate to the modifySeatsResponsive URL first to establish
   WAF session for the seat management domain
3. Take screenshot and capture DOM of the seat selection page
   (saved to /app/data/captures/{flight_id}/)
4. Navigate back to mobile site before making API call
5. Better error logging with link details on failure

The responsive URL approach loads the actual seat selection page
in the browser, which naturally passes WAF challenges. The screenshot
and DOM capture provide visual evidence of available seats for
analysis even if the API call still fails.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…cked API

The seat management API (/v1/mobile-seat-management/reservations) is
blocked by Akamai WAF returning 403 even through browser fetch().

New approach: Navigate the browser to the modifySeatsResponsive URL
(https://mobile.southwest.com/air/seat/select-seats) with token params
and interact with the seat map page directly, like a user would.

Flow:
1. Get reservation, extract modifySeatsResponsive link with token
2. Construct URL with query params (CONFIRMATION_NUMBER, FIRST_NAME, etc.)
3. Navigate browser to the seat selection page (passes WAF as page load)
4. Wait for seat map to render (tries multiple CSS selectors)
5. Capture screenshot and DOM of the seat map page
6. Extract all seat elements via JavaScript (3 strategies):
   - data-seat-number attributes
   - Elements with seat-related classes and matching text (e.g., "12A")
   - aria-label containing seat info
7. Filter out unavailable/occupied/disabled seats
8. Score available seats by preferences (100/80/60/40/20 points)
9. Click the best seat element
10. Screenshot after click, look for confirm/save/continue button
11. Click confirmation if found
12. Update assigned_seat in DB, send notification
13. Navigate back to mobile.southwest.com to restore API session

All steps produce screenshots and diagnostic data saved to
/app/data/captures/{flight_id}/ for analysis.

Removed the old API-based _find_best_seat() function since seat
extraction now happens from the live DOM.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…seat captures

Route conflict fix (fixes 404 on fare history and activity log):
- Removed /api/flights/[id]/route.ts which conflicted with subroutes
  ([id]/logs, [id]/fares, [id]/captures) in Next.js App Router
- Moved PATCH endpoint to /api/flights/update/route.ts (POST with flight_id)
- Updated frontend to use new endpoint path

Better Flight display fix:
- Only show "Better Flight" card when alternative is ACTUALLY cheaper
  than booked flight (price_change < my_flight_fare)
- Previously showed whenever any alternative existed, even at same price
- Savings calculation only shown when positive

Worker fix:
- Only record best_alt_flight when its fare is strictly lower than the
  booked flight's fare. Clears alternative data when not actually better.
- Prevents misleading "Better Flight" entries in fare_history

Seat upgrade captures fix:
- After saving seat map screenshot and DOM, now inserts a record into
  checkin_captures table with manifest JSON
- Screenshots will now appear in the Check-In Captures section of the
  flights page expanded detail

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
The /api/flights/{id}/logs, /api/flights/{id}/fares, and
/api/flights/{id}/captures routes were returning 404 in the Next.js
standalone deployment. Dynamic [id] segments in nested API routes
can have issues in standalone mode.

Fix: Replaced all dynamic-segment routes with query-parameter routes:
- /api/flights/[id]/logs → /api/flights/logs?id=xxx
- /api/flights/[id]/fares → /api/flights/fares?id=xxx
- /api/flights/[id]/captures → /api/flights/captures?id=xxx
- /api/captures/[captureId] → /api/captures/detail?id=xxx
- /api/captures/[captureId]/files → /api/captures/files?id=xxx&name=yyy

Updated all frontend fetch calls to use the new URLs.

Removed all [id] and [captureId] directories from the API routes.
All routes are now flat with query parameters, which is more reliable
across all Next.js deployment modes.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Seat upgrade fixes:
- Widened upgrade window from 47-49h (2-hour window) to 2-48h before
  departure. A-List members can upgrade seats from 48 hours until 2
  hours before the flight.
- Added 4-hour cooldown between attempts per flight. System checks
  every cycle but only attempts each flight every 4 hours (better
  seats may open up as passengers change flights).
- Added last_seat_upgrade_attempt column to track per-flight cooldown.
- Comprehensive logging: "Seat upgrade check: N eligible flights",
  cooldown status, current seat, every step of the process.
- Even if a seat is already assigned, keeps checking for preferred
  seats that may become available.

Mobile responsive UI:
- Sidebar: Hamburger menu on mobile (< md), slides in/out with overlay.
  Fixed sidebar on desktop. Close on nav link click.
- App layout: Removed fixed ml-64 on mobile, added pt-14 for hamburger
  button area.
- Flights page: Card layouts stack vertically on mobile (flex-col md:flex-row).
  Better Flight card wraps on mobile. Expanded detail uses single column
  on mobile (grid-cols-1 md:grid-cols-2).
- Accounts page: Account items stack vertically on mobile, badges wrap.
- Reservations page: Expanded flight table has overflow-x-auto for
  horizontal scrolling on mobile. Items stack vertically.
- Settings page: Twilio form uses single column on mobile
  (grid-cols-1 md:grid-cols-2).

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
The seat upgrade was failing with error "0" (JavaScript fetch returned
status 0 = network error). This happens when the browser session is
in a stale state or not on the correct domain for same-origin fetch.

Fix: Call browser_session.start() before each seat upgrade attempt to
get a completely fresh browser session with clean WAF state. This
mirrors how login_and_get_reservations() restarts fresh before login.

Also added:
- Browser URL capture on error for diagnostics (shows what domain the
  browser was on when the fetch failed)
- Diagnostic entry on seat upgrade failure with response body
- Logging of browser restart and header count before API call

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…ailures

The cooldown was being set BEFORE the upgrade attempt, so failed
attempts (error "0", 403) would block retries for 4 hours.

Fix: Moved cooldown timestamp recording to AFTER the seat map page
successfully loads. Failed attempts no longer trigger a cooldown,
so the system retries on the next poll cycle (60 seconds).

Also reduced cooldown from 4 hours to 3 hours for more frequent
seat availability checks within the 48-hour upgrade window.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
ROOT CAUSE FOUND: The .gitignore had `logs/` which matched ANY
directory named "logs" anywhere in the repo, including
frontend/app/api/flights/logs/. When the route was restructured
from [id]/logs to flights/logs, the new file was never committed
because git add silently skipped it.

Fix: Changed .gitignore from `logs/` to `/logs/` (leading slash
restricts the pattern to only the root-level logs directory).

Also added:
- Error state handling on the flights page: when API calls fail,
  shows a red error banner with status code instead of silently
  showing "No logs yet" / "No fare checks yet"
- Health check endpoint at /api/health: returns DB status, flight
  count, account count, and list of all registered routes. Use to
  verify a deployment has the latest code.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
The previous approach tried to:
1. Call the seat management API (blocked by WAF)
2. Navigate to mobile.southwest.com with simple params (wrong URL)

The actual Southwest seat selection requires:
1. Being logged into www.southwest.com
2. Navigating to the reservation page
3. Clicking "Modify seats" button (generates JWT searchToken)
4. Interacting with the seat map page

New flow:
1. Find account credentials for the flight
2. Log into www.southwest.com via browser_session.login_and_get_reservations()
3. Navigate to manage-reservation page with confirmation number
4. Screenshot the reservation page
5. Find and click "Modify seats" button (multiple selector strategies + JS fallback)
6. Wait for seat map to load
7. Screenshot the seat map
8. Extract seats from DOM (data attributes, classes, aria-labels)
9. Score seats by preferences
10. Click best seat, click Continue to confirm
11. Screenshot each step
12. Update assigned_seat, send notification
13. Navigate back to mobile site to restore API session

Every step produces screenshots and diagnostic logs saved to
/app/data/captures/{flight_id}/ for analysis.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…spam, add Check Seats button

Seat upgrade login fix:
- Previously called login_and_get_reservations() which navigated back
  to mobile.southwest.com after login, losing the www.southwest.com
  session. The seat map page requires being logged in on www.southwest.com.
- Now performs an inline login directly on www.southwest.com:
  1. Restart browser fresh
  2. Navigate to www.southwest.com/loyalty/myaccount
  3. Wait for login form, enter credentials
  4. Navigate to manage-reservation page (still on www.southwest.com)
  5. Find and click "Modify seats"
  6. Interact with seat map
- Browser stays on www.southwest.com the entire time, preserving the
  login session through the seat selection flow.

Cooldown spam removed:
- Changed cooldown messages from add_log() (visible in Activity) to
  logger.debug() (only in container logs). No more "cooldown (174min
  remaining)" cluttering the activity feed.

Manual "Check Seats" button:
- New button on every flight card to trigger an immediate seat check
- Bypasses the 3-hour automatic cooldown
- POST /api/flights/check-seats writes a marker to worker_logs
- Worker picks up __CHECK_SEATS_{flight_id}__ markers on next cycle
- Calls attempt_seat_upgrades(conn, force_flight_id=...) which skips
  cooldown and runs immediately
- Alert confirms the check is queued (~60 second processing time)

Screenshots saved at every step:
- 00_after_login.png (login result on www.southwest.com)
- 01_reservation_page.png (reservation with Modify seats button)
- 03_seat_map_loading.png (seat map page loading)
- 04_seat_map.png (loaded seat map)
- 05_after_click.png (after clicking a seat)
- 06_after_confirm.png (after clicking Continue)

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
The system was navigating to the seat change page but never filling
in the confirmation number, first name, and last name fields, and
never clicking "Lookup reservation". It just took a screenshot of
the empty form.

Fix:
1. Navigate to www.southwest.com/air/change-seats/ (the direct
   seat change URL)
2. Detect the lookup form (confirmation number input field)
3. Fill in confirmation number, first name, last name
4. Screenshot the filled form
5. Click "Lookup reservation" button (multiple selector strategies
   + JS fallback)
6. Wait 8 seconds for reservation details to load
7. Screenshot after lookup
8. THEN look for "Modify seats" button or seat map

Screenshots at every step for diagnostics:
- 01_seat_lookup_page.png (the form page)
- 02_form_filled.png (after filling fields)
- 03_after_lookup.png (after clicking lookup)
- 04_seat_map.png (the seat map if found)

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…ton directly

The previous approach navigated to /air/change-seats/ (a lookup form)
and tried to fill in confirmation number + name fields, but failed
because:
1. CSS :contains() selectors don't work in standard Selenium
2. The form field IDs were guesses that didn't match Southwest's HTML
3. The form_filled gate logic skipped everything if one selector failed

New approach: Stay on the account page after login. The user's
screenshot showed www.southwest.com/loyalty/myaccount displays all
trips with "Modify seats" buttons already visible. No lookup form
needed.

Flow:
1. Login → browser lands on account page
2. Wait 5s for trips to load
3. Screenshot account page (shows all trip cards)
4. JavaScript finds the trip card containing the confirmation number
5. Clicks "Modify seats" link within that card
6. Falls back to clicking the first "Modify seats" on page
7. If not found, logs all visible links for diagnostics
8. Wait for seat map → continue with seat extraction and selection

This eliminates the broken form-filling approach entirely and uses
the same UI path a normal user would follow.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
The login was failing silently. The code typed credentials, slept 8
seconds, and assumed login worked - but the page still showed the
login form with "We are unable to process" error.

Fix: Use the same CDP network listener approach that works for account
processing in browser_session.py:

1. Register CDP listener for Network.responseReceived BEFORE navigating
2. Monitor for login API response at /api/security/v4/security/token
3. Wait for the actual response (polling 0.5s intervals, 30s timeout)
4. Check HTTP status code is 200 before proceeding
5. If login fails: log the status code and error response body
6. Only proceed to find "Modify seats" if login genuinely succeeded
7. After verified login, wait 5s for page to render trips

This replaces the blind time.sleep(8) with an active verification
loop that confirms the login API actually returned success.

Activity log will now show:
- "Login verified (status 200)" on success
- "Login failed with status XXX" on failure (with error details)
- "Login API response not detected" if the form didn't submit

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
From user screenshots, the correct navigation flow on southwest.com is:
1. Login → account page (Trips tab visible)
2. Click "Trips" tab (may already be active)
3. Click "Details" button on the trip card for the target confirmation
4. "Manage my trip" page loads with "Modify seats" link
5. Click "Modify seats" → seat map loads

Previous approach tried to find "Modify seats" directly on the account
page, but it's only available after clicking into trip Details.

Navigation changes:
- Click "Trips" tab after login (screenshot: 01_trips_tab.png)
- Find trip card containing confirmation number, click "Details"
  (walks up DOM from conf number to find parent card with Details btn)
- Falls back to clicking first "Details" button if targeted search fails
- Wait for "Manage my trip" page (screenshot: 02_manage_trip.png)
- Extract current seat assignment from the page text
- Click "Modify seats" link (screenshot: 03_seat_map.png)
- All failures include diagnostic link lists for debugging

Also:
- Filter past flights: Flights API now excludes flights with departure
  time in the past (WHERE f.departure_time > now)
- Filter fare history: Only show entries where price_change or
  best_flight_number differs from previous entry (hides repeated
  "same price" rows)

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…iming)

The "Details" click worked (navigated to /air/manage-reservation/) but
the page content hadn't rendered yet after only 5 seconds. The page
showed footer links only - the "Modify seats" link wasn't in the DOM.

Fix: Replace blind time.sleep(5) with a polling loop that checks for
actual page content every second for up to 20 seconds. Checks for
"Modify seats", "Manage my trip", or "Seat Assignments" text in the
page body. Only proceeds when content is found (or after 20s timeout
with diagnostic logging).

Southwest.com is a React SPA - the page URL changes immediately on
click but content renders asynchronously via JavaScript.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…ails

After clicking the Trips tab, the URL changes to /trips/upcoming
immediately but the trip cards render asynchronously via JavaScript.
The previous 5-second wait wasn't enough - the confirmation number
wasn't in the page DOM yet.

Fix: Poll for the confirmation number in the page HTML every second
for up to 20 seconds after clicking Trips tab. Only proceed to
find "Details" button when the confirmation number is confirmed
present in the DOM.

Also saves DOM as HTML file on failure for diagnostics.

Logs now show: "Trips content loaded after Ns (found C3GDAI)" to
confirm when content renders.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Root cause of 403050700: Each seat upgrade attempt created a fresh
browser and login. With 60-second polling and manual Check Seats
button, this caused many login attempts per hour, triggering
Southwest's rate limiter (error code 403050700).

Fix: Reuse the existing browser session instead of creating new logins.
The browser already has www.southwest.com cookies from the regular
account processing login. The seat upgrade now:

1. Navigates the EXISTING browser to /loyalty/myaccount/trips/upcoming
2. Checks if already logged in (looks for "Hi," or "Upcoming Trips")
3. If logged in: proceeds directly to find Details/Modify seats
4. If NOT logged in: attempts login with 30-MINUTE THROTTLE
   - Max one login attempt per 30 minutes
   - If throttled, logs warning and skips (waits for next cycle)
   - Verified with CDP listener (same proven approach)

This reduces login frequency from "every seat check" to "at most
once per 30 minutes" and only when the session has actually expired.
Most seat checks will reuse the existing session with zero new logins.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Flights API N+1 fix:
- Previously ran 2 queries per flight (latestFare + firstFare) in a
  loop, causing 21 queries for 10 flights (1 + 10 + 10)
- Now uses a single query with LEFT JOIN subqueries for latest and
  baseline fares, reducing to 1 query total regardless of flight count
- Subqueries use MAX/MIN(checked_at) with GROUP BY to find latest/first
  fare check per flight

Capture manifest fix:
- Manifest listed filenames that didn't match actual screenshots
  (01_reservation_page.png vs 01_trips_tab.png)
- Updated to match actual files: 01_trips_tab.png, 02_manage_trip.png,
  03_seat_map.png, seat_map_dom.html

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Flight C3GDAI's check-in FAILED because the seat upgrade function
held browser_session._lock for 10+ minutes while navigating
www.southwest.com. When the check-in thread (24h before departure)
tried to acquire the lock, it was blocked. When it finally got the
lock, the browser was on www.southwest.com (not mobile.southwest.com),
causing fetch() to fail with status 0 (same-origin policy violation).

Three fixes:

1. CRITICAL: Check-in proximity guard (worker.py)
   - attempt_seat_upgrades() now checks if ANY check-in is within
     2 hours. If so, seat upgrades are PAUSED entirely.
   - Check-in is always the jdholtz#1 priority. Seat upgrades can wait.
   - Activity log shows: "Seat upgrades PAUSED: check-in for X in Y minutes"

2. HIGH: Fresh browser session before check-in (checkin_handler.py)
   - 30 minutes before check-in, the handler now calls
     browser_session.start() (not just ensure_alive())
   - This creates a completely fresh browser on mobile.southwest.com
   - Guarantees the browser is on the correct domain with fresh WAF
     tokens, regardless of what seat upgrades did earlier

3. MEDIUM: Lock timeout in make_request (browser_session.py)
   - Changed from blocking `with self._lock:` to timed acquisition
     `self._lock.acquire(timeout=60)`
   - If the lock can't be acquired within 60 seconds, logs a warning
     and retries instead of blocking indefinitely
   - Prevents check-in from being stuck waiting for seat upgrade forever

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
@jdholtz
Copy link
Copy Markdown
Owner

jdholtz commented Mar 31, 2026

Hi @ctkubik, I appreciate your effort on this. However, this PR is super large (over 15,000 lines) and infeasible for me to review. Could you split it up into multiple PRs with one feature each so they can be more easily reviewed and we can discuss each one individually?

@ctkubik
Copy link
Copy Markdown
Author

ctkubik commented Mar 31, 2026 via email

claude added 6 commits March 31, 2026 20:36
The old display was confusing:
- "+$25" in red: the "+" suggests positive/good but red means bad
- "$133" in green: no direction indicator, just a number
- Users couldn't tell at a glance if fare went up or down

New display uses clear directional labels:
- "▼ $133 cheaper" in green = fare dropped (good, saves money)
- "▲ $25 more" in red = fare increased (bad, costs more)
- "No change" in gray = same price
- "No fare data" in gray = not checked yet

Better Flight card simplified:
- Now shows just "Save $67" instead of the confusing "+$25 Save $438"
- The savings amount is the actual difference between your fare and
  the cheaper alternative

The underlying data is unchanged - Southwest's priceDifference API
returns negative for lower fares and positive for higher fares.
Only the presentation changed.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…ole, screenshots

New SeatUpgradeAudit class (worker/lib/seat_audit.py):
- Tracks each step of the seat upgrade with begin_step()/end_step()
- Captures browser console logs (driver.get_log('browser')) at each step
- Captures browser state: URL, page title, cookie count
- Takes screenshots at every step transition
- Records timing (duration_ms) for each step
- Saves DOM snapshots at key moments
- Stores everything in seat_upgrade_audit DB table + JSON files

Database: New seat_upgrade_audit table with:
- steps_json: Full array of step records with timing, URLs, data
- browser_console_json: All browser console messages
- capture_dir: Path to screenshot/DOM files
- status: success/failed/partial
- error_message: If failed, why

Instrumented steps in attempt_seat_upgrades():
1. navigate_trips - Browser navigation + login check
2. click_trips_tab - Wait for SPA content to render
3. click_details - Find and click trip Details button
4. wait_manage_trip - Wait for Manage my trip page
5. click_modify_seats - Find and click Modify seats link
6. wait_seat_map - Wait for seat map to load
7. extract_seats - DOM extraction + scoring
8. click_seat - Click best seat + confirm

Frontend:
- New /api/flights/audits endpoint
- "Seat Upgrade Audit Trail" section in expanded flight detail
- Shows each step with ✓/✗, timing, URL, console errors, and data
- Color-coded status (green=success, red=failed, yellow=partial)

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
…p, polling

- Block images/fonts/media via CDP Network.setBlockedURLs in browser sessions
- Increase SESSION_MAX_AGE from 25min to 2hrs (80% fewer restarts)
- Add daily data retention cleanup: worker_logs 30d, diagnostics 30d,
  fare_history 90d, seat_upgrade_audit 90d, captures 30d after departure
- Clean up stale capture directories from disk
- Reduce frontend polling from 15-30s to 60s on all pages
- Pause polling entirely when browser tab is hidden (visibilitychange)
- Fix N+1 query in reservations API with single LEFT JOIN

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Use explicit key filtering instead of destructuring to separate
flight fields from reservation fields, avoiding unused variable warnings.

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
… page

The seat upgrade was failing because navigating to the authenticated
trips page (www.southwest.com/loyalty/myaccount/trips/upcoming) requires
working www.southwest.com API auth tokens, but the browser session lives
on mobile.southwest.com. The loyalty management API returned 401/403,
so trip cards never rendered despite the page shell showing "Hi, [name]".

New approach: use the public manage reservation page which only needs
confirmation number + first name + last name (no login required).
This eliminates the domain-specific auth token problem entirely.

Flow: manage-reservation form → fill conf/name → submit → trip details
→ Modify seats → seat map → select best seat → confirm

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
- Tighten seat_upgrade_audit and checkin_captures DB retention from
  30/90 days to 5 days (these store large JSON blobs)
- Add filesystem sweep: walk /app/data/captures/ and delete any files
  with mtime > 5 days, regardless of flight status. Removes empty dirs.
- Add SQLite VACUUM after cleanup to reclaim disk space
- get_stale_capture_dirs() now uses created_at instead of flight
  departure_time, catching captures from active flights too

https://claude.ai/code/session_01XeexS9xn1dz8wb9K11MkpV
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants