Owner
Receives inquiry drafts, reviews every outbound response, and gets failure alerts when a workflow breaks.
Business Problem
NTStays receives inquiries and bookings from multiple channels, each with different data semantics. Customer inquiries were handled manually, source attribution was inconsistent, and platform reporting mixed monthly, rolling-window and lifetime metrics. The operating question was simple but hard to answer reliably: which channel is actually producing valuable business, at what cost?
Users & Stakeholders
Three audiences rely on the system in different ways.
Marketing manager
Uses the dashboard, logs platform metrics, and reviews channel economics across Airbnb, Vrbo, Furnished Finder and direct acquisition.
Guests & leads
Use the assistant and request forms, while the system keeps pricing commitments and final customer communication under human control.
Production Capabilities
Connected workflows support lead capture, qualification, communications, payments, reporting, and operational follow-up.
Lead operations
Captures source, property interest, and customer details in a structured record for follow-up.
AI assistance
Scores leads, returns structured output, and prepares response drafts for human review.
Customer experience
Answers common questions from approved property and service information and escalates exceptions.
Marketing analytics
Automates marketing-performance collection, normalization, reporting, and stakeholder communication.
Revenue operations
Supports reliable follow-up for booking, payment, and customer-status actions.
Observability
Tracks workflow health, execution status, exceptions, and incident alerts.
Production Workflows
All workflows are generated from code, versioned in Git, tested, and deployed through GitHub Actions. Production components are used by the business; additional flows have been built and tested and activate as credentials and operations are enabled.
| Workflow | Nodes | What it does | Status |
|---|---|---|---|
| Website inquiries | 75 | Validates requests, runs spam checks, scores and drafts with Claude, creates HubSpot records, routes an approval form, then sends an optional Stripe payment link only after approval. | Live |
| Website assistant | 16 | Answers from approved facts and availability; uses structured output; enforces price and booking-confirmation guardrails; hands off only after explicit consent and a valid email. | Live |
| Marketing data collection and audit system | 17 | Captures platform metrics that cannot be exported cleanly in an append-only data table with attribution. | Live |
| Monthly channel report | 25 | Computes funnels, stays, revenue share and fee percentages in code; Claude generates draft content as structured JSON; unsupported numbers are blocked; owner edits are re-checked. | Live |
| Operational monitoring and incident alerts | 3 | Emails the owner with the failing step when a production workflow errors. | Live |
| Payments | 15 | Stripe webhook confirms direct bookings, updates customer state and publishes the direct-booking calendar. | Built & tested |
| Follow-ups | 20 | Handles unpaid links, thank-yous, and extension outreach through a human-in-the-loop AI workflow. | Built & tested |
| Availability | 8 | Merges Airbnb, Vrbo and Booking.com iCal feeds so the site and assistant know when properties are open. | Built & tested |
System Architecture
The design separates probabilistic AI reasoning from deterministic business controls and durable system state.
Visitor /
campaign source
campaign source
→
Website form
/ chat
/ chat
→
n8n validation
& spam checks
& spam checks
→
Claude API
Score leads · draft responses · return structured JSON
Score leads · draft responses · return structured JSON
→
Code guardrails
schema · prices · booking claims
schema · prices · booking claims
→
Human-in-the-loop
AI workflow
AI workflow
→
CRM / email /
payment actions
payment actions
| Architecture decision | Why |
|---|---|
| n8n for orchestration | Visual orchestration, webhooks, scheduled jobs, branching, API integrations and operational transparency. |
| Claude API for probabilistic tasks | Lead scoring, structured drafting and narrative synthesis are appropriate places for language-model reasoning. |
| Python / JavaScript for deterministic logic | Metrics, business rules and validation should not depend on model interpretation. |
| Structured JSON outputs | Model output is machine-validated before it can enter downstream systems. |
| HubSpot as durable customer state | Customer history belongs in the CRM rather than opaque model memory. |
| Human-in-the-loop AI workflow | Customer-facing communication has pricing, commitment and brand risk. |
| Append-only measurement data | Corrections remain auditable instead of overwriting historical evidence. |
| Tests gate deployment | Generated workflows and validation rules must pass before production changes ship. |
Governance & Reliability
Controls
- Human-in-the-loop approval before customer communications
- Structured outputs for predictable downstream processing
- Input validation and required-field checks before routing
- Role-based access controls for sensitive actions
- Workflow logging and approval audit trails
Operational resilience
- Workflow monitoring, run history, and incident alerts
- Error handling and recovery paths for failed automations
- Data-quality checks for attribution and reporting inputs
- Version-controlled workflow changes and deployments
- Escalation paths for ambiguous requests
Failures & Fixes
A production system becomes credible when the failure modes are visible. These were some of the most useful engineering lessons.
| Failure | Root cause | Change made |
|---|---|---|
| Test data appeared on the production dashboard | The stats test suite wrote sample output into the production data file. | Production builds now use only real exports; a regression test verifies tests never touch the production path. |
| Chat refused real visitors | The Cloudflare Turnstile challenge was off-screen, so users could not complete it. | The widget was made visible, with a 30-second wait and explicit Cloudflare error codes. |
| The website deployed before its backend | Site deployment was automatic but n8n import was still manual. | Added tested, key-restricted automated n8n deployment. |
| Updating one Airbnb rate erased another | The initial “latest row replaces the month” logic was too broad. | Metrics now merge field-by-field; blank values never erase known data. |
| Backfilled leases distorted conversion | Leases were dated by entry date rather than signed date. | Added signed-on date, defaulting appropriately to move-in. |
| Furnished Finder funnel math was misleading | Inquiry totals were cumulative while views were rolling 90 days. | Monthly inquiries are derived from entry differences, per listing; incompatible periods are not divided. |
| Prompt caching was assumed but not active | The prompt was below the model's caching threshold. | Documented the behavior and accepted the negligible cost at this volume. |
Key Learnings
The project reinforced several principles I would carry into larger enterprise AI systems.
1. Put rules in code
A prompt can request safe behavior. A deterministic check can enforce it.
2. Data semantics matter more than charts
Rolling windows, running totals, averages and business dates caused more risk than the visualization layer.
3. Tests must be isolated from production
The most damaging bug came from the test environment writing to a production data path.
4. Label what every number means
Thin or mixed-period data should be visibly qualified so business users do not over-read it.
5. Match metrics to business maturity
The right KPI is the one the business can act on now, not the most impressive-looking dashboard metric.
6. Automate reality, not an idealized API landscape
Where platforms lacked usable host APIs, the workflow made manual input fast, structured and auditable instead of pretending full automation existed.
Next Opportunities
- Richer lead scoring that adds channel, budget, and engagement signals to the inquiry itself
- Retrieval over a larger approved knowledge base (house guides, policies) for the chat assistant
- One review queue for every pending approval (today each one is its own email form)
- Chat-interaction attribution alongside the existing UTM, campaign, and referrer tracking
- A/B testing for calls to action, intake fields, and response timing
- Lifecycle automation for booking reminders, reviews, and repeat stays
- Automated quality evaluation for factuality, brand tone, and escalation accuracy
Production Screenshots
Production evidence from the live NTStays Growth Ops environment. Customer and guest details are redacted where applicable.
