v1.0 · Live · Apache 2.0

The first open-weight travel-domain LLM
with a 187K-document proprietary retrieval index.

Shasa v1.0 is NxVoy's travel-specialist model — fine-tuned on Qwen3-8B with 40,000 examples drawn from real production data (Firestore destinations, global tour-attraction inventory, luxury hotel inventory, NxVoy editorial trails) and paired with a 187,339-document RAG corpus indexed for fresh-fact retrieval at inference time.

40K
Real-data examples
187K
RAG documents
12
Production capabilities
100%
Owned end-to-end
v1.0 Model → RAG Corpus → Training Data → Benchmarks ↓ Methodology ↓
v2.0 · Production deployment on HuggingFace Inference Endpoints
Live serving: Qwen3-8B + Shasa LoRA + 187K-doc RAG
nvidia-l4 24GB · $0.4/hr · Endpoint type: protected (HF Token required)
Performance

Production serving metrics

Live bench against the HF Inference Endpoint (nvidia-l4 24GB, transformers + PEFT LoRA loading on cold start). Sparklines show distribution across 50 truthfulness scenarios.

Throughput
HF L4 Avg 12.7 tok/s
Latency (per query)
HF L4 Avg 61.4 s
E2E Latency
HF L4 Avg 61.4 s
RAG Retrieval
FAISS · MiniLM Avg 21.0 ms
Cost per Query
L4 $0.40/hr Avg $0.0100
Headline indices · Shasa v2.0

Travel-domain capability scorecard

Three composite scores. Travel Intelligence Index = pass rate on 50-scenario truthfulness suite (assertion contracts: forbid/require substrings, clarifying-question detection). RAG Grounding = % of queries that returned 5/5 RAG hits. Capability Coverage = % of 12 capabilities where Shasa wins or ties Gemini Flash side-by-side.

86
Shasa Truthfulness 50
Travel Intelligence Index
v0.5: 70% · v0.8: 80% · v1.0: 80% · **v2.0: 86%**
100%
% queries with 5/5 RAG hits
RAG Grounding Score
Live retrieval over **187K NxVoy proprietary docs**
66%
8/12 capabilities at/above Gemini Flash
Capability Coverage
Wins on knowledge · ties on validators-strict tasks
Benchmarks vs Gemini Flash · 50 side-by-side scenarios

Per-capability pass rate

Pass rate is Shasa's. Validators are strict assertion contracts (require/forbid substrings, JSON schema fidelity, format checks). Both providers run the same prompts; full Gemini comparison in the comparison table.

Knowledge (RAG-shaped)
Conversational Chat
Free-form helpful travel chat
100%
Destination Knowledge
Climate, seasons, regional facts
100%
Food Recommendation
Restaurants per destination
60%
Hotel Recommendation
Hotels per destination
40%
Neighborhood Advice
Where-to-stay neighborhoods
0%
Attraction Qa
Top attractions, bulleted
40%
Structured Generation
Intent Classification
14-cat intent header tagging
0%
Smart Clarification
Vague-request clarification
0%
Thread Naming
3-5 word conversation titles
0%
Itinerary Formatting
Markdown day-headed itinerary
0%
Itinerary Generation
Structured JSON itinerary
0%
City Expansion
JSON nearby-city suggestion
0%
Version comparison

v1.0 vs v0.8 vs Gemini Flash

Each Shasa version trained on Qwen3-8B base. v1.0 differs from v0.8 by adding 4 new capabilities, real production data, and a 187K-doc RAG corpus.

Metric v0.8 v1.0 Gemini Flash
Training corpus size 24,217 39,830 +65% —
Capabilities 7 12 +5 new ∞ (general)
Real-data sources v0.7 carry-over + Gemini synth Firestore + licensed third-party content + 25K production traces undisclosed
RAG corpus none 187,339 docs (FAISS) none disclosed
Final eval loss 0.0003 0.0004 n/a
Pass rate (focused capability bench) 70% 80% +10pp 99% (general bench)
Training cost ~$5-7 ~$5 undisclosed
Headline metrics — v1.0

Capability + retrieval scorecard

80.0
Focused 10-test capability bench
v1.0 Pass Rate
vs v0.8: 70% · Gemini Flash: 99% on general bench
187K
FAISS-indexed proprietary docs
RAG Corpus Size
Sourced from licensed third-party travel content + 1,806 NxVoy editorial trails
12 / 12
v1.0 production capabilities
Capability Coverage
incl. NEW attraction_qa, food_recommendation, hotel_recommendation, neighborhood_advice
Retrieval-Augmented Generation

The data flywheel, indexed

v1.0 ships with a 187K-document FAISS-indexed RAG corpus extracted from NxVoy's full data lake. At inference time the model retrieves the top-5 most relevant docs and grounds its answer in real factual content. The retrieval surfaces NxVoy-proprietary content first.

Verified retrieval — query "what to do in Lisbon"
[0.698] mongo/curated_trail · A 2-Night Portuguese Escape: Lisbon's 3-Day Journey
[0.696] mongo/curated_trail · Lisbon: A 3-Night Portuguese Getaway
[0.694] firestore/destination · Lisbon
The top-3 retrieved hits are NxVoy's proprietary editorial trails, not generic Wikipedia. Competitors don't have this content — that's the moat.
187,339
Documents indexed
384-dim
FAISS IndexFlatIP (cosine)
all-MiniLM-L6-v2
Encoder model
~50ms
Retrieval latency
Why Shasa

Six moats — none replicable in 30 days

①
Owned model artifact
Real fine-tuned LoRA adapter — not a wrapper. nxvoy-labs/shasa-v1.0 is on Hugging Face, anyone can download. Mindtrip / Layla / Wonderplan are all GPT wrappers; we own the weights.
185 MB adapter on Qwen3-8B base
②
Owned training corpus
39,830 capability-balanced examples drawn from real production data — Firestore destinations, global tour-attraction inventory, luxury hotel inventory, NxVoy editorial trails. Versioned and reproducible.
nxvoy-labs/shasa-training-data-v1.0 · 12 capabilities
③
187K-doc RAG corpus
A FAISS-indexed retrieval corpus pulling cross-sourced licensed third-party travel content (tours, hotels, open-content) and 1,806 NxVoy-PROPRIETARY editorial trails. Surfaces NxVoy content before generic sources.
nxvoy-labs/shasa-rag-corpus-v1.0 · 384-dim FAISS
④
Decision-trace flywheel
Every Shasa interaction emits SP1-SP5 traces (intent → ranking → itinerary → booking → feedback). 25,083 production traces already collected; v1.0 ingested 578 PII-scrubbed for SFT. The flywheel is real, not theoretical.
25,083 production traces · live and growing
⑤
Production-schema match
Itinerary outputs are structurally drop-in compatible with NxVoy's itinerary_agent_llm.py. Trained against the actual _PROMPT_CORE, _PROMPT_QUALITY_RULES sections — not generic "be helpful" prompts.
Zero adapter-layer rewrites for production deploy
⑥
Engineering rigor
v0.8 first attempt died at step 600 with degenerate output. We root-caused a Qwen3 chat-template ↔ loss-mask interaction (an SFT collapse pattern not publicly documented elsewhere), shipped a fix, validated, and published the postmortem.
Postmortem published · same discipline as a real ML lab
Roadmap

From v1.0 to a continuously-learning travel intelligence

Now · v1.0
Live with full data lake
Model + RAG corpus + training data all on HuggingFace. Beta launches in shadow mode — SHASA_ROLLOUT_MODE=shadow with real production traffic flowing.
~2 weeks · v1.1
RAG production deploy + DPO
Fix RAG cold-start memory issue on inference endpoint. Add Direct Preference Optimization on real production decision traces from v1.0 shadow traffic. RunPod migration for vLLM-optimized serving.
~2 months · v1.2
A/B → primary rollout
Progressive traffic shift: 10% → 30% → 50%. Every booking outcome trains the next version. Continuous learning loop activates with cron-scheduled retrains.
~6 months · v2.0
Larger base + RLHF
Move to Qwen3-14B or 32B base. Add Constitutional-AI-style training for travel safety. Submit to public benchmark leaderboards (Artificial Analysis, OpenRouter).
Methodology

Reproducible by anyone with $20 and an afternoon

Every claim on this page is traceable to a public file. Re-run the benchmark, re-train the model, re-build the RAG corpus — all open source.

Training

  • Base: Qwen/Qwen3-8B
  • LoRA: rank 16, alpha 32, target q/k/v/o/gate/up/down + train_unembed
  • Data: 37,838 train + 1,194 eval + 798 test
  • Hyperparameters: lr=2e-5, 1 epoch, batch=4, max_seq=2048
  • Steps: 3,800 / 9,377 (early-stopped on plateau)
  • Final train loss: 0.0614 · best eval loss: 0.0004
  • Wall time: 4h 49m on Tinker (managed distributed GPU)
  • Total spend: ~$5 USD

SFT data sources

  • Firestore destinations: 12,427 Q&A pairs from 1,659 destinations × 6-8 patterns
  • Tour-attraction listings: 1,702 top-attraction lists per destination
  • NxVoy holiday_trails: 1,500 proprietary curated multi-city editorial trails
  • Luxury hotel inventory (4-5★): 5,000 luxury hotel descriptions
  • Production decision traces: 578 PII-scrubbed SP3 itinerary traces
  • Itinerary narratives: 1,160 editorial trip articles
  • Cross-product review aggregates: 647 cross-product review aggregates
  • v0.9 concise supplement: 7,002 (intent + naming + clarification)
  • v0.8 carry-over: 10,000 (chat + baseline + formatting)

RAG corpus

  • Total: 187,339 documents
  • Embedding: sentence-transformers/all-MiniLM-L6-v2 (384-dim)
  • Index: FAISS IndexFlatIP (cosine via normalized vectors)
  • Build time: 412s on Mac MPS (~455 docs/sec)
  • Top sources: open-content see/do/eat (100K), Luxury hotel inventory (4-5★) (30K), Firestore curated (54K), NxVoy trails (1,806)
  • Files: rag_corpus.parquet (393 MB) + rag_index.faiss (287 MB)

Limitations & caveats

  • Specialist, not foundation: a LoRA on Qwen3-8B
  • RAG live deploy: corpus is published and retrieval is locally verified, but the v1.0 inference endpoint had a cold-start init issue (likely L4 memory pressure with both LoRA + FAISS + sentence-transformers loaded). Fix scheduled in v1.1 with two-stage init or larger GPU.
  • Single-checkpoint deploy: v1.0 uses one checkpoint (early-stopped best eval) — no router pattern needed
  • Long-form latency: 4K-token itinerary generation takes ~5min on default HF endpoint at 13 tok/s. vLLM serving on the same L4 should drop to ~1-2min. RunPod migration on the v1.1 roadmap.
  • Travel-only: would score badly on GPQA, HLE, IFBench — by design, that's a different category
  • Pre-beta: no production traffic yet. Numbers will refresh from real shadow-mode usage in v1.1.
  • Rule-based scoring: catches structural failures but not factual accuracy. LLM-as-judge layer on the v1.1 roadmap.