Semantic Waypoints

Automatic connections that expand search context

Semantic Waypoints

The magic behind comprehensive search results. Waypoints are automatic connections between related memories that give you complete context, not just exact matches.

Search for one thing, get everything related automatically. No manual tagging required.

---

What Are Waypoints?

Waypoints = automatic connections between semantically similar memories

Think of memories as cities, waypoints as roads:

• Search finds the city you asked for (exact matches)
• Waypoints travel the roads to nearby cities (related context)

Example:

Search: "React state management"

Direct match:
→ "React hooks (useState, useEffect)"

Waypoints (automatic connections):
→ "Context API for global state" (similarity: 0.82)
→ "Redux store patterns" (similarity: 0.78)
→ "When to lift state up" (similarity: 0.81)

Result: 4 memories instead of 1—complete context automatically

Benefit: Search feels like asking a colleague who knows the whole story, not just keywords.

---

How Waypoints Work

1. Store Memory

   You save: “JWT authentication implementation”

   System generates vector embedding (semantic meaning as numbers)

2. Find Similar Memories

   Compares with existing memories using cosine similarity:

   • 1.0 = Identical
   • 0.75-0.99 = Highly similar
   • 0.5-0.74 = Moderately similar
   • Less than 0.5 = Weakly related
3. Create Connections

   If similarity ≥ 0.75, creates bidirectional waypoint

   “JWT auth” ↔ “OAuth2 flow” (0.82) “JWT auth” ↔ “Session management” (0.76) “JWT auth” ↔ “Token refresh” (0.89)

   Waypoint stores similarity weight for ranking

4. Search Traverses

   When you search, results include:

   • Direct matches (hop 0)
   • 1 connection away (hop 1, -20% salience)
   • 2 connections away (hop 2, -40% salience)
   • 3 connections away (hop 3, -60% salience)

Duration: Waypoints created within 5-10 seconds after storing memory (async)

---

Search Expansion

How search uses waypoints:

Without Waypoints

Search: "microservices"

Results (1 memory):
→ "Why we chose microservices"

You get: Exact match only
Missing: Service discovery, API gateways, deployment patterns

Problem: Incomplete context

With Waypoints

Search: "microservices"

Hop 0 (direct matches):
→ "Why we chose microservices" (score: 0.95)
→ "Microservices vs monolith" (score: 0.88)

Hop 1 (1 connection away):
→ "Service discovery with Consul" (0.82)
→ "API gateway patterns" (0.79)
→ "Database per service" (0.76)

Hop 2 (2 connections away):
→ "Event-driven architecture" (0.78)
→ "Distributed tracing" (0.81)

Hop 3 (3 connections away):
→ "Kubernetes deployment" (0.77)

Results: 8 memories with complete context

Benefit: Comprehensive answer automatically

---

Similarity Threshold: 0.75

Why 0.75 is the sweet spot:

Too Low (<0.75)

Problem: Noise

“React hooks” connects to “Angular services”

Both about state, but different frameworks

Result: Polluted results

Just Right (0.75)

Perfect balance

“React hooks” connects to:

• Context API ✓
• Redux ✓
• State lifting ✓

All genuinely related

Result: Relevant context

Too High (>0.85)

Problem: Too narrow**

“React hooks” only connects to near-duplicates

Misses useful related concepts

Result: Incomplete context

---

Salience Decay

Distant connections rank lower than close ones:

Formula: adjusted_salience = original_salience × (0.8 ^ hop_distance)

Visual

Direct match (hop 0): 1.00 ★★★★★
1 connection away:    0.80 ★★★★☆
2 connections away:   0.64 ★★★☆☆
3 connections away:   0.51 ★★☆☆☆

Why: Direct matches most relevant, distant connections provide context but rank lower

Example

Original salience: 1.0

Distance	Adjusted Salience	Rank
Hop 0	1.00	Highest
Hop 1	0.80	High
Hop 2	0.64	Medium
Hop 3	0.51	Lower

Original salience: 0.5

Distance	Adjusted Salience	Rank
Hop 0	0.50	Medium
Hop 1	0.40	Lower
Hop 2	0.32	Low
Hop 3	0.26	Lowest

Result: Best matches float to top, related context follows

---

Real-World Example

Scenario: Searching for auth implementation details

Search: "authentication implementation"

Hop 0 (Direct):
├─ "JWT authentication flow" (salience: 1.0)
└─ "Auth middleware setup" (salience: 0.95)

Hop 1 (Connected via waypoints):
├─ "OAuth2 integration" (via JWT, 0.82 → adjusted: 0.76)
├─ "Session management" (via JWT, 0.79 → adjusted: 0.71)
└─ "Token refresh strategy" (via JWT, 0.89 → adjusted: 0.76)

Hop 2 (2nd degree connections):
├─ "User permissions system" (via OAuth2, 0.78 → adjusted: 0.61)
└─ "Security best practices" (via Session, 0.81 → adjusted: 0.64)

Hop 3 (3rd degree):
└─ "HTTPS configuration" (via Security, 0.77 → adjusted: 0.46)

You searched for auth implementation.

You got:

• Exact auth docs ✓
• Related OAuth details ✓
• Session management ✓
• Security best practices ✓
• Even HTTPS config ✓

Complete context automatically assembled.

---

Control Waypoint Expansion

API parameter: expand_waypoints (default: true)

Enabled (Default)

search_memories({
  query: "microservices",
  expand_waypoints: true,  // Default
  max_hops: 3
})

Returns: Direct matches + 3 hops of connected memories

Use when: You want comprehensive context (most searches)

Disabled

search_memories({
  query: "microservices",
  expand_waypoints: false
})

Returns: Only direct matches, no waypoint traversal

Use when:

• You want exact matches only
• Performance critical (faster)
• Preventing “context explosion”

Custom Hops

search_memories({
  query: "microservices",
  expand_waypoints: true,
  max_hops: 1  // Only 1 connection away
})

Returns: Direct matches + 1 hop

Use when: Want some context but not too much

---

Best Practices

Use Expansion for Research

Scenario: Understanding a topic comprehensively

search_memories({
  query: "authentication patterns",
  expand_waypoints: true,
  max_hops: 3
})

Result: Complete picture of auth in your codebase

Disable for Exact Lookups

Scenario: Finding specific file or command

search_memories({
  query: "deploy command production",
  expand_waypoints: false
})

Result: Just the deploy command, no related deploy docs

Store Rich Context

Why: Better waypoints form automatically

✅ Good:

"JWT auth using RS256, tokens expire 1hr,
refresh tokens 7 days, stored in Redis"

❌ Bad:

"JWT auth"

Rich content = better semantic connections

Let It Auto-Connect

Don’t manually tag or link

System automatically finds:

• Related implementations
• Similar problems
• Connected decisions

Just store memories, waypoints handle the rest

---

Why Waypoints Beat Manual Tags

Feature	Manual Tags	Waypoints
Setup	You tag everything	Automatic
Consistency	”auth”, “authentication”, “login” → chaos	Semantic similarity handles variants
Maintenance	Update tags when content changes	Re-indexes automatically
Discovery	Only find what you tagged	Finds related content you didn’t predict
Effort	High (tag every memory)	Zero (happens automatically)
Accuracy	Depends on your tagging discipline	Mathematical similarity (objective)

---

Technical Details

Vector Embeddings

Typesense (default):

• 384-dim vectors
• Fast, efficient
• Good for most use cases

OpenAI (optional):

• 1536-dim (text-embedding-3-small)
• 3072-dim (text-embedding-3-large)
• More nuanced semantic understanding
• Higher cost

Both use cosine similarity for comparison

Cosine Similarity Formula

similarity = (A · B) / (||A|| × ||B||)

Where:
- A, B are vector embeddings
- · is dot product
- ||A|| is vector magnitude

Result: 0.0 to 1.0 (higher = more similar)

0.75 threshold = strong semantic relationship

Waypoint Storage

Database schema:

waypoints (
  id: uuid,
  from_memory_id: uuid,
  to_memory_id: uuid,
  similarity: float (0.75-1.0),
  created_at: timestamp
)

Index: (from_memory_id) for fast traversal
Index: (similarity DESC) for ranking

Bidirectional: If A→B exists, B→A also exists

BFS Traversal

Breadth-first search (not depth-first)

Why BFS:

• Explores many related topics (breadth)
• Before diving deep into subtopics (depth)
• Ensures diverse context

Algorithm:

1. Start with direct matches (queue)
2. For each memory, get waypoint connections
3. Add to queue if not visited and within max_hops
4. Apply salience decay
5. Continue until queue empty or max_hops reached

---

Next Steps

Search & Retrieval

Master search techniques

Best Practices

Optimize memory usage

Workflows

Put waypoints to work

---

Waypoints turn simple searches into comprehensive answers. Store memories, let the connections form automatically.