The Technology Behind Instant Documentation Search

Ever wonder how ULPI makes your AI instantly know your entire codebase? It’s not magic—it’s a carefully engineered pipeline that transforms scattered documentation into semantic search that actually works. This guide explains:

🔍 How semantic search understands meaning, not just keywords
⚡ Why ULPI is 25x more efficient than loading full docs
🔄 How updates sync in under 60 seconds
🔒 How your private docs stay secure

Not interested in technical details? Skip to Search Features for practical usage tips.Want to integrate via API? Jump to API Integration.

The Problem: Why Keyword Search Fails for Docs

Traditional search tools (GitHub search, grep, etc.) look for exact keyword matches:

Keyword Search Fails
Semantic Search Succeeds

You ask: “How do I handle database schema changes?”Keyword search thinks:

Find files containing:
- "database" AND "schema" AND "changes"

Results:

❌ Misses docs/migrations.md (doesn’t mention “schema changes”)
❌ Misses architecture/versioning.md (doesn’t say “database”)
✅ Finds README.md with phrase “database schema changes” (lucky!)

Problem: Your migrations guide uses words like “migration” and “versioning”—not “schema changes”

You ask: “How do I handle database schema changes?”Semantic search thinks:

Find documents about:
- Database migrations
- Schema versioning
- DDL changes
- Database upgrades
- Data structure evolution

Results:

✅ Finds docs/migrations.md (about database migrations)
✅ Finds architecture/versioning.md (explains schema evolution)
✅ Finds backend/README.md#migrations (even without exact keywords)

Why it works: Understands that “schema changes” = “migrations” = “versioning”

This is why ULPI uses semantic search powered by vector embeddings.

High-Level: How It Works in 30 Seconds

You Connect Repositories (2 minutes)

One-click OAuth connection to GitHub, GitLab, Bitbucket, or GiteaULPI automatically discovers all your documentation:

README files
docs/ directories
Wikis
Markdown files everywhere

AI Indexes Your Docs (2-5 minutes)

Automatic processing in the background:

Parse every documentation file
Break into logical sections (chunking)
Convert to vector embeddings (AI representation)
Store in lightning-fast search engine (Typesense)

You don’t do anything. It just works.

Your AI Asks Questions (instant)

When your AI assistant needs docs:

AI: "I need docs about authentication"
↓
ULPI: Searches all repos semantically (40ms)
↓
ULPI: Returns relevant sections (2,000 tokens)
↓
AI: "Based on your docs/auth.md, here's how..."

Total time: Under 50 milliseconds

Auto-Sync on Every Push (60 seconds)

You push to main:

git push origin main

ULPI automatically:

Receives webhook notification (1 second)
Re-indexes changed files (30-60 seconds)
AI now sees updated documentation

No manual sync button. Always up-to-date.

The magic: AI understands meaning, not just keywords. That’s why it finds the right docs even when you use different terminology.

Deep Dive: The Indexing Pipeline

For developers who want to understand the technical implementation.

Architecture Diagram

Step-by-Step: What Happens During Indexing

Step 1: Repository Connection & Discovery

When you connect a repository:

OAuth Authentication

Read-only access via GitHub/GitLab OAuthPermissions requested:

✅ Read repository contents
✅ Register webhooks for auto-sync
❌ No write access (we never modify your code)

File Discovery

ULPI scans your repository structure:

your-repo/
├── README.md                    ✅ Indexed
├── docs/
│   ├── setup.md                 ✅ Indexed
│   ├── api-reference.md         ✅ Indexed
│   └── architecture.md          ✅ Indexed
├── .github/
│   ├── CONTRIBUTING.md          ✅ Indexed
│   └── workflows/deploy.yml     ❌ Not docs
├── src/
│   ├── UserService.php          ❌ Code (not indexed by default)
│   └── README.md                ✅ Indexed (README)
├── node_modules/                ❌ Excluded
└── vendor/                      ❌ Excluded

Indexed file types:

.md, .mdx (Markdown)
.txt (plain text in doc directories)
README.* (any extension, any directory)
Wiki pages (optional)

Custom exclusions: Add .ulpiignore file

Metadata Extraction

For each discovered file, ULPI extracts:

Path: docs/authentication.md
Branch: main
Last modified: Git commit timestamp
Author: Git commit author
File size: For processing estimates

Step 2: Content Parsing & Chunking

Breaking documents into searchable chunks

Why Chunking?

Problem: A 10,000-line architecture doc contains dozens of distinct topics.Without chunking:

Search returns entire 10,000-line document
AI must process all 10,000 lines to find relevant section
Wastes 8,000 tokens on irrelevant content

With chunking:

Search returns only the relevant 200-line section about your query
AI gets precise context
Saves 97% of tokens

How We Chunk

Smart chunking strategy (not just splitting every N characters):

Respect document structure:
- Preserve headings and sections
- Keep code blocks together
- Don’t split tables or lists
Optimal chunk size: ~512 tokens
- Large enough for context
- Small enough for precision
- Equivalent to 2-3 paragraphs
Add overlap:
- 50-token overlap between chunks
- Prevents losing context at chunk boundaries

Example:

## Authentication

Our API uses OAuth 2.0 for authentication...
[500 tokens about OAuth]

─────── CHUNK BOUNDARY with 50-token overlap ───────

## Authorization

After authentication, users are assigned roles...
[500 tokens about authorization]

Result: When searching for “authentication”, you get the OAuth section—not the entire auth guide.

Step 3: Vector Embedding Generation

Converting text to AI-understandable format

What Are Embeddings?

Simple explanation: Embeddings convert text into numbers that capture meaning.Example:

Text: "How to authenticate users with OAuth"
↓
Embedding: [0.023, -0.891, 0.456, 0.123, ... 1,536 numbers total]

Why numbers? Computers can compare numbers to find similar meanings:

"authenticate users"     → [0.23, -0.89, 0.45, ...]
"login system"           → [0.21, -0.87, 0.44, ...]  ← Very similar!
"database migrations"    → [0.87, 0.23, -0.12, ...] ← Very different

The Process

Send to OpenAI

Each chunk is sent to OpenAI’s embedding API:Model: text-embedding-3-largeDimensions: 1,536 (captures nuanced meaning)

const embedding = await openai.embeddings.create({
  model: "text-embedding-3-large",
  input: "Our API uses OAuth 2.0 for authentication...",
});

// Returns: [0.023, -0.891, 0.456, ..., 1536 numbers]

Store in Vector Database

Embeddings are stored in Typesense:

{
  "id": "doc_123_chunk_4",
  "content": "Our API uses OAuth 2.0 for authentication...",
  "embedding": [0.023, -0.891, 0.456, ...],
  "metadata": {
    "file": "docs/authentication.md",
    "repository": "backend-api",
    "branch": "main",
    "url": "github.com/org/repo/blob/main/docs/auth.md#L42"
  }
}

Privacy Note

Your documentation is NEVER used to train AI models.

Embeddings are mathematical representations, not readable text
OpenAI’s zero data retention policy (embeddings API)
We only use embeddings for search indexing
Original text stays in your control

Step 4: Indexing in Typesense

Storing embeddings for lightning-fast search

Why Typesense?

Typesense is an open-source vector search engine optimized for:

⚡ Speed: Sub-50ms vector similarity search
🎯 Accuracy: Hybrid semantic + keyword ranking
📈 Scalability: Handles millions of documents
🔧 Simplicity: No complex configuration

Index Structure

{
  "name": "documentation",
  "fields": [
    { "name": "content", "type": "string" },
    { "name": "embedding", "type": "float[]", "num_dim": 1536 },
    { "name": "file_path", "type": "string", "facet": true },
    { "name": "repository", "type": "string", "facet": true },
    { "name": "branch", "type": "string", "facet": true },
    { "name": "last_modified", "type": "int64", "sort": true }
  ]
}

Facets: Enable filtering by repository, file type, branch, etc. Sorting: Prioritize recent documentation

Step 5: Webhook Registration

Auto-sync on every git push

How Webhooks Work

When you connect a repository, ULPI registers a webhook:GitHub webhook payload:

{
  "ref": "refs/heads/main",
  "commits": [
    {
      "modified": ["docs/authentication.md"],
      "added": ["docs/new-feature.md"],
      "removed": ["docs/deprecated.md"]
    }
  ]
}

ULPI receives this and:

Queues a re-indexing job
Fetches only changed files
Re-generates embeddings for changes
Updates Typesense index
Invalidates cache

Time: 10-60 seconds from push to searchable

Smart Re-indexing

Full re-index (slow):

New documentation directory created
Branch created/deleted
Manual trigger from dashboard

Partial re-index (fast):

Existing file modified
Only re-process changed chunks
10-30 seconds

No re-index:

Code files changed (.js, .php, etc.)
Excluded directories (node_modules/, vendor/)
Files in .ulpiignore

How Semantic Search Works

When your AI assistant queries ULPI:

The Search Process

Query Embedding

Your AI asks: “How do I deploy to production?”ULPI converts to embedding:

const queryEmbedding = await openai.embeddings.create({
  model: "text-embedding-3-large",
  input: "How do I deploy to production?"
});

// Result: [0.123, -0.456, 0.789, ..., 1536 dimensions]

Same model as document embeddings → comparable vectors

Vector Similarity Search

Find similar documentation chunks:Algorithm: Cosine similarity

similarity = cos(query_vector, doc_vector)

Range: 0.0 (completely different) to 1.0 (identical)

Typesense finds:

[
  {
    "content": "Deploy to production using...",
    "similarity": 0.94,
    "file": "infrastructure/deployment.md"
  },
  {
    "content": "Our deployment process involves...",
    "similarity": 0.89,
    "file": "docs/ci-cd.md"
  },
  {
    "content": "Production environment configuration...",
    "similarity": 0.85,
    "file": "README.md"
  }
]

Threshold: Returns matches with similarity > 0.75

Hybrid Scoring

Combine semantic + keyword matching:

final_score = (0.7 * semantic_score) + (0.3 * keyword_score)

Why hybrid?

Semantic Only (Problem)
Keyword Only (Problem)
Hybrid (Best)

Query: “Redis configuration”Semantic search finds:

✅ “Caching setup” (semantically similar)
✅ “Session storage” (related concept)
❌ Misses exact “redis.conf” reference

Issue: May miss exact keyword matches

Ranking & Filtering

Results are ranked by:

Relevance score (hybrid score)
Document recency (newer docs ranked higher)
Repository priority (if you specified repos)
File type:
- README.md (most important)
- docs/*.md (documentation)
- Other files (lower priority)

Filters applied:

Repository scope (if API key is scoped)
Branch (default: main)
File type (if specified)
Date range (if specified)

Context Assembly

Return top results with metadata:

{
  "results": [
    {
      "content": "Deploy to production using GitHub Actions...",
      "file": "infrastructure/deployment.md",
      "repository": "backend-api",
      "branch": "main",
      "url": "https://github.com/org/backend-api/blob/main/infrastructure/deployment.md#L42-L67",
      "score": 0.94,
      "last_modified": "2025-11-10T14:23:00Z"
    }
  ],
  "total": 12,
  "tokens_used": 2340
}

AI receives this and synthesizes answer:

"Based on your deployment guide in infrastructure/deployment.md:

To deploy to production:
1. Run `npm run build`
2. Push to deploy branch: `git push origin main:deploy`
3. GitHub Actions automatically deploys to AWS

[Link to deployment.md:42-67]"

Why This Is Fast

Sub-50ms average latency:

Optimized Vector Search

Typesense pre-computes indexes for instant similarity searchNo full-text scanning required

Three-Level Caching

Browser cache (5 min)
MCP server cache (1 hour)
API cache (5 min)

Repeated queries: 5ms

Smart Chunking

Returns only relevant sections, not entire files2,000 tokens vs 50,000 tokens

Dedicated Infrastructure

Separate Typesense cluster for searchNo database bottlenecks

Real-Time Updates via Webhooks

How documentation stays synchronized automatically

Webhook Flow

Processing Timeline

What happens after you push:

Instant: Webhook Received

< 1 secondGitHub/GitLab sends webhook to ULPI:

POST /webhooks/github
{
  "repository": "your-org/backend-api",
  "ref": "refs/heads/main",
  "commits": [
    {
      "modified": ["docs/authentication.md"],
      "added": ["docs/new-api.md"]
    }
  ]
}

ULPI responds 200 OK immediately (non-blocking)

5-10 seconds: Job Queued

Background processing starts:

Job added to Redis queue
Laravel Horizon assigns worker
Worker fetches changed files from GitHub

20-40 seconds: Re-indexing

For each changed file:

Parse Markdown content
Chunk into sections
Generate embeddings (OpenAI API call)
Update Typesense index
Invalidate cached searches

30-60 seconds: Searchable

Documentation is now searchable:

✅ AI assistants see updated docs
✅ New sections discoverable
✅ Deleted sections removed
✅ Cache cleared

Total time: 30-60 seconds from push to searchable

Large pushes (100+ files): May take 2-5 minutes. Check indexing status in dashboard.

MCP Integration Architecture

How AI assistants access your documentation

MCP Protocol Overview

MCP (Model Context Protocol) is a standard for connecting AI tools to external data sources. ULPI provides an MCP server that bridges AI assistants to the ULPI API:

┌─────────────────┐
│  Your AI        │  (Claude Desktop, Cursor, etc.)
│  Assistant      │
└────────┬────────┘
         │
         │ MCP Protocol
         │
┌────────▼────────┐
│  ULPI MCP       │  (@ulpi/mcp-server-documentation)
│  Server         │  - Runs locally in your IDE
│                 │  - Authenticates with API key
└────────┬────────┘
         │
         │ HTTPS
         │
┌────────▼────────┐
│  ULPI API       │  (api.ulpi.io)
│  - Search       │  - Hosted in AWS
│  - Indexing     │  - Handles all teams
└─────────────────┘

MCP Server Implementation

The MCP server provides tools to your AI:

// @ulpi/mcp-server-documentation

const tools = [
  {
    name: "search_documentation",
    description: "Search all connected repositories for documentation",
    parameters: {
      query: "Natural language search query",
      repository: "Optional: Filter to specific repo",
      limit: "Max results to return (default: 5)"
    }
  }
];

// When AI calls the tool:
async function search_documentation({ query, repository, limit = 5 }) {
  // 1. Authenticate with API key from environment
  const apiKey = process.env.ULPI_API_KEY;

  // 2. Call ULPI API
  const response = await fetch('https://api.ulpi.io/api/v1/documentation/search', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${apiKey}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({ query, repository, limit })
  });

  // 3. Return results to AI
  return await response.json();
}

AI Tool Invocation Example

How Claude Desktop uses ULPI MCP:

User: "How do we handle authentication in the API?"

Claude (thinking):
  I need to search the user's documentation for authentication info.
  I'll use the search_documentation tool.

Claude → MCP Server:
  search_documentation({
    query: "API authentication implementation",
    limit: 5
  })

MCP Server → ULPI API:
  POST /api/v1/documentation/search
  {
    "query": "API authentication implementation",
    "limit": 5
  }

ULPI API → MCP Server:
  {
    "results": [
      {
        "content": "Our API uses Laravel Sanctum for authentication...",
        "file": "backend-api/docs/auth.md",
        "url": "github.com/org/backend/blob/main/docs/auth.md#L15"
      }
    ]
  }

MCP Server → Claude:
  [Returns search results]

Claude → User:
  "Based on your documentation in backend-api/docs/auth.md, your API uses
   Laravel Sanctum for authentication. Here's how it works:

   1. Client sends email + password to /api/login
   2. Server validates credentials
   3. Sanctum generates API token
   4. Client includes token in Authorization header

   [Link to auth.md:15-42]"

Seamless. User doesn’t see any of this—just accurate answers from their docs.

Caching Strategy

Three-level cache for optimal performance

Level 1: Browser/IDE Cache
Level 2: MCP Server Process Cache
Level 3: API Cache (Redis)

Location: User’s browser or IDEDuration: 5 minutesPurpose: Instant results for repeated queries in same sessionHow it works:

// MCP server caches responses locally
const cache = new Map();

async function search(query) {
  const cacheKey = `search:${query}`;

  // Check cache first
  if (cache.has(cacheKey)) {
    const cached = cache.get(cacheKey);
    if (Date.now() - cached.timestamp < 5 * 60 * 1000) {
      return cached.results; // 5ms response
    }
  }

  // Cache miss - call API
  const results = await callULPIAPI(query);
  cache.set(cacheKey, { results, timestamp: Date.now() });
  return results; // 200ms response
}

Benefit:

First query: 200ms (API call)
Repeat query: 5ms (local cache)

Invalidation: Automatic after 5 minutes

Location: ULPI API servers (Redis)Duration: 5 minutesPurpose: Fast responses for popular queries across all usersHow it works:

// Laravel API endpoint
public function search(Request $request) {
    $cacheKey = "search:{$request->query}:{$request->repository}";

    return Cache::remember($cacheKey, 300, function () use ($request) {
        // Perform vector search in Typesense
        return $this->typesenseSearch($request->query);
    });
}

Invalidation:

After 5 minutes
When repository is re-indexed (webhook)
Manual cache clear from dashboard

Benefit:

Multiple team members get fast responses
Reduces Typesense load

Cache Invalidation on Push

When you push changes:

git push
↓
Webhook fires
↓
Re-indexing job queued
↓
Files re-indexed
↓
Cache invalidated:
  - API cache (Redis): FLUSH search:*:backend-api
  - MCP server cache: Not invalidated (will expire naturally)
  - Browser cache: Not invalidated (will expire naturally)

Result: Within 60 seconds, searches return updated documentation

Security & Privacy

How ULPI protects your private documentation

🔒 Repository Access: Read-Only

What ULPI can do:

✅ Read repository contents
✅ Receive webhook notifications
✅ Clone repositories (temporarily, in-memory)

What ULPI CANNOT do:

❌ Write files
❌ Create commits
❌ Push changes
❌ Delete branches
❌ Modify repository settings

OAuth tokens:

Stored encrypted (AES-256) in MySQL
Never logged or exposed
Refreshed automatically before expiration
Revocable anytime from GitHub/GitLab settings

Revoke access:

GitHub → Settings → Applications → ULPI Documentation → Revoke

Access removed instantly.

🔑 API Keys: Tenant-Scoped

Security model:

Each API key is scoped to your team only
Cannot access other customers’ documentation
Cannot be used for other ULPI products (unless explicitly granted)

Storage:

Hashed using bcrypt (cost factor: 12)
Never stored in plaintext
Never logged
Transmitted only over HTTPS (TLS 1.3)

Key format:

ulpi_live_sk_1a2b3c4d5e6f7g8h9i0j...
^^^^ ^^^^ ^^
|    |    |
|    |    Random 32 characters
|    Environment (live/test)
Prefix

Rotation:

Create new key
Update MCP config
Revoke old key
Zero downtime

Rate limiting:

1,000 requests/hour per key
Prevents abuse if key is compromised

📄 Documentation Content: Private

What gets indexed:

Documentation files only (.md, .mdx, README.*)
NOT source code (unless you explicitly enable comment indexing)

AI training policy:

Your documentation is NEVER used to train AI models.

Not OpenAI’s models
Not Anthropic’s models
Not any third-party models

What happens:

OpenAI generates embeddings (mathematical representations)
OpenAI immediately discards the text (zero data retention policy)
Only embeddings are stored (not human-readable)
Used exclusively for your search

Data retention:

Indexed as long as repository is connected
Deleted within 30 days after repository disconnection
Deleted immediately upon account cancellation

🛡️ Compliance: SOC 2 Type II

Certifications:

✅ SOC 2 Type II compliant
✅ GDPR compliant
✅ CCPA compliant
✅ HIPAA available (Enterprise plan)

Data residency:

US region: AWS us-east-1 (default)
EU region: AWS eu-west-1 (Enterprise only)
Custom region: Available for Enterprise

Encryption:

In transit: TLS 1.3
At rest: AES-256 (database, backups, S3)

Access controls:

2FA required for ULPI employees
Zero standing access to production
Time-limited break-glass access (logged)
No access to customer data without explicit approval

Auditing:

All access logged to immutable S3 bucket
Quarterly security reviews
Annual penetration testing

Full security details

🌐 Network Security

Infrastructure:

Hosted on AWS in VPC
Private subnets (no public IPs for databases)
Security groups (least privilege)
WAF (Web Application Firewall) enabled

DDoS protection:

CloudFront CDN (global edge caching)
AWS Shield Standard
Rate limiting at API gateway

Monitoring:

Sentry for error tracking
CloudWatch for infrastructure
Alerts for anomalous traffic

Performance & Scalability

Response Time Metrics

Real-world performance data from production:

Operation	Average	95th Percentile	99th Percentile
Simple search (1-2 keywords)	45ms	120ms	250ms
Complex search (full sentence)	120ms	300ms	600ms
Re-index (small repo) (under 100 files)	30s	60s	90s
Re-index (medium repo) (100-1k files)	2min	5min	8min
Re-index (large repo) (1k-10k files)	5min	12min	20min
Webhook processing	5s	15s	30s

Why so fast?

Vector Search

Typesense pre-computes indexesNo linear scanning

Redis Caching

Popular queries cached for 5 minutesSub-10ms for cached queries

Chunking Strategy

Returns 2,000 tokens, not 50,00025x fewer tokens processed

Dedicated Hardware

Separate Typesense clusterNo database contention

Scalability Limits

What ULPI can handle:

Repositories per tenant: Unlimited (Enterprise)
Files per repository: Unlimited
Documentation file size: Up to 10MB per file
Concurrent searches: 100/second per tenant
Total index size: Average 1MB per 1,000 documentation pages

Enterprise customers:

Dedicated Typesense cluster (isolated resources)
Custom rate limits
SLA: 99.9% uptime
Priority support

Technology Stack

What powers ULPI Documentation

Backend API
Search & AI
Infrastructure
MCP Integration

Framework: Laravel 12.x (PHP 8.2)Why Laravel?

Robust queue system (Horizon)
Excellent webhook handling
Enterprise-ready
Fast development

Database: MySQL 8.0

Stores metadata, API keys, user accounts
NOT used for search (that’s Typesense)

Queue: Redis + Laravel Horizon

Background job processing
Re-indexing jobs
Webhook processing

Cache: Redis

Search result caching (5 minutes)
Session storage
Rate limiting

Embeddings: OpenAI text-embedding-3-large

1,536 dimensions
Best-in-class semantic understanding
Zero data retention policy

Vector Search: Typesense

Open-source vector database
Sub-50ms similarity search
Hybrid semantic + keyword
Faceted filtering

Similarity Algorithm: Cosine similarity

Range: 0.0 (different) to 1.0 (identical)
Threshold: 0.75 for results

MCP Server: TypeScript (Node.js)Package: @ulpi/mcp-server-documentationWhy TypeScript?

Type safety for API contracts
Excellent IDE support
Easy to maintain

Distribution: npm (public package)Install:

npx -y @ulpi/mcp-server-documentation

No installation required - npx downloads and runs on-demand

Comparison: ULPI vs Alternatives

Why semantic search beats traditional tools

Feature	ULPI	GitHub Search	grep/ripgrep	RAG (DIY)
Search type	Semantic + keyword	Keyword only	Keyword only	Semantic (if configured)
Natural language	✅ Yes	❌ No	❌ No	✅ Yes (requires setup)
Cross-repo search	✅ All repos at once	❌ One at a time	❌ Local only	🟡 Depends on implementation
AI integration	✅ 40+ tools via MCP	❌ No	❌ No	🟡 Custom integration required
Auto-sync	✅ Webhook-based	✅ Real-time	❌ Manual	🟡 Must build yourself
Setup time	⚡ 5 minutes	🟢 0 (built-in)	🟢 0 (built-in)	🔴 Days/weeks
Token efficiency	⚡ 2,000 tokens	🔴 50,000 tokens	🔴 Full file	🟡 Depends
Latency	⚡ Sub-50ms	🟡 100-500ms	🟢 Instant (local)	🔴 Varies
Cost	💰 $29-299/mo	🟢 Free	🟢 Free	🔴 $$$$ (infrastructure + dev time)

When to use each:

ULPI: AI assistants need semantic understanding across repos
GitHub Search: Finding specific code patterns or file names
grep/ripgrep: Local file searching, exact keywords
DIY RAG: Have ML team, custom requirements, budget >$50k

Limitations & Edge Cases

What ULPI doesn’t do (yet)

Known limitations:

Code search:
- ULPI is optimized for documentation, not code
- Code comments can be indexed (opt-in)
- Use GitHub/grep for searching actual code
Binary files:
- PDFs, Word docs, images not indexed
- Convert to Markdown for indexing
Very large files:
- Files >10MB are skipped
- Break large docs into smaller files
Real-time (sub-second sync):
- Webhook processing takes 30-60 seconds
- Not suitable for real-time wikis
Private git servers:
- Self-hosted GitHub Enterprise: ✅ Supported
- GitLab self-hosted: ✅ Supported
- Other git servers: 🟡 Contact us
Non-English documentation:
- Works, but embeddings optimized for English
- Other languages: slightly lower accuracy

FAQ: Technical Questions

Can I use a different embedding model?

Not currently. ULPI uses OpenAI text-embedding-3-large for all embeddings.Why?

Best-in-class accuracy
1,536 dimensions (high-fidelity)
Proven at scale

Enterprise custom models:

Contact us for enterprise plans
We can discuss alternative models
Requires separate deployment

How much does re-indexing cost?

Included in your plan. No per-search or per-embedding fees.What you pay:

Monthly subscription ($29-299)
Token usage for searches (included in plan)

What’s free:

Re-indexing (unlimited)
Webhook processing
Embedding generation
Storage

Overage:

$20 per 100,000 additional tokens (search queries only)

Can I self-host ULPI?

Not yet, but coming soon.Current options:

Cloud: Hosted by ULPI (default)
VPC peering: Connect to your VPC (Enterprise)
On-premise: Planned for Q2 2025

Interested in self-hosting?

Contact sales@ulpi.io
Minimum: Enterprise plan

What happens if OpenAI's API goes down?

Search continues to work. Only new indexing is affected.How?

Existing embeddings already in Typesense
Search uses those embeddings (no OpenAI API call)
Only embedding generation requires OpenAI

If OpenAI is down:

✅ Search works normally
❌ New files can’t be indexed
❌ Updated files can’t be re-indexed

Mitigation:

Jobs automatically retry (exponential backoff)
Usually resolves in under 15 minutes

Next Steps

Try It: Getting Started

Set up ULPI in 5 minutes and see semantic search in actionNo credit card required for trial

Advanced Search Features

Learn filters, repository scoping, and query optimizationMaster semantic search

API Integration

Integrate ULPI directly into your applications via REST APIBuild custom workflows

Repository Management

Manage connected repos, configure indexing, view metricsOptimize indexing

Still have questions?

📧 Email: support@ulpi.io
📚 Docs: docs.ulpi.io
💬 Slack: ulpi.io/slack

Average response time: Under 2 hours during business hours

​The Technology Behind Instant Documentation Search

​The Problem: Why Keyword Search Fails for Docs

​High-Level: How It Works in 30 Seconds

​Deep Dive: The Indexing Pipeline

​Architecture Diagram

​Step-by-Step: What Happens During Indexing

​Why Chunking?

​How We Chunk

​What Are Embeddings?

​The Process

​Privacy Note

​Why Typesense?

​Index Structure

​How Webhooks Work

​Smart Re-indexing

​How Semantic Search Works

​The Search Process

​Why This Is Fast

Optimized Vector Search

Three-Level Caching

Smart Chunking

Dedicated Infrastructure

​Real-Time Updates via Webhooks

​Webhook Flow

​Processing Timeline

​MCP Integration Architecture

​MCP Protocol Overview

​MCP Server Implementation

​AI Tool Invocation Example

​Caching Strategy

​Cache Invalidation on Push

​Security & Privacy

​Performance & Scalability

​Response Time Metrics

Vector Search

Redis Caching

Chunking Strategy

Dedicated Hardware

​Scalability Limits

​Technology Stack

​Comparison: ULPI vs Alternatives

​Limitations & Edge Cases

​FAQ: Technical Questions

​Next Steps

Try It: Getting Started

Advanced Search Features

API Integration

Repository Management

The Technology Behind Instant Documentation Search

The Problem: Why Keyword Search Fails for Docs

High-Level: How It Works in 30 Seconds

Deep Dive: The Indexing Pipeline

Architecture Diagram

Step-by-Step: What Happens During Indexing

Why Chunking?

How We Chunk

What Are Embeddings?

The Process

Privacy Note

Why Typesense?

Index Structure

How Webhooks Work

Smart Re-indexing

How Semantic Search Works

The Search Process

Why This Is Fast

Real-Time Updates via Webhooks

Webhook Flow

Processing Timeline

MCP Integration Architecture

MCP Protocol Overview

MCP Server Implementation

AI Tool Invocation Example

Caching Strategy

Cache Invalidation on Push

Security & Privacy

Performance & Scalability

Response Time Metrics

Scalability Limits

Technology Stack

Comparison: ULPI vs Alternatives

Limitations & Edge Cases

FAQ: Technical Questions

Next Steps