Integration Examples

This page shows how to integrate VCAL Semantic Cache into an LLM application, RAG flow, or agentic workflow.

VCAL works as a lightweight HTTP semantic cache. Your application checks VCAL before calling the LLM. If a similar question, answer, or agent step was handled before, VCAL can return the cached answer immediately. If there is no suitable match, your application calls the LLM as usual and stores the new answer in VCAL for future reuse.

VCAL is model-agnostic. Your application can use any embedding provider or model, as long as the vector dimensions match the VCAL_DIMS setting used by the server.

---

Minimal Python example

This example uses:

• VCAL Semantic Cache running locally on http://127.0.0.1:8080
• Ollama for local embeddings
• nomic-embed-text as the embedding model
• Python requests
• a placeholder LLM fallback function

The flow is:

1. Create an embedding vector for the user question.
2. Ask VCAL Semantic Cache /v1/qa for a matching cached answer.
3. If VCAL returns a hit, use the cached answer.
4. If VCAL returns a miss, call the LLM and store the answer with /v1/upsert.

---

Prerequisites

Start VCAL Semantic Cache first. See the Installation guide if needed.

Install Python dependencies:

pip install requests

Make sure Ollama is running and the embedding model is available:

ollama pull nomic-embed-text

The example assumes the embedding dimension matches the VCAL Semantic Cache server configuration. For nomic-embed-text, use:

VCAL_DIMS=768

If your VCAL Semantic Cache server requires authentication, set:

export VCAL_API_KEY="<your_app_key>"

---

Example script

#!/usr/bin/env python3

import hashlib
import os

import requests


VCAL_BASE = os.getenv("VCAL_URL", "http://127.0.0.1:8080").rstrip("/")
VCAL_KEY = os.getenv("VCAL_API_KEY", "")

OLLAMA_URL = os.getenv("OLLAMA_URL", "http://127.0.0.1:11434").rstrip("/")
EMBED_MODEL = os.getenv("EMBED_MODEL", "nomic-embed-text")

SIM_THRESHOLD = float(os.getenv("VCAL_SIM_THR", "0.85"))
EF_SEARCH = int(os.getenv("VCAL_EF_SEARCH", "128"))

HEADERS = {"Content-Type": "application/json"}
if VCAL_KEY:
    HEADERS["X-VCAL-Key"] = VCAL_KEY


def embed_text(text: str) -> list[float]:
    """Create an embedding vector using Ollama."""
    r = requests.post(
        f"{OLLAMA_URL}/api/embeddings",
        json={"model": EMBED_MODEL, "prompt": text},
        timeout=30,
    )
    r.raise_for_status()

    vec = r.json().get("embedding")
    if not isinstance(vec, list) or not vec:
        raise RuntimeError("Bad embedding response from Ollama")

    return vec


def ext_id_for(question: str) -> int:
    """Create a stable numeric external ID for the question."""
    normalized = question.strip().lower().encode("utf-8")
    h = hashlib.blake2b(normalized, digest_size=8).digest()
    return int.from_bytes(h, "big", signed=False)


def vcal_qa(vec: list[float]) -> dict:
    """Query VCAL Semantic Cache for a matching answer."""
    r = requests.post(
        f"{VCAL_BASE}/v1/qa",
        headers=HEADERS,
        json={
            "query": vec,
            "k": 1,
            "ef": EF_SEARCH,
            "sim_threshold": SIM_THRESHOLD,
        },
        timeout=30,
    )
    r.raise_for_status()
    return r.json()


def vcal_upsert(ext_id: int, vec: list[float], answer: str) -> None:
    """Store a new answer in VCAL Semantic Cache."""
    r = requests.post(
        f"{VCAL_BASE}/v1/upsert",
        headers=HEADERS,
        json={
            "ext_id": ext_id,
            "vector": vec,
            "answer": answer,
        },
        timeout=30,
    )
    r.raise_for_status()


def call_llm_fallback(question: str) -> str:
    """
    Replace this function with your real LLM call.

    For example, this could call OpenAI, Anthropic, Ollama,
    a local model, or your internal RAG pipeline.
    """
    return f"(LLM fallback) Answer to: {question}"


def main() -> None:
    question = input("Ask a question: ").strip()
    if not question:
        return

    vec = embed_text(question)
    qa = vcal_qa(vec)

    if qa.get("hit") and qa.get("answer"):
        print("VCAL HIT")
        print(qa["answer"])
        return

    print("VCAL MISS -> calling LLM fallback...")
    answer = call_llm_fallback(question)

    vcal_upsert(ext_id_for(question), vec, answer)

    print("Stored in VCAL")
    print(answer)


if __name__ == "__main__":
    main()

---

Run the example

Save the script as:

vcal_minimal_example.py

Run it:

python3 vcal_minimal_example.py

Ask a question once. The first request should usually be a miss because the answer is not cached yet.

Ask the same or a very similar question again. VCAL should return a hit if the similarity threshold is satisfied.

---

Environment variables

Variable	Default	Description
VCAL_URL	http://127.0.0.1:8080	VCAL Semantic Cache base URL
VCAL_API_KEY	empty	Optional API key sent as X-VCAL-Key
OLLAMA_URL	http://127.0.0.1:11434	Ollama base URL
EMBED_MODEL	nomic-embed-text	Ollama embedding model
VCAL_SIM_THR	0.85	Similarity threshold for cache hits
VCAL_EF_SEARCH	128	HNSW search parameter

---

Notes

• VCAL Semantic Cache does not replace your LLM. It reduces repeated calls by reusing semantically similar answers.
• Your application is responsible for creating embeddings and sending vectors to VCAL.
• The embedding dimension must match the server VCAL_DIMS value.
• The example uses a simple deterministic ext_id. Production systems may prefer their own IDs based on tenant, application, document, workflow step, or prompt hash.
• Tune VCAL_SIM_THR based on your workload. A higher threshold is stricter; a lower threshold allows broader reuse.
• For full API details, see the API Reference.