Skip to main content

VCAL Semantic Cache

VCAL Semantic Cache is a production-grade semantic cache and memory layer for LLM applications. It helps reduce repeated LLM calls, latency, token usage, and cost while keeping cache data inside your own infrastructure.

Powered by the open-source vcal-core, VCAL Semantic Cache adds:

  • REST API for app traffic and operational control under /v1/*
  • Automatic TTL/LRU eviction
  • Background-safe snapshot persistence
  • Prometheus metrics at /metrics
  • API-key authentication with app and admin keys
  • Mandatory signed-license enforcement
  • Signed binaries and Docker images
  • Tiered licensing: Trial, Growth, Enterprise

Latest release: v0.6.0​

VCAL Semantic Cache v0.6.0 introduces background-safe snapshot persistence.

  • Autosave and manual snapshot saves no longer enter drain mode

    • Runtime snapshot saves capture a consistent in-memory bundle under a cache-state read lock.
    • Snapshot files are then written without pausing normal request processing.

  • Single logical cache-state lock

    • Index, answers, and tombstones are grouped behind one logical state lock.
    • Mutations happen under one write lock.
    • Snapshot capture happens under one read lock.

  • Shutdown-only quiescent final save

    • Graceful shutdown still enters drain mode.
    • VCAL waits for in-flight work and saves a final paired snapshot before exit.

  • Snapshot restore verified

    • Startup restores vcal.index, answers.json, and tombstones.json when valid files are present.
    • Fresh startup creates an empty index when no snapshot exists and VCAL_DIMS is provided.

Why VCAL Semantic Cache​

  • Millisecond cache hits for repeated or semantically similar queries
  • Private, self-hosted, and recovery-friendly
  • Simple HTTP integration with existing LLM, RAG, and agent systems
  • Built-in persistence, licensing, authentication, and metrics

Quickstart (Docker)​

The server will not start without a valid license file. Trial licensing is self-served via the CLI.

# 0) Create a persistent host directory
mkdir -p ./vcal-data
sudo chown -R 10001:10001 ./vcal-data

# 1) Request a 30-day trial code
docker run --rm -it   -v "$(pwd)/vcal-data:/var/lib/vcal"   -e VCAL_LICENSE_PATH=/var/lib/vcal/license.json   ghcr.io/vcal-project/vcal-server:v0.6.0   license trial <your_email>

# 2) Verify the code and write ./vcal-data/license.json
docker run --rm -it   -v "$(pwd)/vcal-data:/var/lib/vcal"   -e VCAL_LICENSE_PATH=/var/lib/vcal/license.json   ghcr.io/vcal-project/vcal-server:v0.6.0   license verify <code>

# 3) Start VCAL Semantic Cache
docker run --rm -p 8080:8080   -v "$(pwd)/vcal-data:/var/lib/vcal"   -e VCAL_LICENSE_PATH=/var/lib/vcal/license.json   -e VCAL_DIMS=768   -e VCAL_CAP_MAX_VECTORS=50000   -e VCAL_TTL_SECS=2592000   -e VCAL_AUTOSAVE_SECS=3600   -e VCAL_AUTOSAVE_ATOMIC=1   -e RUST_LOG=info   ghcr.io/vcal-project/vcal-server:v0.6.0

# 4) Health check
curl -fsS http://localhost:8080/healthz && echo "OK"

For authenticated deployments, configure app/admin keys and pass them with X-VCAL-Key.

Integrations​

ComponentDescription
vcal-coreRust library powering the semantic index and eviction logic
VCAL Semantic CacheREST service layer with licensing, persistence, authentication, and metrics
Grafana / PrometheusDashboards for latency, hit ratio, evictions, snapshots, and license status
Docker / binariesProduction deployment options for Linux x86_64 and musl targets
License systemSigned JSON validated via Ed25519 public key

Next Steps​