Skip to main content

Troubleshooting

Problem: "VCAL_KEY_APP not set"​

You’ve enabled authentication (VCAL_AUTH_REQUIRED=1) but didn’t define any API keys.

Fix: Save the keys to /opt/vcal-server/app.keys and /opt/vcal-server/admin.keys.
Specify the paths in env vaiables:

VCAL_KEYS_APP_FILE=/opt/vcal-server/app.keys
VCAL_KEYS_ADMIN_FILE=/opt/vcal-server/admin.keys

You can include multiple keys each on a new line. Restart the server after adding keys.

Problem: "Snapshot not found"​

Ensure /data is mounted and the snapshot files exist:

ls ./data/vcal.index ./data/answers.json

If the directory is empty, either disable autosave or run a manual snapshot via:

curl -X POST -H "X-VCAL-Key: <admin_key>" http://localhost:8080/v1/snapshot/save -d '{"path":"vcal.index"}'

Problem: Prometheus shows no metrics​

Metrics are exposed at /metrics.
Check that your Prometheus target is correctly configured and not blocked by a firewall or reverse proxy.

curl http://localhost:8080/metrics | head

If VCAL_AUTH_REQUIRED=1 and metrics must be public, set:

VCAL_ALLOW_METRICS_WITHOUT_AUTH=true

Problem: High memory usage​

Check and adjust your caps:

VCAL_CAP_MAX_BYTES=1073741824   # 1 GB
VCAL_CAP_MAX_VECS=1000000       # optional cap by vector count

Monitor eviction metrics (vcal_evictions_total) and consider increasing VCAL_EVICT_INTERVAL_SECS to tune background sweeps.

Problem: Autosave not working​

Make sure autosave is enabled:

VCAL_AUTOSAVE_SECS=3600
VCAL_AUTOSAVE_ATOMIC=1

You should see logs such as:

INFO Autosave (paired) completed pause_ms=xx

If not, check that the process has write permissions to ./data.

Problem: "missing field dims" or snapshot load error​

Occurs when older snapshots (legacy JSON) are loaded after upgrading to Envelope v1 format.
Fix: remove legacy files and start fresh:

rm ./data/vcal.index ./data/answers.json

VCAL will rebuild the index automatically.

Problem: Cannot access Swagger UI​

The Swagger UI is served at /api-docs.
Ensure that the swagger-ui feature is enabled and no proxy blocks /api-docs requests.

Problem: API returns 403 Forbidden​

Occurs if the license key or API key is invalid, expired, or missing.
Verify your key with:

curl -H "X-VCAL-Key: <admin_key>" http://localhost:8080/v1/license/status

License enforcement events are tracked via Prometheus metric vcal_license_enforcement_events_total.

Problem: Cache hit ratio remains low​

Possible causes:

  • Embedding model mismatch (VCAL_DIMS differs from your embedder)
  • TTL too short (VCAL_TTL_SECS)
  • Query variations (paraphrased input → new vector every time)

Use the /metrics endpoint to monitor vcal_hits_total and vcal_misses_total for diagnosis.

Logs and Diagnostics​

To view runtime logs:

Docker:

docker logs --tail=50 vcal-server

Binary:

RUST_LOG=info ./vcal-server

Verbose debugging:

RUST_LOG=debug VCAL_LOG_FILE=./data/vcal.log ./vcal-server

Rotate or clean up logs periodically if persistent logging is enabled.

Still stuck?​

  1. Run bash paired-files-check.sh for snapshot integrity.
  2. Check /metrics for anomalies.
  3. Rebuild from fresh state if corruption is suspected.
  4. Contact support (Growth/Enterprise licenses) with logs and index metadata.