Jose Antonio Soriano

llm-output-safety-gate

A small REST service that scores LLM output for toxicity and PII at the application boundary, so policy holds regardless of what the model does internally. POST /check returns a safety score, a safe verdict, and the flags that fired. Toxicity comes from detoxify, PII from Presidio (emails, phones, names, card numbers, SSNs). Each classifier call has a 10-second timeout and three retries before the service answers 504. Logs are structured JSON and never contain the raw input text.

Run itdocker compose up. Models are baked into the image, so there is no download step. A CI workflow and a test directory are in the repo.

NextRefusal-pattern detection, then a hardening layer that turns the scorer into a gateway: API-key auth, rate limiting, ALLOW / REVIEW / BLOCK policy decisions, and an audit log.

driftwatch

A model drift monitor. It compares a batch of predictions against a baseline using a per-feature Kolmogorov–Smirnov test (via Evidently AI), and ships as a library plus a driftwatch check CLI that exits non-zero when drift is found, so it can gate a CI pipeline. Unit-tested, with ruff, mypy, and pytest required to pass before anything merges.

NextPSI as a second test, reading from S3, then an AWS Lambda, CloudWatch alarm, and SNS alert provisioned with Terraform.

ai-safety-observability-platform

The capstone: the safety gate and the drift monitor wired into one platform around a real vLLM inference server. The plan is vLLM in CPU mode with a small model (Qwen2.5-0.5B or TinyLlama) on local k3s, watched through Prometheus and Grafana for output drift, safety-gate violations, latency SLOs, and error rates. The repository currently holds only the project's working notes.

NextBuild and iterate on k3s locally, provision AWS EKS with Terraform for a 48-hour demo window only, then tear it down. CI/CD runs through GitHub Actions.

activation-stager

A utility for capturing a transformer's hidden states during inference without holding them all in RAM. PyTorch forward hooks are registered across every GPT-2 layer and fire correctly. The README works through the memory math (about 1 GB per forward pass on Llama-3-8B) and lists exactly what is still broken.

NextMove logic into a library layer, write each layer's tensor to a safetensors shard as the hook fires, then verify with a round-trip check and a memory profile.

rust_knn

A hybrid Rust and Python k-nearest-neighbors classifier, being pulled out of a self-directed machine learning fundamentals challenge into its own standalone repository. Today the repo holds the Cargo project and the Rust source.

NextThe Python package, a Docker setup, and a README that opens with the problem.