SMILE Serve – a production-ready inference server on Quarkus supporting Classic ML, ONNX, and Llama 3 chat models with bundled React web UI

SMILE Serve bundles three inference capabilities on the JVM: Classic ML (serialized SMILE models), ONNX Runtime (open-format models), and LLM Chat (Llama 3 chat completions with conversation persistence), each accessed via separate API prefixes.

Deployment options include Docker (quickest start), JAR packaging, Uber-JAR (single self-contained file), and native executable compilation via GraalVM for sub-millisecond startup and lower memory use.

Classic ML and ONNX models are auto-discovered from mounted directories at startup; the Chat API supports a configurable maximum sequence length of 4096 tokens and can be backed by PostgreSQL for chat history persistence.