Blog · October 22, 2026

How to Run an AI Model on Your Own Infrastructure (No OpenAI)

Run an AI model on your own infrastructure, end to end: pick a model, run the server, call it from n8n or code, and monitor it. One concrete walkthrough.

Saina · October 22, 2026 · 5 min read

One walkthrough, start to finish: how to run an AI model on your own infrastructure. The example is a decision model (Saina Helm) on your own box, run with a pinned Python package, called from code and from n8n, with monitoring. The same skeleton works for other self-hostable models; swap the image and the request shape.

Hosted decision models, including OpenAI's GPT-6 Luna Decisions, are simpler to start with: no server to run, and if your data policy allows a hosted API, they may be all you need. This guide is for when the model has to run on your hardware.

Step 1: Pick the model

For workflow decision steps (route, tag, gate, rate), a small open-weights decision model fits: Helm is 0.8B, CPU-sized (~4 GB RAM), and answers typed questions with probabilities. The weights are Apache-2.0, including for commercial use. If your step generates text, that's a chat model, not this guide.

Step 2: Run the server on your own infrastructure

This walkthrough targets Saina Helm 2 0.8B, weights revision 723490269872074d677c491f449a32cffe919c84, with saina 0.6.1. The package and weights were available on 2026-10-23. Use Python 3.11 or newer. The earlier version of this article used Helm 1 and server 0.1.2; do not combine that server with Helm 2 weights.

Pinned pip setup:

python3 -m venv .venv
. .venv/bin/activate
python -m pip install --index-url https://download.pytorch.org/whl/cpu 'torch==2.14.1+cpu'
python -m pip install 'saina[local,server]==0.6.1' 'transformers==5.20.2'
export SAINA_API_KEY=$(openssl rand -hex 32)
export SAINA_CHECKPOINT=run-saina/saina-helm-2-0.8b
export SAINA_REVISION=723490269872074d677c491f449a32cffe919c84
export SAINA_DEVICE=cpu
export SAINA_MAX_LENGTH=8192
export SAINA_CORS_ORIGINS=""
python -m uvicorn saina.server:create_app --factory --host 127.0.0.1 --port 8000 --workers 1 --no-access-log

Use a fresh environment for this CPU recipe; do not reuse an environment with optional GPU kernels installed. The server stays in the foreground. First start downloads about 1.7 GB of weights into the Hugging Face cache. The 8,192-token cap is a conservative starting configuration; measure memory on your own inputs before raising it. Keep the API key available in a second terminal through your secret store, then check readiness and the loaded revision:

curl --fail-with-body http://127.0.0.1:8000/healthz
curl --fail-with-body http://127.0.0.1:8000/v1/ask \
  -H "Authorization: Bearer $SAINA_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"saina-helm-2-0.8b","state":"I was charged twice.","mode":"decision","threshold":0.8,"questions":{"team":{"type":"single_choice","question":"Which team?","options":{"billing":"Charges, invoices, refunds","technical":"Outages, errors, account access","other":"Anything else"}}}}'

Require a successful response with answers.team, its probabilities, selection and reason. A fallback is a valid decision response, not a failed installation. /healthz should report the pinned revision before you connect a workflow.

Docker release status (2026-10-23): the deployment checkout describes 0.6.0 and 0.6.0-cuda, but both tags returned 404 from the public registry during this review. The pip path above is the available release path. Do not use an unverified Docker tag or a floating Compose checkout for this walkthrough; restore a container recipe only after an anonymous pull and inference smoke test pass.

The server reads its configuration from environment variables: SAINA_API_KEY (required; comma-separated for one key per caller), SAINA_CHECKPOINT and SAINA_REVISION (Hub repo plus commit, or a local path), SAINA_DEVICE, SAINA_MAX_LENGTH (token limit; longer inputs are rejected, not truncated) and SAINA_CORS_ORIGINS. Drop --no-access-log if you want uvicorn's request lines. The SDK README has the details.

Start with at least 4 GB RAM and measure peak memory and latency on your inputs. The October 6 CPU timing was for the older release and is not a Helm 2 measurement. Server 0.6.1 uses a bounded queue: by default one execution slot, four waiting requests and a 30-second queue timeout. A full queue returns 429; a queue timeout returns 503. Honor Retry-After and bound retries.

Step 3: Put TLS and auth in front

The command above binds to localhost. For anything beyond same-host calls: reverse proxy (Caddy or nginx) with a real certificate, and per-consumer API keys. No plain HTTP between hops, no keys in workflow exports. Keep SAINA_CORS_ORIGINS="" for private use; explicitly allow a browser origin only if needed.

Step 4: Call it from code

In the second terminal, activate the same virtual environment. This example calls the local server; use your TLS URL for a remote caller.

import os
from saina import Saina, SainaError

client = Saina('http://127.0.0.1:8000', api_key=os.environ['SAINA_API_KEY'])
result = client.ask(
    model='saina-helm-2-0.8b',
    state='I was charged twice for my subscription this month.',
    mode='decision',
    threshold=0.8,
    questions={
        'team': {
            'type': 'single_choice',
            'question': 'Which team should handle this?',
            'options': {'billing': 'Charges, invoices, refunds',
                        'technical': 'Outages, errors, account access',
                        'other': 'Anything else'},
        },
    },
)
answer = result['answers']['team']
# accepted: answer['selection'] is a label, answer['reason'] == 'accepted'
# rejected: answer['selection'] is None, answer['reason'] is
#   'below_threshold' / 'below_margin' / 'tie'; probabilities are still there
print(answer['selection'], answer['reason'], answer['probabilities'])

Errors (for example a bad key, an invalid request, or a 503 from a busy server) raise SainaError; retry on the busy case. A JS SDK exists too (@run-saina/sdk); the HTTP API (POST /v1/ask) is simple enough for curl if you'd rather not depend on either.

Step 5: Call it from n8n

Use an HTTP Request node for this release: POST to your server's /v1/ask, send JSON using the curl body above, and store the bearer key in n8n credentials. Set model to saina-helm-2-0.8b. Follow it with an IF node checking answers.team.reason === "accepted"; send all other decisions to review. Keep HTTP failures on a separate error path.

The community-node source reviewed on 2026-10-23 hard-codes saina-helm-0.8b, so it is not the Helm 2 path in this guide. Imported templates must also have their model ID checked.

A containerized n8n instance cannot reach the host server through its own localhost. Use a reachable TLS endpoint with authentication. n8n Cloud likewise needs a publicly reachable HTTPS endpoint; if that conflicts with your data policy, run n8n inside your infrastructure. Configure bounded retries for 429 and retryable 503 responses.

Step 6: Monitor it

Minimum dashboard, all cheap to collect:

Step 7: Establish the update ritual

Checklist

Sign up to try a decision All posts