Skip to content

OptimaNode Overview

OptimaNode is the inference engine of OptimaGPT. It runs on the machine that hosts your AI model files, manages the processes that serve them, and handles inference requests forwarded to it by the Gateway.

What the Node does

  • Scans a designated directory for GGUF model files and registers them as available executors
  • Launches a dedicated inference process for each enabled model
  • Accepts inference requests from the Gateway over an encrypted WebSocket connection
  • Reports live hardware metrics (CPU, RAM, GPU, VRAM) back to the Gateway
  • Automatically restarts model processes if they crash

The Node has no user-facing interface of its own. All configuration and monitoring is done through the Gateway's admin interface.

How the Node connects to the Gateway

The Node runs as a background service and listens for incoming connections from the Gateway on port 8022 (WSS — encrypted WebSocket). The Gateway initiates the connection; the Node does not need to know the Gateway's address.

When a Gateway connects to a Node for the first time, it performs a handshake using the Node access key — a shared secret that ensures only authorised Gateways can communicate with the Node. This key is configured in the Gateway's Configuration page and must match the key set on the Node.

Running multiple Nodes

A single Gateway can connect to any number of Nodes simultaneously. Each Node operates independently and the Gateway distributes inference requests across all connected Nodes automatically.

Multiple Nodes are useful when: - You have several machines with GPUs and want to spread inference load across them - Different machines are suited to running different models - You need more concurrent request capacity than a single machine provides

Node as a background service

The Node is installed and runs as a system service. It starts automatically when the machine boots and does not require anyone to be logged in. Service management (start, stop, restart) can be done through the operating system's standard service tools if needed:

Windows — Services (services.msc) or PowerShell: 

Restart-Service OptimaNode

Linux — systemd: 

sudo systemctl restart optimanode

macOS — launchctl (via the installer-registered LaunchDaemon).