The original title is "Building an AI Runtime Operating System for Commodity Hardware (UGR)"

Short summary

UGR is an open-source AI runtime orchestration layer that treats inference workloads like OS processes, paging tensors, KV cache, and MoE experts across VRAM, RAM, and NVMe tiers instead of requiring an entire model to fit in GPU memory. Built for commodity hardware like a 4 GB Quadro GPU, it sits above engines like llama.cpp and exposes an OpenAI-compatible API. The project includes a simulation engine to predict memory usage and throughput before execution.

• •UGR is an open-source orchestration layer that pages AI resources across VRAM, RAM, NVMe, and remote storage tiers
• •Designed for commodity and older hardware; developed on a 4 GB Quadro M1200 GPU
• •Includes a simulation engine to predict memory usage, throughput, and bottlenecks before model execution

Generated with AI, which can make mistakes.