Infrastructure tools AI Tools

Self-Hosted AI Stack is an open-source Docker Compose platform for deploying local AI models, chat, document embeddings and retrieval, speech interfaces, and tool integrations. It provides model serving, conversational interfaces, document processing and embedding pipelines, spee... Read more →

Colibrì is an open-source inference engine that streams Mixture-of-Experts (MoE) model weights from disk into a unified VRAM/RAM/storage hierarchy to reduce fast-memory requirements.It implements per-layer expert prefetching, a measured LRU plus learned pinning cache, and batched... Read more →

Featherless is a production workspace for private model hosting and deployment.It provides dedicated GPU allocation and resource management for running and scaling hosted models.A centralized model catalog supports multiple LLM families and model types with versioning and deploym... Read more →

SGLang is a high-performance serving framework for large language and multimodal models. It provides scalable, low-latency inference with high throughput across deployments from a single GPU to distributed clusters.The engine supports a wide range of open models and diverse hardw... Read more →

Experiential Labs is an open-source AI gateway that exposes a single API endpoint for routing requests across hosted providers, user-managed keys, and self-hosted GPUs.It includes an intelligence layer that monitors traffic and selects or routes models per prompt while applying c... Read more →

QuickPod provides access to idle GPU and CPU resources to support scalable model training and inference for ML workloads. The platform includes hosting and a web console for deploying, scheduling, and managing compute jobs.An AI Hub centralizes models, datasets, and integrations... Read more →

Unsloth Desktop is an open-source desktop app for running, training, and deploying AI models locally on macOS, Windows, and Linux.It supports LLMs, diffusion image/video models, audio models (text-to-speech and speech-to-text), and embedding workflows for on-device inference and... Read more →

foundry local runs AI models locally with on-device inference to keep data on device and reduce cloud dependencies.Requires an Azure subscription and supports ONNX Runtime with CPU, GPU, and NPU hardware acceleration.Provides an OpenAI-compatible API for integration with existing... Read more →

Unsloth Studio is a no-code web UI for training, running, and exporting open AI models locally.It allows users to quickly experiment with models like Qwen3.5, NVIDIA Nemotron 3, and GLM-4.7-Flash, without needing extensive technical expertise.Key features include: running GGUF an... Read more →

Empiriolabs AI provides AI model hosting and inference services for developers, enterprises, and model builders.Host open-source models on GPU infrastructure or run optimized proprietary endpoints with extended context and higher-resolution support.Access models via API or a web... Read more →

TextGen is an open-source desktop app for running local LLMs on Windows, macOS, and Linux.It supports text and multimodal (vision) inputs, file attachments (TXT, PDF, DOCX), image understanding, and a notebook tab for free-form generation.Multiple backends and model formats are s... Read more →

Odysseus is a self-hosted AI workspace for running and serving local LLMs, autonomous agents, and multi-turn chat.Local-first, privacy-first architecture lets developers, researchers, and teams run models on private hardware or connect to external endpoints while retaining data c... Read more →

Lemonade is a self-hosted local AI platform for text, image, code and speech workflows that provides a GUI, CLI, REST API and embeddable SDKs.It supports model hosting and inference backends (vllm, Qwen, GLM and others), a model registry with Hugging Face import, and deployment o... Read more →