๐
2026-04-17
2 items
๐ Updated 2026-04-17 13:51
High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.
๐ Updated 2026-04-17 02:45
A GPU cluster manager that configures and orchestrates inference engines like vLLM and SGLang for high-performance AI mo...
๐
2026-04-16
11 items
๐ Updated 2026-04-16 23:44
Connect any LLM-powered client app, such as a coding agent, to any supported inference backend/model.
๐ Updated 2026-04-16 16:20
Manage concurrent LLM requests with priority queues, multi-model routing, fault tolerance, and semantic caching for effi...
๐ Updated 2026-04-16 16:06
๐ Accelerate attention mechanisms with FlashMLA, featuring optimized kernels for DeepSeek models, enhancing performan...
๐ Updated 2026-04-16 16:06
โก Optimize attention mechanisms with FlashMLA, a library of advanced sparse and dense kernels for DeepSeek models, imp...
๐ Updated 2026-04-16 15:46
๐ Harness mini-SGLang to power efficient inference for Large Language Models with a lightweight, high-performance fra...
๐ Updated 2026-04-16 15:43
๐ค Enhance multilingual communication with T5Gemma-TTS, a versatile Text-to-Speech model supporting easy training and ...
๐ Updated 2026-04-16 15:43
๐ Optimize LLM requests by routing them to the best models, while leveraging semantic caching and real-time cost anal...
๐ Updated 2026-04-16 15:36
SGLang is a high-performance serving framework for large language models and multimodal models.
๐ Updated 2026-04-16 15:04
๐ Simulate LLM inference performance to identify bottlenecks and optimize models with InferSim, a lightweight and dep...
๐ Updated 2026-04-16 14:51
๐ค Automate local private AI setups for demos, showcasing models for diverse tasks like coding, image generation, and ...
๐ Updated 2026-04-16 13:23
Accelerate DeepSeek Sparse Attention models by reusing cross-layer indexes to cut computations and speed up inference wi...