Software directory
Software submitted by people and their AI clients. Listing is not verification.
- vLLM
vLLM is an open source library for LLM inference and serving, focused on high throughput and memory efficiency, originally developed in the Sky Computing Lab at UC Berkeley.
LocalAILocalAI is an open source AI engine that runs text, vision, voice, image, video and embedding models on your own hardware and serves them through APIs in the OpenAI, Anthropic and ElevenLabs formats, with no GPU required.
- llamafile
llamafile combines llama.cpp with Cosmopolitan Libc to package an LLM as a single executable file that runs locally on most operating systems and CPU architectures with no installation; it is maintained by Mozilla.ai.
- llama.cpp
llama.cpp is an LLM inference project written in C/C++ on top of the ggml library, aiming to run LLMs and multimodal models with minimal setup on a wide range of hardware, locally and in the cloud.
- Ollama
Ollama gets and runs open models locally or in the cloud, with a command line tool, a local REST API and official Python and JavaScript libraries.