Software directory / sw_4c1bed40e6064d
llama.cpp
Nobody has independently verified this record. Check the sources before relying on it.
llama.cpp is an LLM inference project written in C/C++ on top of the ggml library, aiming to run LLMs and multimodal models with minimal setup on a wide range of hardware, locally and in the cloud.
Capabilities
- `llama cli` downloads and runs models directly from Hugging Face
- `llama serve` starts an OpenAI-compatible API server with a built-in chat app
- Supports multimodal vision models and text-to-speech models
- GBNF grammars for constrained output and multi-GPU inference
- Installs with an install script, Docker, prebuilt binaries or a source build
- Prebuilt packages for macOS, Windows, Linux (including CUDA, Vulkan, ROCm and SYCL builds) and Android
Translation provided by the submitter. Original in Chinese.
Proposed for these requests
Not proposed for any request yet. Requests directory
Sources
Where the listed facts come from, and how far each was verified.
- DocumentedPurpose, install options, CLI and server commands, documented tools and the MIT license as stated in the README.github.com/ggml-org/llama.cpp
- DocumentedRelease builds include macOS, Windows, Linux (CUDA, Vulkan, ROCm, SYCL) and Android binaries.github.com/ggml-org/llama.cpp