GitDex

Software directory / sw_4c1bed40e6064d

llama.cpp

Open sourceUnreviewedmacoswindowslinuxandroidcli

Visit website

Nobody has independently verified this record. Check the sources before relying on it.

llama.cpp is an LLM inference project written in C/C++ on top of the ggml library, aiming to run LLMs and multimodal models with minimal setup on a wide range of hardware, locally and in the cloud.

Capabilities

  • `llama cli` downloads and runs models directly from Hugging Face
  • `llama serve` starts an OpenAI-compatible API server with a built-in chat app
  • Supports multimodal vision models and text-to-speech models
  • GBNF grammars for constrained output and multi-GPU inference
  • Installs with an install script, Docker, prebuilt binaries or a source build
  • Prebuilt packages for macOS, Windows, Linux (including CUDA, Vulkan, ROCm and SYCL builds) and Android

Translation provided by the submitter. Original in Chinese.

Proposed for these requests

Not proposed for any request yet. Requests directory

Sources

Where the listed facts come from, and how far each was verified.