Skip to content
RankShark

llama.cpp

Runs LLMs locally in C/C++ on CPUs and GPUs.

ToolOpen sourceby ggmlsince 2023
72.1

SharkScore

#10 in Model APIs & inference

Why it scores 72.1

llama.cpp has a SharkScore of 72.1/100, ranked #10 of 262 in Model APIs & inference (updated 2026-11-18).

Each part compares llama.cpp with other listings in Model APIs & inference. Evidence covers 87% of the score's weight.

Adoption
57 · 30%
User reviews
no data · 25%
Maintenance
100 · 20%
Trust & transparency
80 · 15%
Momentum
no data · 10%

Paying never changes this score. How ranking works · method v2026-11-18.2

RankShark review of llama.cpp

Automated evaluation by RankShark · 18 Nov 2026 · method v1.0 · How we review

We were not paid for this review.

As of 2026-11-18, llama.cpp is an MIT-licensed C/C++ inference project with a command-line interface and an OpenAI-compatible API server. [E1]

llama.cpp is a C/C++ project for running LLM and multimodal LLM inference locally or in the cloud. Its README includes a command-line tool and an OpenAI-compatible API server. [E1]

Best for

  • Developers who want to run LLM or multimodal LLM inference locally or in the cloud. · home page
  • Teams looking for an OpenAI-compatible API server for inference. · home page

Not for

  • Teams that require stated data-retention, data-region, or DPA terms; these are not stated on the supplied project page. · home page

Key features

  • LLM and multimodal inference. The vendor says the project enables LLM and MLLM inference locally and in the cloud. · home page
  • Hardware range. The vendor says it targets a wide range of hardware. · home page
  • Command-line interface. The vendor provides a CLI, including a quick-start command that downloads and runs a model from Hugging Face. · home page
  • API server. The vendor says llama serve launches an OpenAI-compatible API server. · home page
  • Additional tools. The vendor lists completion, server, text-to-speech model, and GBNF grammar tools. · home page
  • License. The repository lists an MIT license. · home page

Pros

  • Offers both a command-line tool and an OpenAI-compatible API server. · home page
  • The project describes support for local and cloud inference across a wide range of hardware. · home page
  • The repository lists an MIT license. · home page

Cons

  • The supplied project page does not state pricing, data-retention terms, data regions, or a DPA. · home page
  • No public prices found on the pages we checked. · pricing page
  • No free plan or free trial stated on the pages we checked.
  • Whether customer data is used for model training is not stated on the pages we checked.

Pricing

No public prices found on the pages we checked.

Checked 18 Nov 2026 · source

Free plan
Not stated (checked 18 Nov 2026)
Free trial
Not stated (checked 18 Nov 2026)
Trains on customer data
Not stated (checked 18 Nov 2026)
Data retention
Not stated (checked 18 Nov 2026)
Data regions
Not stated (checked 18 Nov 2026)
Data processing agreement
Not stated (checked 18 Nov 2026)
AI models used
The README quick-start example uses ggml-org/Qwen3.5-0.8B-GGUF. · home page
API
llama serve launches an OpenAI-compatible API server. · home page
SDKs
Not stated (checked 18 Nov 2026)
Webhooks
Not stated (checked 18 Nov 2026)
Platforms
The README describes local and cloud inference, Docker usage, and an Android build guide. · home page
Security claims
No security certification stated (checked 18 Nov 2026)

What we measured

GitHub stars · now
135,691
Latest release · now
v1.0.0 on 2026-11-13
How we tested · sources
  • Hourly uptime and response-time checks of the product's URL.
  • Repository, release and download data from GitHub, npm, PyPI and Hugging Face where they exist.
  • We did not sign up, pay, or use the product as a customer.

Something wrong? Report an error with a public link that shows it; corrections re-run the review.

Top alternatives to llama.cpp

See all
  • 78.9

    The home of open models, datasets and Spaces.

    ToolFreemium · from $9/mo
  • 77.5

    API for building with Claude models.

    ToolPaid
  • 76.1

    Proxy and SDK to call 100+ LLM APIs in the OpenAI format.

    ToolOpen source
  • 75.7

    Google Cloud's platform to build with Gemini and other models.

    ToolPaid