List of Open Source LLM: Self-Host or Use an API

The list, by family

Here is a working list of open source llm families you will meet most often. A caveat first: most of these are open-weight rather than fully open source. The weights are published, but the training data and sometimes the training code are not, and licences differ. Some use permissive licences such as Apache 2.0; others use custom community licences with conditions. Read the licence for the exact model and size before you build a product on it.

The list changes quickly, and new versions appear often, so treat this as a map of who publishes what and not as a ranking. For quality comparisons, run your own prompts. Public leaderboards are a start, but they measure tasks that may not match yours.

Why some open models still refuse

Open weights do not mean open behaviour. Most released chat models are tuned to decline a range of requests. Community fine-tunes exist that remove or reduce those refusals, and which one fits depends on what you need.

The hosted model on this site is one open-weight model, run by an independent provider and tuned to answer without content refusals for lawful adult use. It is not the official API of any vendor in the table. The hard limit is no sexual content involving minors. If your use case is fiction, roleplay, companion apps or moderation research for adults, that behaviour is the reason to pick it. If your use case is something else, you may be fine with a standard model.

What it takes to run one yourself

The first question is memory. A rough rule: parameters x bytes per weight, plus extra for the context cache and runtime. At 16-bit precision each weight takes 2 bytes. At 4-bit quantization it takes about half a byte.

• A 7B model at 16-bit needs about 14 GB for weights. At 4-bit, about 3.5 GB.
• A 13B model at 4-bit needs about 6.5 GB.
• A 70B model at 4-bit needs about 35 GB, which is more than any single consumer graphics card offers, so it takes two cards or a large workstation card.

Add several GB for long contexts. The cache grows with every token in the window, so a 100,000-token conversation is far heavier than a 4,000-token one.

Tools for running models locally include llama.cpp, Ollama and vLLM. llama.cpp and Ollama are popular for single-user machines; vLLM is built for serving many requests on GPUs. All three can expose an OpenAI-style endpoint, so your client code can stay the same, which makes it easy to test both paths.

Budget for the work too: picking a quantization, tuning batch sizes, updating when a better model appears, and handling a crash at 3 a.m. That time has a cost even when the software is free.

Break-even math: server versus per-token pricing

The API charges $0.15 per 1M input tokens and $0.50 per 1M output tokens. With a typical chat mix of four input tokens for each output token, the blended price is (0.8 x $0.15) + (0.2 x $0.50) = $0.12 + $0.10 = $0.22 per 1M tokens.

Now suppose, as a made-up example, that a rented GPU server costs $300 a month. That server matches the API bill only when you process $300 / $0.22 = about 1,364M tokens a month, roughly 1.4 billion. Spread over 30 days, that is 45 million tokens every day, around the clock. A server that sits idle at night or on weekends makes the real figure worse.

The picture reverses at the low end. Here are three monthly workloads on the API:

• 50M input and 10M output tokens: $7.50 + $5.00 = $12.50.
• 200M input and 40M output tokens: $30.00 + $20.00 = $50.00.
• 1,000M input and 200M output tokens: $150.00 + $100.00 = $250.00.

Even the third workload is under the example server price, before counting electricity, setup time or a spare for failures. Your own server costs will differ. Replace the $300 with a real quote and the same formula gives your break-even point.

How to read a model name

Model names in the list pack a lot of information, and knowing how to decode them saves downloads.

• Size, such as 7B or 70B. Billions of parameters. Bigger usually means better quality and more memory.
• Instruct or chat. Tuned to follow instructions. A base model only continues text and is not what you want for a chat app.
• MoE or mixture of experts. Only some of the parameters are active per token, so it runs faster than its total size suggests, but you still need memory for all of it.
• Quantization labels such as Q4 or 4-bit. Lower precision weights that cut memory at some cost in quality. Q4 to Q5 is a common compromise for local use.
• Context length. The advertised maximum window. Check that your runtime and memory can actually serve it.

When you test a candidate, use your real prompts and judge the output yourself. A model that tops a public leaderboard may still write flat dialogue or mishandle your data format.

When self-hosting is the right call

An API is not always the answer. Run the model yourself when one of these is true:

• Data must never leave your network. Legal or contractual rules can rule out any hosted service. Prompts here are not used for training, but they still reach a remote server.
• Volume is high and steady. If you saturate the hardware every hour of the month, owning it can beat per-token billing.
• You need fine-tuning or a specific model. This API serves one model and offers no fine-tuning, no embeddings and no image or audio generation.
• You work offline, on a plane, on an air-gapped machine, or in a place with weak connectivity.
• You enjoy it. Running models is a fine hobby and teaches you a lot about how they work.

Choose the API when your traffic is spiky, your team is small, you want a 100,000-token window without buying memory for it, or you just want to ship. Pay as you go means no cost when nobody is using it, and credit never expires.

Trying the hosted option in five minutes

The service is OpenAI-compatible, so any OpenAI SDK works once you change the base URL and key. The model ID is uncensored.

Copyfrom openai import OpenAI

client = OpenAI(
    base_url="https://api.opensourcellmapi.com/v1",
    api_key="YOUR_API_KEY",
)

resp = client.chat.completions.create(
    model="uncensored",
    messages=[{"role": "user", "content": "Name three trade-offs of 4-bit quantization."}],
)
print(resp.choices[0].message.content)

Sign up with Google or with email and password, claim the $0.50 trial on the API key page, and run the code above. That trial is valid for 7 days, needs no card, and there is one per person. The key page explains each step, the docs cover streaming, tools and JSON mode, and the pricing page lists top-ups from $20 to $500 with bonuses of +5%, +10% and +15% on larger amounts.

A simple decision rule

Write down your expected monthly tokens, split into input and output. Multiply by the two rates. If the result is well below what a GPU server would cost you, start with the API and revisit when volume grows. If you already own a capable machine and care about locality, start local with one of the models in the list above and use the API as overflow or as a comparison. Many projects end up with both.

More guides

• Open Source LLM API Key: Get One and Use It
• Perplexity API Guide