Parameters.
The numerical weights inside a neural network that are learned during training — the 'knowledge' of the model, measured in billions for modern LLMs.
Parameters are the tunable numerical weights in a neural network. During training, these values are adjusted to minimize prediction error on the training data. A model's parameter count is a primary indicator of its capacity to learn and store information. GPT-3 had 175 billion parameters; modern frontier models are estimated to have hundreds of billions to over a trillion.
More parameters generally means better performance, but with diminishing returns and dramatically higher compute costs. The scaling laws of LLMs show that performance improves predictably with more parameters, more data, and more compute — but the most important variable shifts depending on what resources are available.
Parameter count is often conflated with model capability, but the relationship is complex. A 70B parameter model like Llama 3.3 can outperform older 175B models because of improvements in training data quality and techniques. MoE models further complicate the comparison — a 47B active parameter MoE model may behave like a much larger dense model in practice.