technovangelist / notes / models reference

models reference

All official models on start with a max context size of 2048 tokens with some exceptions. To increase that to 12345 tokens you would create a modelfile like this:

FROM sourcemodel
PARAMETER num_ctx 12345

Then create the new model and optionally publish it.

model name parameters default context size max context size Mem for full context Full Context Model
gemma 2 9 billion 4096 8192 ? m/gemma2
gemma 2 27 billion 4096 8192 m/gemma2 27b
llama3 7 billion 2048 8192 ? m/llama3
qwen2 7 billion 2048 32768 m/qwen2

If you have any details you want to add, ping me on twitter (@technovangelist),