Hugging Face Transformers
Quick Start Guide
Note: AI_MODEL should stay
default unless there is a folder in
prompts specific to the model that you're using. You can also create one and add your own prompts.
Update your agent settings
MODEL_PATHto the path of your llama.cpp model (for docker containers
models/is mapped to
defaultor the name of the model from the
AI_TEMPERATUREto a value between 0 and 1. The higher the value, the more creative the output.
MAX_TOKENSto the maximum number of tokens to generate. The higher the value, the longer the output.