Quick Start Guide
AI_MODEL should stay
default unless there is a folder in
prompts specific to the model that you're using. You can also create one and add your own prompts.
Update your agent settings
defaultor the name of the model from the
- The default
host.docker.internalrefers to docker's dns, which resolves to the internal IP address address of the host.
5001is the default port of KoboldCpp. Update this if you're using a different port on localhost or if you need to speicfy a different URI for the Kobold server.
- If you encounter issues using
host.docker.internal, refer to the troubleshooting section.
Refer to the API documentation for more details. For both KoboldAI and KoboldCpp you can view them locally by adding
/api at the end of the endpoint url (e.g
Here are some key notes and considerations on certain parameters:
MAX_CONTEXT_LENGTH: Defaults to the current configuration from the host if unset.
MAX_LENGTH: If unset, AGiXT dynamically calculates this by subtracting the tokens sent from
MAX_CONTEXT_LENGTHto fit within the context window. If
MAX_LENGTHis greater than or equal to
MAX_CONTEXT_LENGTH, it defaults to
STOP_SEQUENCE: If unset, AGiXT will utilize
PROMPT_SUFFIXif available. Avoid enclosing keywords in quotes and be mindful of whitespaces and escaped characters between commas. e.g
<s> , \n</s>is interpreted as
["</s> ", " \n</s> "].
- Parameters such as
"use_authors_note", etc., are intentionally excluded as they modify the context window and could lead to unintended results with agents.
"trim_stop"is omitted as AGiXT automatically applies it to all outputs.
Note: For any input field, strings are already encapsulated so no need to add
Important: There's a chance that an extra backslash is added to escaped characters (e.g
\n). If this is the case for you, edit the
config.json of the agent and manually remove them.
Setting the correct host for Koboldcpp
- Find out the name of your local host using the command in your system's terminal
hostnameand use it in
AI_PROVIDER_URIis set as
- In the new graphical interface of Koboldcpp, set the HOST to
0.0.0.0in the network settings to make the server listen on all interfaces, and choose any available port, for example,
External access: If the Kobold server listens on all interfaces, it can be accessible from outside your network if the corresponding ports are not properly secured. This means you should configure adequate firewall rules and security measures to protect your server from unauthorized access.
More complex and correct way:
Updating Koboldcpp server settings:
- Open the settings of
Koboldcppin the graphical interface.
- In the network settings section, find the
HOSTparameter and set its value to
0.0.0.0. This will allow the Koboldcpp server to accept connections on all interfaces.
- Choose a free port, for example,
8000, and set it in the
PORTparameter or a similar parameter in the graphical interface.
- Save the changes in the settings of the Koboldcpp server.
Determining the IP address of the AGiXT Docker network gateway:
To find the IP address of the Docker network gateway for the AGiXT project, follow these steps:
- Open a terminal on your system.
- Execute the following command to get a list of all active Docker networks: docker network ls
- In this list, find the network associated with the AGiXT project. It may have a name specified in the AGiXT settings. Note: usually, the network name starts with the prefix specified in the AGiXT settings. For example, it might be named
- Once you find the AGiXT project network, record its name or identifier.
- Now, execute the following command to get the IP address of the Docker network gateway for the AGiXT project network. Replace
<NETWORK_NAME>with the name or identifier of the network from the previous step: docker network inspect
| grep Gateway For example: docker network inspect agixt_default | grep Gateway
- The result of this command will be a line containing the IP address of the Docker network gateway. Record this IP address.
Updating AGiXT agent settings:
- Open the settings of your AGiXT agent.
- Find the
AI_PROVIDER_URIparameter and set it to
IP_GATEWAY_NETWORKis the IP address of the AGiXT Docker network gateway that you obtained in the previous steps. For example:
Checking the availability of Koboldcpp for Docker AGIXT containers:
- You must have Koboldcpp running with your selected model.
- The Koboldcpp server must be available at
http://0.0.0.0:8000on your host system.
- Your AGiXT project must be running.