Chat Settings

This toggle allows you to turn on or off streaming mode. When enabled, the model will stream responses word by word or in chunks rather than delivering the full response at once. This can create a more dynamic, conversational experience.

The temperature controls the randomness of the AI’s responses.

• A lower temperature (< 1) makes the model more deterministic, meaning it will generate more focused and repetitive responses.
• A higher temperature (> 1) increases randomness and creativity, making the responses less predictable but can lead to less coherent results.
  For balanced creativity and coherence, a temperature between 0.7 and 1.0 is commonly used.

This setting affects how the model selects words based on cumulative probability.

• Top P sets a threshold for considering the most likely word choices. For example, a Top P of 0.9 means the model will sample from the smallest possible set of words whose combined probability is 90%.
• Lower Top P values result in more conservative responses, while higher values (closer to 1.0) allow for more diversity.

This approach balances diversity and relevance by focusing on a subset of probable outcomes.

Top K controls how many of the highest probability word choices are considered at each step.

• For instance, Top K = 40 means the model will only consider the top 40 most likely words, filtering out less probable options.
• A lower Top K (e.g., 5 or 10) will make the responses more deterministic, while a higher Top K (e.g., 50 or 100) allows for more variety in responses.
  Recommendation: A typical value is between 20 and 50, depending on how much diversity you want in responses.

This field allows you to input a custom prompt that adjusts the context or behavior of the AI.

You can add specific instructions here, such as “Respond concisely” or “Act as a friendly assistant,” which will influence how the AI interacts within the session.