Please enable JavaScript.
Coggle requires JavaScript to display documents.
LLM - Coggle Diagram
LLM
Parameters
Top K
limits the pool of candidate tokens to the k most
probable tokens at each generation step
“top-k” parameter specifies the maximum number of tokens to consider at each generation step based on their probability.
only the top-k most probable tokens are taken into account
Top P
known as nucleus sampling
selection of most probable words while limiting the selection of subset of words
threshold of 0.8 means only words with a cumulative probability reaching 80% will be considered
more
targeted and coherent responses while maintaining some variety.
It is recommended not to use top-p together with temperature and to
choose one or the other.
-
degree of variability in responses,
if higher response means more diverse less coherent, may lead to hallucination.
if lower - more deterministic and focused
-
-
Best Of (best_of)
The “best_of” option is used for optimal completions and returns the “best” completion (the one with the highest log-probability per token).
It is important to note that the value of “best_of” should be greater than “n”.
-
Number of Responses
The number of responses refers to the number of responses generated by the model. For example, if you generate 3 responses, the model will generate 3 responses based on the given context.
-
-