Class LocalModelOptions

Options for configuring the LLama inference engine.

public sealed class LocalModelOptions

Inherited Members: object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Properties

Context window size in tokens.

public uint ContextSize { get; set; }

Number of GPU layers to offload. -1 = all (when GPU available), 0 = CPU only.

public int GpuLayers { get; set; }

Maximum tokens to generate per response.

public int MaxTokens { get; set; }

Temperature for sampling (0 = deterministic).

public float Temperature { get; set; }

Top-P nucleus sampling.

public float TopP { get; set; }