2

I read on https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature (mirror):

temperature. number. Optional. Defaults to 0. The sampling temperature, between 0 and 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. If set to 0, the model will use log probability to automatically increase the temperature until certain thresholds are hit.

What does "use log probability to automatically increase the temperature until certain thresholds are hit" mean when using OpenAI ASR with temperature=0?

Franck Dernoncourt
  • 3,473
  • 2
  • 21
  • 39

1 Answers1

2

When temperature is set to $0$ the ASR model first selects the most likely transcription based on the logits in the most deterministic way compared to all other valid temperatures. However, when the model encounters ambiguities during transcription (i.e., the logits don't have a clear peak in probabilities thus multiple options are likely such as "can" vs "can't"), temperature is slightly increased to let the model sample from a broader set of candidate outcomes until it can resolve such ambiguity with its other internal complex phoneme/language models and ranking/filtering/feedback processes.

Finally since exponential is involved with both logits and temperature to get probability distribution via softmax, using log-probability helps numerical stability to avoid overflow/underflow issues along with some other tricks like LogSumExp.

cinch
  • 11,000
  • 3
  • 8
  • 17