2

I read on https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-temperature (mirror):

temperature. number. Optional. Defaults to 0. The sampling temperature, between 0 and 1. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. If set to 0, the model will use log probability to automatically increase the temperature until certain thresholds are hit.

and on https://platform.openai.com/docs/api-reference/chat/create#chat-create-temperature :

temperature. number or null. Optional. Defaults to 1. What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. We generally recommend altering this or top_p but not both.

Does the non-ASR API (ASR=Automatic Speech Recognition) also "use log probability to automatically increase the temperature until certain thresholds are hit" when temperature=0, is it that just one OpenAI ASR? Or is there a reason why that technique would be specific to ASR?

Franck Dernoncourt
  • 3,473
  • 2
  • 21
  • 39

1 Answers1

2

For ASR this flexibility can be important since audio inputs can vary greatly in clarity and context, for example, even English-speaking natives cannot identify "can" vs "can't", "map" vs "nap" without more context in many cases, while for other OpenAI models like Chat here each input token has no ambiguity or loss of information through their communication channel, reducing the need for automatic temperature adjustments as ASR in principle.

Therefore since the Chat API doesn't explicitly mention any automatic temperature adjustments feature, from above argument I recommend we should not believe Chat has this feature by default.

cinch
  • 11,000
  • 3
  • 8
  • 17