1

Is there neural machine translation methods, that for one input sentence outputs multiple alternative output sentences in that target language. It is quite possible, that sentence in source language have multiple meanings and it is not desirable that neural network discards some of the meanings if there is no context for disambiguation provided. How multiple outputs can be acommodated into encode-decoder architecture, or different architecture is required?

I am aware of only one work https://arxiv.org/abs/1805.10844 (and one referen herein) but I am still digesting whether their network outputs multiple sentences or whether it just acommodates variations during training phase.

TomR
  • 903
  • 6
  • 18

1 Answers1

1

The LSTM seq2seq model typically used for language translation actually can output multiple variations, and does in many implementations.

The decoder stage outputs confidence/probabilities for each word, and it is possible to run the decoder multiple times, taking different samples on each run.

Not only is it possible, but this is actually done by translation engines. A technique called BEAM search is used to manage multiple translations in progress and pick an eventual one to display, based on whether the language model predicts an overall high or low probability for a sentence (this allows for sentences that start with an unusual initial word choice but that are a better choice in context of what is being translated). It would be a minor change to existing algorithms to display a selection of these instead.

Technically you don't need to sample randomly, and can just take the top N probabilities on each step, and recalculate each time on combined probabilities, culling partial sentences that have low probability in favour of continuing more of the high probability sentences.

Neil Slater
  • 33,739
  • 3
  • 47
  • 66