How Nualingo works out who is speaking

Most translation apps make you take turns. You tap the button for your language, speak, then pass the phone across so the other person can tap theirs. Nualingo has no turn buttons. You press Start listening once, put the phone between you, and talk.

There are still a couple of buttons on the screen. Stop ends the conversation, and Finish closes a sentence early if you don't want to wait. Neither is for taking turns. So how does the app know whose words are whose? Mostly, it listens to the language.

It listens for the language, not for a tap

Before you start, you choose two languages: yours and theirs. From then on, the speech service behind Nualingo does two jobs with everything the microphone hears. It writes down the words, and it labels which of your two languages they are in. Because it knows the pair in advance, it listens for those two and is less likely to mistake the opening words of a sentence for a neighbouring language.

The direction of the translation follows the language. Say you chose English and Thai. Thai words are translated into English, and English words into Thai. Nobody has to tell the app who is talking, because the language already says it. Your words appear as large captions in the other person's language while you speak, with the words it is still unsure of shown faintly until they settle.

Why the voice waits for a pause

Captions can change as you talk. A spoken voice can't take its words back, and a voice that read out half a sentence and then corrected itself would be hard to follow. So the calm AI voice that reads the translation aloud waits until the speaker pauses.

The pause does another job too. It is how Nualingo decides a turn is over. That decision is a judgement call: a pause might mean someone has finished, or it might mean they are looking for the next word. How long the app waits before it decides is up to you.

Quick, Balanced or Patient

The Pace setting on the conversation screen sets how long Nualingo waits after someone stops talking before it translates. There are three choices, and each one trades waiting against interruptions.

• Quick translates soon after a short pause. It suits brisk back and forth, like ordering at a counter. The cost is that it may cut in when someone stops to think, and a sentence with a pause in the middle can be split into separate turns.
• Balanced waits for a natural pause. It is the default and suits most conversations.
• Patient waits longer before translating. You get fewer interruptions, and the speech service has more time to correct its words before the turn closes, but there is a little more waiting. It helps with slower speakers, older listeners and noisy rooms.

We kept Balanced as the default for a reason. In our own recorded test runs, Quick cut more sentences short at a pause than Balanced did, and on a typical turn the voice came no sooner. Changing the pace reconnects the microphone, so it is easiest to pick one before you begin.

Talking over each other, or changing language mid-sentence

Real conversations aren't tidy. If someone switches from one of your languages to the other partway through, the switch starts a new turn, translated the other way. If they drop in a word from a third language, such as a place name, it stays with the sentence it was said in, so nothing that was heard is thrown away.

The service also tries to tell voices apart, and a change of voice starts a new turn. That matters when both of you slip into the same language, or talk at once. Overlapping speech is the hardest case for any listener, and two voices that sound alike can get mixed up. If a caption ends up under the wrong person, you can change who said it on that caption.

When the voice plays through the phone's speaker, it holds back while someone is still talking, so it doesn't read a translation over the top of a new sentence. And if the microphone picks up the app's own voice, Nualingo checks whether what it heard repeats what it just said. If it does, those words are shown dimmed and not spoken again. You can still play them if it guessed wrong.

Tips for a smoother conversation

• Pause at the end of a thought, not in the middle of one. A short breath after a sentence is the signal the voice is waiting for.
• If you have finished and don't want to wait, press Finish.
• If you often stop to think mid-sentence, switch to Patient.
• Let the voice finish before you reply, especially on the loudspeaker.
• Turn the television down. Other voices nearby make it harder to tell who is speaking.
• Keep the screen on. Phones stop listening when the screen locks.

Nualingo can make mistakes, and for medical, legal or official matters you should use a professional interpreter. For everything else, the easiest way to get a feel for the pause is to try it with someone. Three minutes a day are free on each device, with no account.

Nualingo can make mistakes. For medical, legal or official matters, use a professional interpreter.