Mid-Build Figuring Out Conversation Turns

Six months into building AI companion robots and the hardest problem so far isn't vision, wiring (although I keep burning out my track brains somehow), or speech. It's teaching my robots how to have a real conversation.

The white one is my v3 prototype (mid-build), brain talking to the cloud. The little pink one is v1, a Raspberry Pi 3B with a lav mic and a servo neck. They live on the same bench and can hear everything in the room, which turns out to be problem at times.

Things that went wrong before it went right:

• They answered each other's echoes. Robot A hears robot B through its own mic, garbles it, and replies to a sentence nobody said.
• Speaker attribution fell apart. One of them decided the other robot's voice was me. I was not talking.
• They were too polite to stop. Two assistants that both always answer will volley forever.
• They replied instantly to the FIRST sentence of a reply, while the other robot was still mid-thought.

What actually works:

1. Name-first etiquette. You say a robot's name to talk to it, and they extend each other the same courtesy. Voice enrollment is a thing too but flaky.
2. They stopped listening to each other acoustically. When one speaks, the other gets the text over a link, complete and correctly attributed. The sound in the room is for the humans.
3. Delivery waits until the speaker has actually finished saying the words out loud.
4. A circuit breaker: three exchanges with no name used and they stand down.

The result is two robots that chat about their morning and then shut up. Getting them to shut up politely was harder than getting them to talk.

Getting them to handle a group conversation is next. I have a 4-mic array on order. Hoping hardware will help.

Curious how others have handled turn-taking with multiple voice agents and people in one room.