Audio examples

Spatially-Guided Speaker Extraction Based on Relative Position Cues

Aviad Eisenberg, Sharon Gannot, and Shlomo E. Chazan

Faculty of Engineering, Bar-Ilan University, Ramat-Gan, Israel · OriginAI, Israel

The target speaker is selected by where they are relative to the other speakers, for example “the speaker to the left of the interferer” or “the closer speaker”, without a direction-of-arrival estimate or an enrollment recording. Each mixture below is processed once with the command of each speaker. The same network, given a different command, returns a different speaker.

Microphones

Speaker A Speaker B Speaker C Noise

Two speakers

One mixture from each test set of the paper. Each mixture contains two speakers and one directional noise source in a simulated reverberant room. Uniform linear array with 8 cm spacing; the 2-mic models use the two middle microphones.

Test set (i)

Large angular and radial gaps

Angle gap: 96°
Distance gap: 0.56 m
T₆₀: 0.37 s
SNR: 6 dB

Top view. Larger angles are to the left of the array.

Noisy mixture, reference microphone

SI-SDR 3.0 / -5.3 dB

Command #8“The target speaker is farther than and to the left of the interferer”speaker A

Reference, speaker A

U-Net RPC

SI-SDR 18.4 dB · other speaker -34.4 dB

SpatialNet RPC

SI-SDR 21.5 dB · other speaker -41.5 dB

Command #7“The target speaker is closer than and to the right of the interferer”speaker B

Reference, speaker B

U-Net RPC

SI-SDR 13.5 dB · other speaker -33.3 dB

SpatialNet RPC

SI-SDR 17.3 dB · other speaker -47.6 dB

Test set (ii)

Same direction, different distance

Angle gap: 6°
Distance gap: 0.79 m
T₆₀: 0.28 s
SNR: 9 dB

Top view. Larger angles are to the left of the array.

Noisy mixture, reference microphone

SI-SDR 1.6 / -1.7 dB

Command #2“The target speaker is closer to the array than the interferer”speaker A

Reference, speaker A

U-Net RPC

SI-SDR 15.6 dB · other speaker -28.5 dB

SpatialNet RPC

SI-SDR 20.2 dB · other speaker -37.4 dB

Command #1“The target speaker is farther from the array than the interferer”speaker B

Reference, speaker B

U-Net RPC

SI-SDR 14.1 dB · other speaker -30.7 dB

SpatialNet RPC

SI-SDR 18.8 dB · other speaker -38.9 dB

Test set (iii)

Same distance, different direction

Angle gap: 64°
Distance gap: 0.32 m
T₆₀: 0.25 s
SNR: 5 dB

Top view. Larger angles are to the left of the array.

Noisy mixture, reference microphone

SI-SDR -3.0 / -0.1 dB

Command #3“The target speaker is to the right of the interferer”speaker A

Reference, speaker A

U-Net RPC

SI-SDR 14.3 dB · other speaker -42.3 dB

SpatialNet RPC

SI-SDR 19.0 dB · other speaker -56.4 dB

Command #4“The target speaker is to the left of the interferer”speaker B

Reference, speaker B

U-Net RPC

SI-SDR 16.8 dB · other speaker -43.1 dB

SpatialNet RPC

SI-SDR 20.9 dB · other speaker -54.1 dB

Diffuse babble noise

Mixtures from test sets (i) and (ii) in which the directional noise is replaced by babble from 20 talkers placed around the array. The models were trained with directional noise only and were not retrained.

Test set (i)

Large angular and radial gaps

Angle gap: 122°
Distance gap: 0.83 m
T₆₀: 0.25 s
SNR: 11 dB

Top view. Larger angles are to the left of the array.

Noisy mixture, reference microphone

SI-SDR -0.8 / -0.5 dB

Command #7“The target speaker is closer than and to the right of the interferer”speaker A

Reference, speaker A

U-Net RPC

SI-SDR 13.7 dB · other speaker -33.5 dB

SpatialNet RPC

SI-SDR 15.3 dB · other speaker -45.1 dB

Command #8“The target speaker is farther than and to the left of the interferer”speaker B

Reference, speaker B

U-Net RPC

SI-SDR 13.3 dB · other speaker -38.0 dB

SpatialNet RPC

SI-SDR 14.8 dB · other speaker -47.5 dB

Test set (ii)

Same direction, different distance

Angle gap: 6°
Distance gap: 0.93 m
T₆₀: 0.57 s
SNR: 12 dB

Top view. Larger angles are to the left of the array.

Noisy mixture, reference microphone

SI-SDR -1.6 / 0.6 dB

Command #2“The target speaker is closer to the array than the interferer”speaker A

Reference, speaker A

U-Net RPC

SI-SDR 12.8 dB · other speaker -31.7 dB

SpatialNet RPC

SI-SDR 14.1 dB · other speaker -38.5 dB

Command #1“The target speaker is farther from the array than the interferer”speaker B

Reference, speaker B

U-Net RPC

SI-SDR 13.9 dB · other speaker -30.4 dB

SpatialNet RPC

SI-SDR 15.4 dB · other speaker -36.6 dB

Three speakers

Mixtures of three speakers and directional noise. The command identifies speaker A, circled in the diagram, by its rank among the three speakers in distance and in direction. With 2 microphones there are more speakers than microphones.

Test set (i)