Selective Speaker Clients

I cannot figure out how to enable only specific speaker clients while using BLSpeech as a speech proxy. I can define the list of Speaker Clients in HS SETUP, but this is only honored when BLSpeech proxy is unregistered.

I have only been able to find "Speaker Client to use as main speaker client" in BLSpeech -> Options. But this limits me to a single speaker client, again not useful for my application.

I've tried seeing if it's possible to run a script (prior to HS speaking) that would set the speaker clients, but that too is ignored when BLSpeech proxy sends speech to HS.