So question about this question: do you think this can be trained (and if so, would that training help with job performance) or it's just an inherent characteristic?
The thing is if you're good at this, it translates into real work performance. So I almost don't care if people have "cheated" and studied up on the question. None of the people I've hired suddenly stopped communicating well on the job after doing well on this question.
I think this is the part where you're doing some magical thinking backed up by anecdotes. Have you done any actual experimental work on those who have failed the question but that you've hired anyways? Have you then been able to objectively measure how well they communicate? Your sample size seems pretty small (a couple hundred) so my guess would be no. I frequently see this kind of thinking about interview questions: "If somebody fails this problem this way, then they'll perform poorly on the job in the following way." Unless you actually have performed a study on this kind of thing, then you're probably just guessing when you say those kinds of things, and you're showing a lot of not particularly rigorous thinking about the relationship between the interviewing process and on the job performance.
It's definitely a small sample size, but I have more data on this than any other techniques I've seen out there.
I've been overruled once after no hiring a candidate. Low and behold the same person ended up nearly getting fired 9 months later for the very same issues that came up from this question. They struggled to answer even the simplest questions with one sentence responses.
So, that's a sample size of 1. Not quite what we would call scientifically rigorous ;). I'm not saying it's not a useful question. I'm just saying, you jump to a whole lot of conclusions that are clearly not based on evidence in your hiring process. That's fine, just be aware that your hiring process is not doing what you're thinking or asserting it's doing.
