Confidence hijacks the human brain. Without direct, personal expertise or experience to the contrary, spending time around your hypothetical "friend who's very well-read and talkative but is also extremely confident and loves the sound of their own voice" is going to subconsciously influence your opinions, possibly without you even knowing.
It's easy to laugh and say, well I'm smart enough to defeat this. I know the trick. I'll just mentally discount this information so that I'm not unduly influenced by it. But I suspect you are over-indexing on fields where you are legitimately an expert—where your expertise gives you a good defense against this effect. Your expertise works as a filter because you can quickly discard bad information. In contrast, in any area where you're not an expert, you have to first hold the information in your head before you can evaluate it. The longer you do that, the higher the risk you integrate whatever information you're given before you can evaluate it for truthfulness.
But this all assumes a high degree of motivation and effort. Like the opening to this article says, all empirical evidence clearly points in the direction of people simply not trying when they don't need to.
Personally, I solve the problem in my friend circle by avoiding overconfident people and cultivating friendships among people who have a good understanding of their own certainty and degree of expertise, and the humility to admit when they don't know something. I think we need the same with these AIs, though as far as I understand getting the AI to correctly estimate its own certainty is still an open problem.
It's easy to laugh and say, well I'm smart enough to defeat this. I know the trick. I'll just mentally discount this information so that I'm not unduly influenced by it. But I suspect you are over-indexing on fields where you are legitimately an expert—where your expertise gives you a good defense against this effect. Your expertise works as a filter because you can quickly discard bad information. In contrast, in any area where you're not an expert, you have to first hold the information in your head before you can evaluate it. The longer you do that, the higher the risk you integrate whatever information you're given before you can evaluate it for truthfulness.
But this all assumes a high degree of motivation and effort. Like the opening to this article says, all empirical evidence clearly points in the direction of people simply not trying when they don't need to.
Personally, I solve the problem in my friend circle by avoiding overconfident people and cultivating friendships among people who have a good understanding of their own certainty and degree of expertise, and the humility to admit when they don't know something. I think we need the same with these AIs, though as far as I understand getting the AI to correctly estimate its own certainty is still an open problem.