I know a lot of people well enough that they know they don't have to be afraid to admit such things to me. This is not a critical distinction because assuming you know what people think is a great way to completely misunderstand them.
As someone who spent 30 years living in the rural south, I can state this with the confidence of personal experience: some people will not reveal their worst opinions until they are extremely sure in the safety of doing so. In some cases, they'll only reveal those opinions to someone else who has repeatedly shared the same opinions with them.
So unless you're giving big racist or misogynist vibes in how you communicate with them, the racists and misogynists you know "well" may be keeping their opinions to themselves.