At some point, as an individual user, you have to trust someone: the browser's vendor, the OS vendor, the hardware vendor, the component's vendor, the factory/distribution chain. For now, trusting the browser seems relatively inevitable. Trusting CAs might not be the only alternative for long, however: https://namecoin.info/ (as one of a few possible but not quite ready for prime time solutions).
You will probably have to trust the browser vendor and the OS vendor somewhat for the foreseeable future, though. In theory, open source can help up to the hardware layer, but even that only really matters if you assume a large enough number of people are auditing every piece of code you run in practice, as well as every tool used to build it. User-auditable hardware seems unlikely any time soon, even when you assume the kind of user that reads diff patches before updating their browser install... a demographics composed of about three guys at Mozilla who are also Gentoo users.
You will probably have to trust the browser vendor and the OS vendor somewhat for the foreseeable future, though. In theory, open source can help up to the hardware layer, but even that only really matters if you assume a large enough number of people are auditing every piece of code you run in practice, as well as every tool used to build it. User-auditable hardware seems unlikely any time soon, even when you assume the kind of user that reads diff patches before updating their browser install... a demographics composed of about three guys at Mozilla who are also Gentoo users.