If Face ID constantly fails for someone they'll get another phone, so even if 95% of current users are happy it's hard to tell how well it works.

Given the numbers the SE is selling, it might totally be a factor, on top of the price.