HN implements some significant limitations to the model that make it work. Things like not being able to downvote replies, can't edit / delete your replies after certain criteria are met, limited upvoting until a certain amount of contribution, etc.

These are constraints that limit the types of people that are interested in remaining here. But HN is by no means "small".