For some reason I always imagine porn sites to be run by guys in cheap suits and lots of hair gel. Then I realize that in order to run a site with the kind of traffic that some of them have, you really need to have some decent engineers on staff.
You'd be surprised if you went to meetings with porn business companies in Japan. Stylish offices, elegant suits, great manners, no hint of actual porn anywhere. In fact, you could easily confuse the office with an investment bank.
I once had a surreal meeting with a hotel chain where they mentioned they had detailed management reports on stuff like the average time films are watched for and that they had a graph showing how long porn movies are watched for.
This was all being discussed in the context of a fairly formal business meeting.
Ok, I'm feeling rather naïve and waking up slowly this morning. Why do they care to know that? I mean, what about their business does that help them improve? Because, I think we can take it somewhat for granted that such information probably isn't something which the individuals involved, their customers in this situation, enjoy knowing is being collected about them. "Hey, you know that activity you probably consider rather private, we like to record everything about it we easily can, because umm... customer service."
If I kept track of the porn watching habits of my neighbors, that would be considered strange and maybe rather pervy. And I don't see how it makes a difference in that evaluation whether they are or aren't renting the tv they use from me.
It probably lets them know how long the movies they produce should be. If it's too long before any action is in there then people move on. If it's not long enough then the customer doesn't get what they want. Optimizing this means that they can take a single 8 hour shoot, that'll produce maybe an hour and a half of good footage (just pulling numbers out of my ass) and make more than one video of the right length to get the most profit out of it.
Ok, thanks, I'm restored to cynical now... and the business advantage to be found here is that they can go even further than just optimizing length to match what people want, they can make them just a bit shorter than more people would like, so people will buy two instead of just one.
It's probably an extra-cost service to the hotel guest. The hotel wants to know how much/how often the service is used so they can optimize it for their needs. No sense paying for more bandwidth than you need.
In any case, they probably have no need for individual statistics: the aggregated watching habits of guests is enough to provide useful information.
"For some reason I always imagine porn sites to be run by guys in cheap suits and lots of hair gel."
Pretty much the exact opposite actually. Porn industry guys tend to be 30 - 40 on average and super cool and laid back, though in an LA sort of way. I'm sure there's a lot of crap going on beneath the surface, but at least from the outside it seems like a much nicer work culture than silicon valley.
Sure, not only there is huge traffic, they also generate tons of money, so porn sites are definitely not second citizens in the web from a technical point of view.
Some time ago I've read an IAmA on reddit with staff member of some big porn portal. It was quite interesting but I don't remember what porn site it was and don't want to blindly trying to google it as I'm at work. Writing this in case anyone knows and would post it here :)
the guy from pinklab http://www.pinklab.com/reddit.html talked around there, but can't find the link. there were also some others, even a guy who ran a 2g1c kind of site.
First off, I am not saying that youporn guys are not decent but IMHO, people shouldn't get too excited about this.
Youporn is somewhat quite a simple site I'd imagine. It's just categorized videos with a bunch of stats (views, related video...), and users. There's almost no social interaction between users (i've just registered an account just to confirm that).
That's a very clear (and good) reason why 100% redis suits for this website, apart from the goodness of redis like sorted sets are perfect for the "top rated" or "being watched" sections
Imagine the site now wants to add a little bit of social interaction, like allowing users to post video, follow each others... I'd say 100% redis wouldn't work any more
I'm pretty sure i know what I'm looking for in a porn site. There's many things youporn can do to help me watch better stuff.
Say I'm a fan of a porn star XXX and I'd like to see a simple notification that there's recently a new video tagging her. Same thing if I like some particular user's taste for porn and would like to get noticed if he/she has posted a video. Basically if it gets more social, my wall or dashboard or whatever, would be more sorta relevant.
You would imagine wrong. I'm running a video sharing page (think FB&youtube mashed together with a dash of blogger) with about 800k unique/month users and it's no picnic.
Sure, but the sharing is what makes it hard - as soon as your pages are per-user it gets interesting. For youporn, each video page is essentially static (maybe there are comments? Even if there are, I doubt they display a different set of comments to each /viewer/). So video pages are easy to cache, the database layout is simple, and sure there's some effort involved in scaling up but it's nowhere near as hard as e.g. tumblr's architecture that was featured recently (where every viewer sees a page with all the posts from everyone they're following, and so everyone's page is different)
I apparently did NOT say that it's picnic. And I don't see how your reply is related to my comment. Does your site have enough social features? Like follow, walls... And is your site 100% redis too?
I was just stating that building a site like that is not as trivial as you made it out to be. Yes you "only" have some videos, but you also have views of said videos, and comments and rankings. Building different views of videos based on those factors is hard if you have lots and lots of this data points. Not to mention that you then have to (somewhere) display just some user's videos sorted by how they ware ranked, or maybe just some user's videos from some category based on number of views (which is not a simple INT field in the db). Etc etc etc.
What I'm saying is that there is more than it meets the eye.
Nothing fancy actually. We have 3 dedicated storage servers (two in the same DC as our web server, one in germany), with lots and lots of disk space. We then SCP converted videos to them and serve them trough nginx (supports seeking).
Hey, thank you so much for the information. I hope you don't mind to share a little bit more (it's okay if you don't reply due to IP or whatnot, I'll understand).
How do you handle where to put the files on which disks? do you write custom software or there's some sort of off-the-shelf software to do this?
The reason I asked about this is because it looked like a few people I talked to seem to prefer to use Distributed File System such as HBase, GlusterFS or something else so that they don't have to write extra code.
And I'm guessing you back them all up on daily base as well?
We split files into two levels of folders. For instance if a video has a filename "somethingsomethingXY" it will go into "videos/y/x/" folder. Then it's up to our sys admins to mount those folders to whichever disk they want. So from the app's point of view it doesn't matter where the file actually is. And we backup everything every couple of hours.
Like I said, nothing fancy but it works without any issues (apart from a small delay when you have to copy the file to 2 or 3 different servers (redundancy) before it becomes available, but it's not a big problem but I have few ideas how to solve that if it becomes one :)).
