Cool! What software are you using for the crawler? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		Jaruzel on April 22, 2016 \| parent \| context \| favorite \| on: A list of domains emitting [Terry Pratchett] clack... Cool! What software are you using for the crawler?

jonathanmh on April 23, 2016 [–]

I built a not very advanced one with node.js and mongodb :)

Mainly in use in the crawler: * request * cheerio

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact