Web crawler

wlrd · on April 30, 2014

If you have an urge to work in python, I would strongly recommend Scrapy. You could also use Beautiful Soup if you wanted. BS would be a little harder to use but is very useful.

kachen · on May 3, 2014

thank u bros, actually I am planning to make a livescore app. in android and need online feeds for it.. using API seems limited and I don't know any thing of web crawling...So suggest what should I do.

jafingi · on April 30, 2014

You should read about screen scraping :-) That's the essence of web crawling.

kachen · on May 3, 2014

thank u bro.. Mr jafingi do u have any good refrence of screen scraping

jafingi · on May 3, 2014

Well, you'd have to understand the fundamentals of DOM: http://www.w3.org/TR/DOM-Level-2-Core/introduction.html

Also learn about XPath, which is often used as a language to selecting nodes in documents.

Scrapy and BeautifulSoup are popular libraries for scraping in Python.