Go Community Linklog

Made a library? Written a blog post? Found a useful tutorial? Share it with the Go community here or just enjoy what everyone else has found!

Colly - Fast and Elegant Scraping Framework for Gophers

Colly - Fast and Elegant Scraping Framework for Gophers.

With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving.


Clean API
Fast (>1k request/sec on a single core)
Manages request delays and maximum concurrency per domain
Automatic cookie and session handling
Sync/async/parallel scraping
Automatic encoding of non-unicode responses
Robots.txt support
Google App Engine support
Posted by:  Brian Scott