summaryrefslogtreecommitdiff
path: root/crawler/utility.go
AgeCommit message (Expand)AuthorFilesLines
2018-06-16Adds support for cl. (crawler)horus1-3/+25
2018-06-15Introduces config for user agent, robots.txt and crawler delay. (crawler)horus1-6/+0
2018-06-15Introduces central crawler config. (crawler)Max1-0/+6
2018-02-19Less Fatal(), more Warn(). (crawler)horus1-2/+4
2018-02-19Fix nasty bug. (crawler)horus1-2/+2
2018-02-17Adds crawler for whiskysite.nl. (crawler)horus_arch1-0/+14
2018-02-17Introduces context on errors. (crawler)horus_arch1-12/+11
2018-02-16Structured logging part two. (crawler)horus1-7/+3
2018-02-16Adds structured logging. (crawler)horus1-15/+18
2018-02-12Improvements, bug fixes, more utility functions, etc... (crawler)horus1-1/+64
2018-02-10Crawler extracts volume, price per litre and abv. (Whisky World)horus1-0/+3
2018-02-10Crawler extracts volume, price per litre and abv. (Whisky.de)horus1-0/+3
2018-02-10Improves detecting of spirit_type. (crawler)horus1-9/+19
2018-02-08 Crawler extracts volume, price per litre and abv. (MC Whisky, Rum & Co, Whic)Max1-0/+49
2018-02-08Adds missing utility functions.Max1-0/+65