summaryrefslogtreecommitdiff
path: root/crawler/utility.go
AgeCommit message (Collapse)AuthorFilesLines
2022-11-16link shorter reuses short urlsMaximilian Möhring1-0/+10
2022-11-15No external link shorter anymore.Maximilian Möhring1-0/+16
2021-01-21Detects Rye/Bourbon as Whisky.Maximilian Möhring1-0/+7
2021-01-12Accepts sparkling wine from d12.Maximilian Möhring1-1/+1
2021-01-12Fix detecting spirit types.Maximilian Möhring1-28/+51
2019-05-19Workaround for a bug in d12s article description.Max1-1/+1
2019-05-15Filter nonalcoholic drinks from Bottleworld. (crawler)Maximilian Möhring1-1/+2
2019-05-07Improves detection of different spirit types and removes a lot of dead code.horus1-5/+54
2019-05-07Better spirit type detection. (crawler)horus1-1/+64
2018-06-16Adds support for cl. (crawler)horus1-3/+25
2018-06-15Introduces config for user agent, robots.txt and crawler delay. (crawler)horus1-6/+0
2018-06-15Introduces central crawler config. (crawler)Max1-0/+6
2018-02-19Less Fatal(), more Warn(). (crawler)horus1-2/+4
2018-02-19Fix nasty bug. (crawler)horus1-2/+2
2018-02-17Adds crawler for whiskysite.nl. (crawler)horus_arch1-0/+14
2018-02-17Introduces context on errors. (crawler)horus_arch1-12/+11
2018-02-16Structured logging part two. (crawler)horus1-7/+3
2018-02-16Adds structured logging. (crawler)horus1-15/+18
2018-02-12Improvements, bug fixes, more utility functions, etc... (crawler)horus1-1/+64
2018-02-10Crawler extracts volume, price per litre and abv. (Whisky World)horus1-0/+3
2018-02-10Crawler extracts volume, price per litre and abv. (Whisky.de)horus1-0/+3
2018-02-10Improves detecting of spirit_type. (crawler)horus1-9/+19
2018-02-08 Crawler extracts volume, price per litre and abv. (MC Whisky, Rum & Co, Whic)Max1-0/+49
2018-02-08Adds missing utility functions.Max1-0/+65