summaryrefslogtreecommitdiff
path: root/crawler
AgeCommit message (Expand)AuthorFilesLines
2018-02-20Better error reporting. (crawler)horus1-2/+3
2018-02-20Bugfix. (crawler)horus3-14/+15
2018-02-20Bugfix. (crawler)horus2-5/+10
2018-02-19Better error reporting. (crawler)horus1-4/+9
2018-02-19Less Fatal(), more Warn(). (crawler)horus3-4/+8
2018-02-19Saving scraped offers runs in Tx. (crawler)horus3-6/+27
2018-02-19Adds config option to crawl only specific shops. (crawler)horus2-1/+14
2018-02-19Bugfix. (crawler)horus3-3/+6
2018-02-19Removes dead code.horus1-14/+0
2018-02-19Adds repair function. (crawler)horus2-11/+73
2018-02-19Improves sanitizing. (crawler)horus3-5/+39
2018-02-19Bugfix. (crawler)horus2-12/+12
2018-02-19Implements retry on error. (crawler)horus1-9/+16
2018-02-19Fix nasty bug. (crawler)horus2-16/+16
2018-02-19Bugfix + better error reporting. (crawler)horus1-5/+18
2018-02-19Bugfix + detects age. (crawler)horus_arch3-4/+34
2018-02-19Changes log level while crawling from warning to info. (crawler)horus_arch8-52/+52
2018-02-17Merge branch 'master' of git.iamfabulous.de:alkobote.dehorus1-5/+4
2018-02-17Minor improvement in logging. (crawler)horus7-16/+22
2018-02-17Bugfix. (crawler)horus_arch1-2/+0
2018-02-17Fix detect duplicates query, prevents error message. (crawler)horus_arch1-3/+4
2018-02-17Messing with database constraints. (crawler)horus1-33/+29
2018-02-17Bugfix. (crawler)horus1-1/+1
2018-02-17Add error context for MC Whisky. (crawler)horus_arch1-3/+27
2018-02-17Checks for correct abv per spirit type. (crawler)horus1-2/+27
2018-02-17Bugfix in crawler for whiskysite.nl. It's now active (crawler)horus2-9/+19
2018-02-17Sets config option to disable url shorter. (crawler)horus_arch2-11/+20
2018-02-17Adds crawler for whiskysite.nl. (crawler)horus_arch2-5/+93
2018-02-17Introduces context on errors. (crawler)horus_arch15-111/+252
2018-02-17Minor refactoring. (crawler)horus_arch2-37/+30
2018-02-16Log level is now configurable. (crawler)horus3-13/+22
2018-02-16Pass the config file via command line flag. (crawler)horus1-2/+6
2018-02-16Structured logging part two. (crawler)horus10-57/+102
2018-02-16Adds structured logging. (crawler)horus17-64/+196
2018-02-15Bugfix. (crawler)horus3-3/+7
2018-02-14Bugfix. (crawler)horus2-1/+2
2018-02-14Add shop short url. (crawler)horus3-10/+19
2018-02-14Adds URL shorting with polr. (crawler)horus4-2/+82
2018-02-14Changes config params.horus1-16/+18
2018-02-14Bugfix. (crawler+website)horus1-1/+1
2018-02-13Bugfix. (database)horus1-4/+4
2018-02-12Bugfix. (crawler)horus2-6/+6
2018-02-12Commits missing function parameter. (crawler)horus1-1/+1
2018-02-12Crawls now whiskworld.de and whiskyzone.de. (crawler)horus2-19/+102
2018-02-12Crawler saves whole webpage.horus5-0/+20
2018-02-12Improvements, bug fixes, more utility functions, etc... (crawler)horus4-4/+94
2018-02-12Changes in the database schema.horus1-14/+23
2018-02-10Crawler extracts volume, price per litre and abv. (Whisky World)horus3-2/+29
2018-02-10Crawler extracts volume, price per litre and abv. (Whisky.de)horus2-15/+19
2018-02-10Improves whiskyde. (crawler)horus_arch1-0/+27