summaryrefslogtreecommitdiff
path: root/crawler/config.go
AgeCommit message (Expand)AuthorFilesLines
2019-05-14Changes imports to lower case. (crawler)Maximilian Möhring1-1/+1
2018-09-16Rename flags. Adds new flag to exclude shops. (crawler)horus1-3/+6
2018-06-15Introduces config for user agent, robots.txt and crawler delay. (crawler)horus1-3/+19
2018-02-21Adds some docu. (crawler)horus1-4/+7
2018-02-21Prevents panic when passing a non-existent path as config file. (crawler)horus1-2/+7
2018-02-20Reintroduces debug as a config setting. (crawler)horus1-0/+3
2018-02-20Refactoring + adds a more granular log level setting. (crawler)horus1-4/+12
2018-02-19Adds config option to crawl only specific shops. (crawler)horus1-0/+3
2018-02-19Adds repair function. (crawler)horus1-9/+13
2018-02-17Sets config option to disable url shorter. (crawler)horus_arch1-10/+13
2018-02-16Log level is now configurable. (crawler)horus1-6/+1
2018-02-14Bugfix. (crawler)horus1-1/+1
2018-02-14Add shop short url. (crawler)horus1-5/+5
2018-02-14Adds URL shorting with polr. (crawler)horus1-0/+2
2018-02-14Changes config params.horus1-16/+18
2018-02-14Bugfix. (crawler+website)horus1-1/+1
2018-02-06Moves crawler to designated directory.Max1-0/+76