Website
.
Fuselkönig.de
.
Projects
.
Status
index
:
fk_crawler
master
The crawler which powers kategorischeraperitif.de/angebote
summary
refs
log
tree
commit
diff
log msg
author
committer
range
path:
root
/
crawler
/
utility.go
Age
Commit message (
Collapse
)
Author
Files
Lines
2018-06-15
Introduces config for user agent, robots.txt and crawler delay. (crawler)
horus
1
-6
/
+0
2018-06-15
Introduces central crawler config. (crawler)
Max
1
-0
/
+6
2018-02-19
Less Fatal(), more Warn(). (crawler)
horus
1
-2
/
+4
2018-02-19
Fix nasty bug. (crawler)
horus
1
-2
/
+2
2018-02-17
Adds crawler for whiskysite.nl. (crawler)
horus_arch
1
-0
/
+14
2018-02-17
Introduces context on errors. (crawler)
horus_arch
1
-12
/
+11
2018-02-16
Structured logging part two. (crawler)
horus
1
-7
/
+3
2018-02-16
Adds structured logging. (crawler)
horus
1
-15
/
+18
2018-02-12
Improvements, bug fixes, more utility functions, etc... (crawler)
horus
1
-1
/
+64
2018-02-10
Crawler extracts volume, price per litre and abv. (Whisky World)
horus
1
-0
/
+3
2018-02-10
Crawler extracts volume, price per litre and abv. (Whisky.de)
horus
1
-0
/
+3
2018-02-10
Improves detecting of spirit_type. (crawler)
horus
1
-9
/
+19
2018-02-08
Crawler extracts volume, price per litre and abv. (MC Whisky, Rum & Co, Whic)
Max
1
-0
/
+49
2018-02-08
Adds missing utility functions.
Max
1
-0
/
+65