<feed xmlns='http://www.w3.org/2005/Atom'>
<title>hnimdbbot, branch main</title>
<subtitle>Pure Cinema!</subtitle>
<link rel='alternate' type='text/html' href='https://git.iamfabulous.de/hnimdbbot/'/>
<entry>
<title>feat: add license_short from Wikipedia license identifier</title>
<updated>2026-06-26T16:36:46+00:00</updated>
<author>
<name>dev</name>
</author>
<published>2026-06-26T16:36:46+00:00</published>
<link rel='alternate' type='text/html' href='https://git.iamfabulous.de/hnimdbbot/commit/?id=2d72767874f6972726ef09082373e6ac01da169a'/>
<id>2d72767874f6972726ef09082373e6ac01da169a</id>
<content type='text'>
- Extract license.identifier (e.g. CC-BY-SA-4.0) into new
  license_short column
- Warn if a license array has more than 1 entry (none seen yet)
- Include license_short IS NULL in getExistingWikiArticles query
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- Extract license.identifier (e.g. CC-BY-SA-4.0) into new
  license_short column
- Warn if a license array has more than 1 entry (none seen yet)
- Include license_short IS NULL in getExistingWikiArticles query
</pre>
</div>
</content>
</entry>
<entry>
<title>fix: only count awards tables in num_accolades</title>
<updated>2026-06-26T16:15:45+00:00</updated>
<author>
<name>dev</name>
</author>
<published>2026-06-26T16:15:45+00:00</published>
<link rel='alternate' type='text/html' href='https://git.iamfabulous.de/hnimdbbot/commit/?id=b8255840d99c1d0bd81db1a40afc74ab13bf8501'/>
<id>b8255840d99c1d0bd81db1a40afc74ab13bf8501</id>
<content type='text'>
extractAccolades was summing rows from all tables (including
episode lists), producing inflated counts (e.g. 708 for
Unreported_World which has 0 actual awards). Now it filters
to tables whose headers contain 'Award'.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
extractAccolades was summing rows from all tables (including
episode lists), producing inflated counts (e.g. 708 for
Unreported_World which has 0 actual awards). Now it filters
to tables whose headers contain 'Award'.
</pre>
</div>
</content>
</entry>
<entry>
<title>feat: add three-level logging with per-request debug output</title>
<updated>2026-06-26T12:14:52+00:00</updated>
<author>
<name>dev</name>
</author>
<published>2026-06-26T12:14:52+00:00</published>
<link rel='alternate' type='text/html' href='https://git.iamfabulous.de/hnimdbbot/commit/?id=06536f57b1fdc76212da6b85fbc9287cc4f0de70'/>
<id>06536f57b1fdc76212da6b85fbc9287cc4f0de70</id>
<content type='text'>
- New --log-level flag: debug (default info), info, silent
  debug: every API request logged (method, URL, status, duration)
  info:  normal events (batch progress, entry counts, summaries)
  silent: only warnings and fatal errors
- Replaced all log.Printf/Fatalf calls with level-gated helpers
- API request timing added to queryWikiArticle, queryWikidataBatch, downloadFile
- Retries and backoff logged in debug mode
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- New --log-level flag: debug (default info), info, silent
  debug: every API request logged (method, URL, status, duration)
  info:  normal events (batch progress, entry counts, summaries)
  silent: only warnings and fatal errors
- Replaced all log.Printf/Fatalf calls with level-gated helpers
- API request timing added to queryWikiArticle, queryWikidataBatch, downloadFile
- Retries and backoff logged in debug mode
</pre>
</div>
</content>
</entry>
<entry>
<title>chore: sync schema.sql with live database</title>
<updated>2026-06-26T11:54:23+00:00</updated>
<author>
<name>dev</name>
</author>
<published>2026-06-26T11:54:23+00:00</published>
<link rel='alternate' type='text/html' href='https://git.iamfabulous.de/hnimdbbot/commit/?id=13992fedaa0beaf93f6214993c95e685d249638f'/>
<id>13992fedaa0beaf93f6214993c95e685d249638f</id>
<content type='text'>
- Add missing columns to imdb: has_no_wiki_article, description,
  license, license_url, num_accolades, wiki_status_code, has_people
- Fix links table: update to match actual DDL with story_id FK
- Fix orphaned links_imdb definition (was missing CREATE TABLE header)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- Add missing columns to imdb: has_no_wiki_article, description,
  license, license_url, num_accolades, wiki_status_code, has_people
- Fix links table: update to match actual DDL with story_id FK
- Fix orphaned links_imdb definition (was missing CREATE TABLE header)
</pre>
</div>
</content>
</entry>
<entry>
<title>fix: add nil checks in extractPeople for missing infobox/section data</title>
<updated>2026-06-26T11:19:02+00:00</updated>
<author>
<name>dev</name>
</author>
<published>2026-06-26T11:19:02+00:00</published>
<link rel='alternate' type='text/html' href='https://git.iamfabulous.de/hnimdbbot/commit/?id=64ef4e5f48e31e3f003fd4abf0c499e87beb3c43'/>
<id>64ef4e5f48e31e3f003fd4abf0c499e87beb3c43</id>
<content type='text'>
- Guard section.has_parts type assertion in extractPeople
- Guard Cast section has_parts iteration with ok check
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- Guard section.has_parts type assertion in extractPeople
- Guard Cast section has_parts iteration with ok check
</pre>
</div>
</content>
</entry>
<entry>
<title>feat: extract actors, directors, screenwriters from Wikipedia API</title>
<updated>2026-06-26T11:10:41+00:00</updated>
<author>
<name>dev</name>
</author>
<published>2026-06-26T11:10:41+00:00</published>
<link rel='alternate' type='text/html' href='https://git.iamfabulous.de/hnimdbbot/commit/?id=ef75353f3710d7566aa8b41922f776ecb3968830'/>
<id>ef75353f3710d7566aa8b41922f776ecb3968830</id>
<content type='text'>
- Extract directors from infobox 'Directed by' field/list
- Extract screenwriters from infobox 'Screenplay by' list
- Extract actors from Cast section list (first link = person name)
- Upsert into people table, link via who table (profession: actor=1, director=2, screenwriter=3)
- Track processed entries with has_people flag column
- Consumer inserts people and marks has_people=1 on success
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- Extract directors from infobox 'Directed by' field/list
- Extract screenwriters from infobox 'Screenplay by' list
- Extract actors from Cast section list (first link = person name)
- Upsert into people table, link via who table (profession: actor=1, director=2, screenwriter=3)
- Track processed entries with has_people flag column
- Consumer inserts people and marks has_people=1 on success
</pre>
</div>
</content>
</entry>
<entry>
<title>fix: prevent dropped wiki entries when channel fills</title>
<updated>2026-06-26T02:19:44+00:00</updated>
<author>
<name>dev</name>
</author>
<published>2026-06-26T02:19:44+00:00</published>
<link rel='alternate' type='text/html' href='https://git.iamfabulous.de/hnimdbbot/commit/?id=6abbf29de5b08448005df974e95bf773de304550'/>
<id>6abbf29de5b08448005df974e95bf773de304550</id>
<content type='text'>
- Remove non-blocking select/default that silently dropped entries
- Channel sized to hold all pending entries (existing + SPARQL)
- Blocking send backpressures SPARQL if consumer is slow
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- Remove non-blocking select/default that silently dropped entries
- Channel sized to hold all pending entries (existing + SPARQL)
- Blocking send backpressures SPARQL if consumer is slow
</pre>
</div>
</content>
</entry>
<entry>
<title>feat: add -wiki-only flag to rerun only wiki data extraction</title>
<updated>2026-06-26T01:37:51+00:00</updated>
<author>
<name>dev</name>
</author>
<published>2026-06-26T01:37:51+00:00</published>
<link rel='alternate' type='text/html' href='https://git.iamfabulous.de/hnimdbbot/commit/?id=15d06c9802d08037283aa218ccc2f92a9236fcc9'/>
<id>15d06c9802d08037283aa218ccc2f92a9236fcc9</id>
<content type='text'>
- fetchWikiArticlesData is standalone again (re-extracted from consumer)
- -wiki-only flag skips SPARQL pipeline, runs only wiki data fetch
- Default behavior: full pipeline (SPARQL + wiki data in parallel)
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- fetchWikiArticlesData is standalone again (re-extracted from consumer)
- -wiki-only flag skips SPARQL pipeline, runs only wiki data fetch
- Default behavior: full pipeline (SPARQL + wiki data in parallel)
</pre>
</div>
</content>
</entry>
<entry>
<title>refactor: pipeline SPARQL and wiki data in parallel</title>
<updated>2026-06-26T01:26:07+00:00</updated>
<author>
<name>dev</name>
</author>
<published>2026-06-26T01:26:07+00:00</published>
<link rel='alternate' type='text/html' href='https://git.iamfabulous.de/hnimdbbot/commit/?id=8e2d742e59b3923852e1ef6e7a5e2ee1de14ce45'/>
<id>8e2d742e59b3923852e1ef6e7a5e2ee1de14ce45</id>
<content type='text'>
- Merge fetchWikiArticles + fetchWikiArticlesData into one pipeline
- SPARQL producer fetches batches, commits each to DB, forwards resolved articles
- Wiki data consumer runs concurrently, fetching at 2s/request
- Each SPARQL batch commits independently (no global transaction)
- Rate limits respected for both Wikidata SPARQL and wiki server
- No parallel requests to either endpoint
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
- Merge fetchWikiArticles + fetchWikiArticlesData into one pipeline
- SPARQL producer fetches batches, commits each to DB, forwards resolved articles
- Wiki data consumer runs concurrently, fetching at 2s/request
- Each SPARQL batch commits independently (no global transaction)
- Rate limits respected for both Wikidata SPARQL and wiki server
- No parallel requests to either endpoint
</pre>
</div>
</content>
</entry>
<entry>
<title>.</title>
<updated>2026-06-26T00:48:18+00:00</updated>
<author>
<name>dev</name>
</author>
<published>2026-06-26T00:48:18+00:00</published>
<link rel='alternate' type='text/html' href='https://git.iamfabulous.de/hnimdbbot/commit/?id=f3ed9396113de4d81ae0fadabc10ae59f7076862'/>
<id>f3ed9396113de4d81ae0fadabc10ae59f7076862</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
</feed>
