summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)AuthorFilesLines
3 daysfix: correct TSV parsing — use line-by-line reader and proper column indicesdev1-30/+57
- Replace csv.Reader with bufio.Scanner to avoid quote-parsing issues that skipped ~355 entries (e.g. tt1853728 was on line 4.8M and got lost when csv.Reader encountered malformed quoted fields earlier) - Fix column indices: startYear=rec[5], runtimeMinutes=rec[7] (was rec[4]/rec[5] which mapped to isAdult/startYear) - Update basics for ALL imdb entries, not just those missing ratings
3 dayschore: delete .gz files after extracting in downloadImdbDatasetsdev1-0/+3
3 daysmove download pathdev1-1/+1
3 daysfeat: fetchAndUpdateImdbData — download IMDB datasets and populate imdb tabledev2-0/+352
- Check for imdb entries with NULL average_rating - Download title.basics.tsv.gz and title.ratings.tsv.gz to imdbdata/ - Decompress alongside gzip originals - Parse only rows matching our imdb_ids (memory-efficient) - Update: average_rating, num_votes, title_type, primary_title, original_title, start_year, runtime_minutes - Results: 3394 ratings, 3093 basics updated out of 3448 entries
3 daysfeat: populate imdb table with unique title IDs from linksdev1-0/+91
- Extract distinct IMDb title IDs from links.param (host=imdb.com) - Skip IDs already in imdb table and non-title params (nm, ls, etc.) - Insert 3448 unique title IDs into imdb.imdb_id
3 daysfeat: extract IMDB title IDs from links URLs into param fielddev3-15/+87
- Query links table for IMDB title URLs (field=1, host=imdb.com) - Extract ttIDs via regex and batch-update links.param - 5662 rows updated successfully
3 daysfeat: add AccessToken back to Config struct (json:"-" to exclude from ↵dev1-0/+1
serialization)
3 dayschore: remove access_token from config (calculated by program)dev2-4/+0
3 daysfeat: switch config to JSON; add go.mod and config.json.exampledev4-88/+57
- Replace Viper-based config with encoding/json (config.go) - Add config.json with sensible defaults (gitignored) - Add config.json.example with empty values as reference - Initialize go module (go.mod) - Update main.go to use LoadConfig()
3 dayschore: commit existing config.go changesdev1-1/+2
3 daysInitial commitdev2-0/+143