torrent-dedup/README.md

torrent-dedup

Collect torrent infohashes from multiple qBittorrent WebUI instances into a single CSV, with one column per host.

Setup

1. Copy the example host list and fill in real credentials:

   cp qbittorrent-hosts.example.csv qbittorrent-hosts.csv
   

2. Install the dependency:

   pip install requests
   

3. Run:

   python dump_torrent_hashes.py
   

Output is written to torrent-hashes.csv: the header row lists the hosts, and each column contains that host's sorted, de-duplicated infohashes. Hosts that fail still get a column (empty) and are reported on the console.

Options

Option	Default	Description
--hosts	qbittorrent-hosts.csv	Credentials CSV (host,username,password)
--out	torrent-hashes.csv	Output CSV
--only	all hosts	Query only the given URLs or ports, e.g. --only 8083
--workers	8	Hosts queried in parallel
--timeout	60	Per-request timeout in seconds
--page-size	5000	Torrents fetched per API call
--retries	2	Retries for timeouts and 5xx responses
--insecure	off	Skip TLS verification for self-signed HTTPS hosts

Exit codes: 0 all hosts succeeded, 1 some failed, 2 none succeeded.

Filter duplicates

filter_duplicates.py reads torrent-hashes.csv and writes a new CSV with the same columns but only the infohashes that appear on two or more hosts. Hashes unique to a single host are removed. The input file is never modified.

python filter_duplicates.py

Output is written to torrent-hashes-duplicates.csv. Because columns are independent lists, a duplicated hash can sit on different rows in each host's column; the console summary reports how many duplicate hashes exist and how many extra copies that represents.

Option	Default	Description
--input	torrent-hashes.csv	Source CSV
--out	torrent-hashes-duplicates.csv	Filtered CSV
--min-hosts	2	Keep hashes present on at least this many hosts

qbittorrent-hosts.csv, torrent-hashes.csv, and torrent-hashes-duplicates.csv are gitignored because they contain credentials and local data.