Follow-up to the previous commit on this branch: that one added
--invalid/--include-invalid to check_channels.py but deliberately left
them unwired, because generate_dashboard.py's "most recent row per
(list, channel)" view assumed every row was a working channel -
turning --include-invalid on in a workflow would have bloated the
"needs attention" table with hundreds of already-known-dead Invalid
rows, while hiding the one thing that's actually interesting: an
Invalid channel that's alive again wouldn't show up there at all,
since that table only lists rows that are *not* alive.
- generate_dashboard.py: every existing count (cards, trend, by-list,
"needs attention") is now built from `>` (working) rows only, via a
`marker_of()` helper that treats history written before this field
existed as `>` too, since only working channels were ever checked
back then - so this is a no-op for every dashboard that hasn't
started using --include-invalid yet.
- New "Flagged Invalid, answering again" section lists `x` rows whose
latest check came back alive/flaky - the dashboard's equivalent of
check_channels.py's own `RECOVERED?` line - with a note that this is
a shortlist for a human, not something to auto-restore.
- check_channels_deep.yml now runs with --include-invalid, and its
step-summary blurb mentions the new section.
Tested: wrote a small synthetic history.jsonl mixing a marker-less
legacy row, `>` rows, and `x` rows (one alive, one dead) and confirmed
the channel count/cards only reflect the `>` rows, the alive `x` row
shows up under "Flagged Invalid, answering again", the dead `x` row
shows up nowhere (expected - it's just confirming the status quo), and
an all-`>` history renders the new section's "none right now" empty
state. Also ran it against this repo's real 77MB
.github/checker-history/history.jsonl (all pre-dates this change, so
no `x` rows in it) - renders in ~1.5s, unchanged output. pylint:
10.00/10 on both files.
Builds on the idea in #1145 (a checker that maps failures back to a channel
and list, and tells a geo-blocked channel from a dead one) with a few
additions:
check_channels.py
- Standard library only, same as #1145's version.
- Adds a `disputed` state: an optional --confirm-dead pass gives ffprobe a
second opinion on anything that looks dead over HTTP, before it gets
reported as dead. This is one-directional (can only pull a verdict out
of `dead`, never push one into it) because ffprobe itself is not
reliable enough to trust in the other direction - a known-good DASH
channel needed longer than any sane per-channel budget to open while
testing this, and a header check alone had already been fooled by an
isolated media fragment sitting at a URL that looked like a live channel
(a mistake made and caught while triaging #1149/#1151 - see the
docstring for the details).
- Restructured to share one worker pool across every list in a run
instead of a fresh pool per list, which stopped small lists from paying
the same wall-clock floor as large ones; a full run across all lists
dropped from not finishing in 15 minutes to about 10.
- --json writes one record per channel per run, for building a history.
generate_dashboard.py
- Turns that history into a single self-contained docs/index.html: current
state breakdown, an alive-share trend across every run kept, a per-list
breakdown, and a searchable/filterable table of everything that is not
currently alive.
Two new scheduled workflows
- check_channels_fast.yml: every 6 hours, HTTP checks only.
- check_channels_deep.yml: every 2 days, with --confirm-dead (needs ffmpeg).
Both append to .github/checker-history/history.jsonl (pruned to 90 days),
then build and deploy the dashboard to GitHub Pages.
Kálmán „KAMI” Szalai
Kálmán „KAMI” Szalai