docs(ops): make site monitoring aware of log rotation

Signed-off-by: LauraGPT <[email protected]>

docs/operations/funasr-com-site-release.md

@@ -84,21 +84,62 @@ the direct GitHub URL so attribution does not change search metadata. Redirect
 targets are defined only in `web-pages/nginx/conversion-map.conf`; never accept a
 target from a query parameter.
 
-Check after one hour and again after 24 hours:
+For each new release, check after one hour and again after 24 hours, recording
+completion once. For release `20260930T211757Z`, both checks are complete; the
+[follow-up record](https://github.com/modelscope/FunASR/pull/3745) reports passing
+static checks with legacy-upstream and log-rotation warnings. Do not repeat these
+completed windows as pending work.
+
+Before reading logs, discover the serving master and verify its PID, command and
+start identity. A failed systemd unit or an empty PID file does not mean that the
+manually started master has stopped. Set `NGINX_MASTER_PID` to the freshly verified
+master, then inspect log descriptors and file metadata without sending signals:
 
 ```bash
-tail -200 /var/log/nginx/error.log
-tail -200 /var/log/nginx/funasr-conversions.log
-curl -fsSI https://www.funasr.com/
-curl -fsSI https://www.funasr.com/deploy/vllm.html
-curl -fsSI https://www.funasr.com/blog/
+set -eu
+master_pid=${NGINX_MASTER_PID:?set the verified Nginx master PID}
+case "$master_pid" in
+  ''|*[!0-9]*) printf '%s\n' 'NGINX_MASTER_PID must be numeric' >&2; exit 2 ;;
+esac
+ps -p "$master_pid" -o pid=,ppid=,lstart=,args=
+find "/proc/$master_pid/fd" -maxdepth 1 -type l -lname '/var/log/nginx/*' \
+  -printf '%f -> %l\n' -exec stat -Lc '%d:%i %s %n' {} \;
+stat -Lc '%d:%i %s %n' /var/log/nginx/error.log /var/log/nginx/funasr-conversions.log
 ```
 
-Count non-smoke conversion requests by route:
+Compare device/inode identities, not just filenames. After rotation, live
+descriptors may still reference `.log.1` or deleted files while the current
+`.log` paths are empty. Empty files are not evidence of no requests or errors.
+Read the actual open logs and relevant rotated and compressed history for the
+chosen time interval; do not draw a health conclusion if that coverage is missing.
+Use equal-duration before/after windows and retain aggregate status/upstream
+counts only, not visitor addresses or raw requests. Filter known validation
+traffic when possible, but remaining requests are not organic traffic by default
+and do not establish star-growth attribution.
+
+The 2026-10-02 read-only inspection found an empty configured PID file while the
+manual master was alive. The packaged rotation selector could not find that
+master in test mode, and its init-script helper is written to return success unconditionally.
+The event that emptied the PID file was not established. Repairing process
+ownership or log reopening is a separate operational action: do not rewrite the
+PID file, restart the working master, or invoke service actions during acceptance.
+
+Check legacy upstream errors separately. Existing WebSocket error-page routing
+converts upstream `502/503/504` failures into `302` redirects, so zero visible 5xx
+is not proof of backend health. Confirm the upstream/listener against the current
+configuration; do not attribute pre-existing backend timeouts to a static release.
+
+Header probes supplement, but do not replace, the page-body hashes, indexed-route,
+asset, mobile-layout and fixed-redirect checks described above:
 
 ```bash
-grep -vE '"(FunASR release smoke|curl/)' /var/log/nginx/funasr-conversions.log \
-  | awk '{print $7}' | sort | uniq -c | sort -nr
+curl --max-time 15 -fsSI https://www.funasr.com/
+curl --max-time 15 -fsSI https://www.funasr.com/deploy/vllm.html
+curl --max-time 15 -fsSI https://www.funasr.com/blog/
 ```
 
-Roll back on elevated 5xx responses, missing indexed routes, mobile overflow, missing assets, invalid conversion redirects, or a failed static validation.
+Investigate elevated 5xx responses, missing indexed routes, mobile overflow,
+missing assets, invalid conversion redirects or failed static validation. Roll
+back when the static release is implicated, after checking the current release
+and retained backup; do not roll back healthy static content solely because a
+legacy backend or log-rotation path is unhealthy.

web-pages/product-site/tests/test_release_scripts.py

@@ -146,3 +146,52 @@ def test_nginx_contract_uses_current_release_and_hardening():
     assert 'funasr-conversions.log' in config
     assert r'\.[0-9a-f]{12}' in config
     assert 'max-age=31536000, immutable' in config
+
+
+def test_monitoring_docs_use_actual_open_log_inodes():
+    document = (REPO_ROOT / 'docs/operations/funasr-com-site-release.md').read_text()
+    monitoring = document.split('## Monitoring\n', 1)[1]
+
+    assert '/proc/$master_pid/fd' in monitoring
+    assert "stat -Lc '%d:%i %s %n'" in monitoring
+    assert '.log.1' in monitoring
+    assert 'rotated and compressed history' in monitoring
+    assert 'tail -200 /var/log/nginx/' not in monitoring
+    assert "grep -vE" not in monitoring
+
+
+def test_monitoring_metadata_snippet_requires_explicit_numeric_master_pid():
+    document = (REPO_ROOT / 'docs/operations/funasr-com-site-release.md').read_text()
+    monitoring = document.split('## Monitoring\n', 1)[1]
+    blocks = re.findall(r'\x60\x60\x60bash\n(.*?)\x60\x60\x60', monitoring, re.S)
+    assert blocks and '{NGINX_MASTER_PID:?' in blocks[0]
+    for block in blocks:
+        subprocess.run(['bash', '-n'], input=block, text=True, check=True)
+        assert not re.search(r'(?m)^\s*(kill|invoke-rc.d|start-stop-daemon)\b', block)
+
+    environment = os.environ.copy()
+    environment.pop('NGINX_MASTER_PID', None)
+    missing = subprocess.run(
+        ['bash', '-c', blocks[0]], env=environment, capture_output=True, text=True,
+    )
+    assert missing.returncode != 0
+    assert 'verified Nginx master PID' in missing.stderr
+
+    environment['NGINX_MASTER_PID'] = 'not-a-pid'
+    invalid = subprocess.run(
+        ['bash', '-c', blocks[0]], env=environment, capture_output=True, text=True,
+    )
+    assert invalid.returncode == 2
+    assert 'NGINX_MASTER_PID must be numeric' in invalid.stderr
+
+
+def test_monitoring_docs_distinguish_static_checks_from_backend_health():
+    document = (REPO_ROOT / 'docs/operations/funasr-com-site-release.md').read_text()
+    monitoring = document.split('## Monitoring\n', 1)[1]
+
+    assert '502/503/504' in monitoring
+    assert '302' in monitoring
+    assert 'not proof of backend health' in monitoring
+    assert 'equal-duration' in monitoring
+    assert 'not organic traffic' in monitoring
+    assert 'static release is implicated' in monitoring