09b2add673
New evalharness/fingerprint/ package (from evalstone fp_fusion v1.1, 2026-09-07 pruning final): probe battery -> concurrent collection -> five scoring views (verify/attribution/variant/adversarial/robustness), bundled family aliases + 27 reference fingerprints (12 fp_fusion schema). - CLI: 'evalharness fingerprint run ...' (REMAINDER passthrough, single source of arg definitions) + 'fingerprint list' for bundled references - imports rewritten package-relative; direct 'python3 run_fp_fusion.py' execution kept working via package bootstrap - offline analysis/collection scripts made path-independent (previously pinned to a /opt/evalscope path absent on this host) - shell scripts: hardcoded API key -> FP_API_KEY/OPENAI_API_KEY env vars - --reference accepts short names resolved against bundled references/ - pyproject: +httpx dependency, package-data references/*.json - tests/test_fingerprint.py: 10 offline tests (battery definitions, assembly counts, normalization, signals, verdict ladder, CLI wiring) - README: fingerprint section + architecture entry Verified on H20-1: tests 10/10, installed CLI OK, full-protocol run vs vectron GLM-5.3 reproduces baseline (score 0.9451, s_idn 0.846).
15 lines
592 B
Python
15 lines
592 B
Python
#!/usr/bin/env python3
|
|
"""Show reference summary and incomplete cells."""
|
|
import json
|
|
import sys
|
|
|
|
ref = json.load(open(sys.argv[1]))
|
|
print("protocol:", ref["protocol"], "| model:", ref["model"],
|
|
"| cells:", len(ref["cells"]), "| sppc:", ref["samplesPerCell"])
|
|
tv = sum(c["validCount"] for c in ref["cells"].values())
|
|
tt = sum(c["totalCount"] for c in ref["cells"].values())
|
|
print(f"valid {tv}/{tt}")
|
|
bad = [(cid, c["validCount"], c["totalCount"])
|
|
for cid, c in sorted(ref["cells"].items())
|
|
if c["validCount"] < c["totalCount"]]
|
|
print("incomplete:", bad if bad else "NONE") |