ruoxi_sun
|
09b2add673
|
fingerprint: integrate fp_fusion model fingerprint benchmark
New evalharness/fingerprint/ package (from evalstone fp_fusion v1.1,
2026-09-07 pruning final): probe battery -> concurrent collection ->
five scoring views (verify/attribution/variant/adversarial/robustness),
bundled family aliases + 27 reference fingerprints (12 fp_fusion schema).
- CLI: 'evalharness fingerprint run ...' (REMAINDER passthrough, single
source of arg definitions) + 'fingerprint list' for bundled references
- imports rewritten package-relative; direct 'python3 run_fp_fusion.py'
execution kept working via package bootstrap
- offline analysis/collection scripts made path-independent (previously
pinned to a /opt/evalscope path absent on this host)
- shell scripts: hardcoded API key -> FP_API_KEY/OPENAI_API_KEY env vars
- --reference accepts short names resolved against bundled references/
- pyproject: +httpx dependency, package-data references/*.json
- tests/test_fingerprint.py: 10 offline tests (battery definitions,
assembly counts, normalization, signals, verdict ladder, CLI wiring)
- README: fingerprint section + architecture entry
Verified on H20-1: tests 10/10, installed CLI OK, full-protocol run vs
vectron GLM-5.3 reproduces baseline (score 0.9451, s_idn 0.846).
|
2026-09-11 03:52:21 +00:00 |
|
