A4 | Model Update Tampering

Where It Appears In The Edge AI Architecture

A4 sits in the trusted artifact path before runtime inference ever begins.

Tampered artifact loads The Jetson accepts a changed model artifact because the loader trusts path and filename instead of cryptographic provenance.

Threat Model

The attacker can alter or replace a model artifact before it reaches the Jetson runtime.

Attacker capability

The attacker may compromise a build workspace, artifact directory, transfer step, update share, deployment script, or weakly protected model registry. They do not need to reach the public inference endpoint if the update path is trusted without verification.

Security assumption being tested

The Jetson should only load model artifacts produced by the trusted GPU workstation pipeline and approved by integrity metadata. A4 tests whether filename, directory, or operator habit has accidentally become the trust anchor.

Assets at risk

Model weights, preprocessing graph, output labels, calibration metadata, TensorRT rebuild inputs, benchmark comparability, update history, and downstream decisions that trust Jetson predictions.

Out of scope

No real model poisoning instructions, no third-party registry abuse, no malware-bearing model formats, and no deployment to live devices. The included lab signs and verifies toy JSON artifacts only.

Attack Intuition

The model artifact is code-like operational state, not a passive data file.

A model update changes what the system believes. If an artifact named model.onnx is replaced after evaluation but before deployment, normal API tests can still pass: the route is reachable, TensorRT rebuilds an engine, and the Jetson returns predictions. The failure is semantic integrity. The loaded model is not the one you approved.

A4 is especially damaging because it persists across requests. Unlike a single adversarial input, a tampered update changes future behavior for every user until detected and rolled back. It can also poison research conclusions: Jetson and Zynq comparisons become meaningless if the Jetson model is not bit-for-bit tied to the evaluated artifact.

Safe framing: this page demonstrates artifact integrity only. The lab uses harmless JSON files and HMAC signatures to show how a loader should reject tampering. It does not train, modify, or deploy a real malicious model.

Technical Explanation

The vulnerable pattern is "load latest artifact"; the secure pattern is "verify then load exact artifact."

Vulnerable sequence

GPU workstation exports model.onnx and copies it to a transfer directory.
A deployment script on Jetson loads the path without checking a signed digest or provenance record.
A changed artifact keeps the same filename and becomes the runtime model.

Secure sequence

Build pipeline computes a digest over the exact artifact bytes and signs a manifest.
Jetson verifies manifest signature, digest, model version, allowed producer, and rollback policy.
Only verified artifacts are promoted to the active runtime slot.

Edge AI-specific detail

The model artifact may trigger secondary artifacts such as TensorRT engines, INT8 calibration caches, preprocessing configs, and class-label maps. D7 should cover the complete model bundle, not only one ONNX file.

A signed model pipeline also creates a useful measurement boundary. If an inference result was produced by model version v17, you should be able to prove which artifact hash, calibration file, label map, and deployment decision produced it.

Mathematical Formulation

A4 is an integrity failure in the mapping from approved artifact to deployed function.

Approved model: f_theta Deployed model: f_theta' Integrity objective: f_theta' == f_theta under artifact identity H(A) Artifact bundle: A = {weights, graph, preprocessing, labels, calibration, metadata} Manifest: M = {artifact_hash = H(A), version, producer, timestamp, policy} signature = Sign(sk_producer, H(M)) Secure load condition: Load(A, M, signature) = Verify(pk_producer, H(M), signature) * ConstantTimeEqual(H(A), M.artifact_hash) * VersionAllowed(M.version) * ProducerAllowed(M.producer) Conceptual backdoor behavior, for analysis only: f_theta'(x) = target_label when trigger_condition(x) is true f_theta'(x) = f_theta(x) otherwise

The final line is a conceptual risk model, not an implementation recipe. The practical defense is to make any unauthorized change to A fail before runtime by binding artifact bytes to a signed manifest and deployment policy.

Step-By-Step Safe Lab Demonstration

The lab shows how signed manifests change loader behavior for a toy artifact bundle.

Save the Python code from the next section as a4_model_update_integrity_lab.py.
Run python3 a4_model_update_integrity_lab.py --mode vulnerable. The vulnerable loader accepts a changed toy artifact.
Run python3 a4_model_update_integrity_lab.py --mode secure. The secure loader rejects the changed toy artifact because the digest no longer matches the signed manifest.
Review the generated log lines: artifact_loaded, artifact_rejected, digest prefix, signature status, and rollback selection.
Map the same logic to your real pipeline: ONNX file, TensorRT engine cache, label map, calibration table, model card, and deployment record.

Interactive toy replay

Replay the expected local behavior directly in the page. It is display-only and does not touch the filesystem or devices.

ready: no artifact simulation replayed yet

Full Code For A Local Simulated Lab

This code signs and verifies toy JSON artifacts. It never loads real ML models or contacts a network service.

a4_model_update_integrity_lab.py

#!/usr/bin/env python3
"""
A4 local-only simulator: model update tampering.

This lab uses harmless JSON artifacts and HMAC signatures to model the
integrity gate a Jetson deployment script should apply before loading a
model bundle. It does not train, modify, or execute a real ML model.
"""

import argparse
import hashlib
import hmac
import json
import shutil
import tempfile
from pathlib import Path

LAB_SECRET = b"local-demo-signing-key"


def canonical_json(document):
    return json.dumps(document, sort_keys=True, separators=(",", ":")).encode("utf-8")


def sha256_bytes(data):
    return hashlib.sha256(data).hexdigest()


def sign_bytes(data):
    return hmac.new(LAB_SECRET, data, hashlib.sha256).hexdigest()


def write_json(path, document):
    path.write_text(json.dumps(document, indent=2, sort_keys=True), encoding="utf-8")


def read_json(path):
    return json.loads(path.read_text(encoding="utf-8"))


def build_clean_bundle(root):
    bundle = {
        "kind": "toy-edge-classifier",
        "version": "v4.0-clean",
        "producer": "gpu-workstation",
        "labels": ["synthetic-cat", "synthetic-dog"],
        "weights_digest_hint": "demo-only-no-real-weights",
        "expected_backend": "jetson",
        "behavior_note": "approved synthetic baseline",
    }
    artifact_bytes = canonical_json(bundle)
    artifact_hash = sha256_bytes(artifact_bytes)
    manifest = {
        "artifact": "model_bundle.json",
        "artifact_hash": artifact_hash,
        "version": bundle["version"],
        "producer": bundle["producer"],
        "policy": "jetson-prod-allowlist",
    }
    manifest_bytes = canonical_json(manifest)
    signature = {"hmac_sha256": sign_bytes(manifest_bytes)}
    write_json(root / "model_bundle.json", bundle)
    write_json(root / "manifest.json", manifest)
    write_json(root / "manifest.sig", signature)
    return artifact_hash


def tamper_bundle(root):
    bundle_path = root / "model_bundle.json"
    bundle = read_json(bundle_path)
    bundle["version"] = "v4.0-replaced"
    bundle["behavior_note"] = "changed after approval"
    bundle["labels"] = ["synthetic-cat", "synthetic-dog", "unexpected-class"]
    write_json(bundle_path, bundle)
    return sha256_bytes(canonical_json(bundle))


def verify_bundle(root):
    artifact = read_json(root / "model_bundle.json")
    manifest = read_json(root / "manifest.json")
    signature = read_json(root / "manifest.sig")
    manifest_bytes = canonical_json(manifest)
    expected_sig = sign_bytes(manifest_bytes)
    signature_ok = hmac.compare_digest(signature["hmac_sha256"], expected_sig)
    artifact_hash = sha256_bytes(canonical_json(artifact))
    digest_ok = hmac.compare_digest(artifact_hash, manifest["artifact_hash"])
    producer_ok = manifest["producer"] == "gpu-workstation"
    version_ok = manifest["version"].startswith("v4.")
    return {
        "signature_ok": signature_ok,
        "digest_ok": digest_ok,
        "producer_ok": producer_ok,
        "version_ok": version_ok,
        "artifact_hash": artifact_hash,
        "manifest_hash": manifest["artifact_hash"],
        "artifact_version": artifact["version"],
        "manifest_version": manifest["version"],
    }


def vulnerable_loader(root):
    artifact = read_json(root / "model_bundle.json")
    print(
        "event=artifact_loaded mode=vulnerable version=%s labels=%s note=%s"
        % (artifact["version"], len(artifact["labels"]), artifact["behavior_note"])
    )
    return True


def secure_loader(root):
    result = verify_bundle(root)
    checks = [
        result["signature_ok"],
        result["digest_ok"],
        result["producer_ok"],
        result["version_ok"],
    ]
    if all(checks):
        print(
            "event=artifact_loaded mode=secure version=%s digest=%s"
            % (result["manifest_version"], result["artifact_hash"][:16])
        )
        return True
    print(
        "event=artifact_rejected mode=secure signature_ok=%s digest_ok=%s artifact_digest=%s manifest_digest=%s"
        % (
            result["signature_ok"],
            result["digest_ok"],
            result["artifact_hash"][:16],
            result["manifest_hash"][:16],
        )
    )
    print("event=rollback_selected slot=previous_verified_model reason=integrity_failure")
    return False


def run(mode):
    root = Path(tempfile.mkdtemp(prefix="a4_model_integrity_"))
    try:
        clean_hash = build_clean_bundle(root)
        tampered_hash = tamper_bundle(root)
        print("event=build clean_digest=%s tampered_digest=%s" % (clean_hash[:16], tampered_hash[:16]))
        if mode == "vulnerable":
            vulnerable_loader(root)
        else:
            secure_loader(root)
    finally:
        shutil.rmtree(root)


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument("--mode", choices=["vulnerable", "secure"], default="vulnerable")
    args = parser.parse_args()
    run(args.mode)


if __name__ == "__main__":
    main()

local simulator commands

# Vulnerable loader: accepts the changed toy artifact by path.
python3 a4_model_update_integrity_lab.py --mode vulnerable

# Secure loader: verifies signature, digest, producer, and version policy.
python3 a4_model_update_integrity_lab.py --mode secure

# Expected defensive result:
# event=artifact_rejected mode=secure signature_ok=True digest_ok=False ...
# event=rollback_selected slot=previous_verified_model reason=integrity_failure

Practical Example For Your GPU + Pi + Jetson + Zynq Setup

A4 is about preserving equivalence between what was evaluated and what is deployed.

Expected topology

The GPU workstation trains in PyTorch, exports ONNX, and produces model metadata. The Pi gateway or deployment script moves artifacts toward the private Jetson backend. The Jetson builds or loads TensorRT state. The Zynq path may receive a corresponding quantized or synthesized representation for comparison.

GPU factory Pi transfer gate Jetson runtime

A4 failure mode

The Jetson loads a replacement model.onnx or related bundle after evaluation has passed. The Pi still dispatches normally, users still receive predictions, and the private subnet still holds, but the model behavior has silently changed.

model.onnx drift TensorRT rebuild D7 missing

For your research portal, bind every performance result to artifact identity. A Jetson latency plot should include model hash, TensorRT engine hash, precision mode, calibration hash, label-map hash, and deployment signature status. FPGA comparisons should reference the corresponding signed bitstream path under D9.

Observable Signals Or Logs

A4 should create strong provenance logs before runtime inference begins.

Signal	Vulnerable observation	Hardened observation
Deployment log	Loaded model.onnx from path, no digest or signer recorded	artifact_verified version=v4 digest=... signer=gpu-workstation
Digest check	Missing or calculated after promotion only	Digest calculated before promotion and compared to signed manifest
Behavior canary	Baseline test set not tied to update gate	Canary accuracy, confidence drift, and label-map checks run before activation
Jetson trace	TensorRT engine rebuilt from unknown ONNX input	Engine cache keyed by verified ONNX hash and calibration hash
Rollback	Manual operator recovery after bad behavior is noticed	Automatic refusal and previous verified slot retained

Impact Analysis

A4 is primarily an integrity attack with persistent operational consequences.

Integrity

The deployed model may make different predictions from the approved model while preserving normal API behavior. If output drives decisions, those decisions inherit the tampered model's behavior.

Confidentiality

A tampered artifact can expose model metadata through altered outputs or, in unsafe serialization formats, increase host compromise risk. This page avoids executable model formats and treats such formats as a separate hardening topic.

Availability

Bad artifacts can fail TensorRT conversion, crash preprocessing assumptions, or trigger fallback loops that make the Jetson backend unavailable while the Pi continues dispatching work.

Mapping To CIA, STRIDE, PASTA, And MITRE ATLAS

A4 maps naturally to AI supply-chain and model-manipulation risk.

Framework	A4 mapping	Research interpretation
CIA	Integrity primary; Availability and Confidentiality secondary	Prove that only approved model bundles become runtime state and that bad updates fail closed.
STRIDE	Tampering, Elevation of Privilege, Repudiation, Denial of Service	An unauthorized actor changes a trusted artifact, gains influence over predictions, and may leave poor provenance evidence.
PASTA	Stage 2 technical scope, Stage 3 decomposition, Stage 4 threat analysis, Stage 5 vulnerability analysis, Stage 6 attack modeling	Decompose the model supply chain into build, sign, transfer, verify, promote, rollback, and runtime phases.
MITRE ATLAS	Relevant to AI Supply Chain Compromise, Manipulate AI Model, Poison AI Model, and Publish Poisoned Models concepts in ATLAS-style AI threat modeling.	A4 is the local lab version of a poisoned or replaced model artifact entering deployment.
MITRE ATT&CK	Related to supply-chain compromise and poisoned pipeline execution concepts.	Traditional CI/CD compromise can become AI-specific when the modified output is a model artifact rather than application code.

Defense Mapping To Existing D1-D11 Controls

D7 is the direct control; D9 is the FPGA counterpart for bitstreams.

Control	Role against A4	Validation
D7 Signed Model Artifacts	Primary control. Sign model bundles, verify digests and signer identity on Jetson before promotion, and retain a previous verified slot for rollback.	Tampered artifact is rejected before TensorRT rebuild or runtime activation.
D6 Sanitized Logging	Records digest, version, signer, policy decision, and rollback reason without exposing model internals or secrets.	Each deployment event has traceable artifact identity and a safe audit trail.
D1 JWT Auth + Object Checks	Protects artifact management APIs so only authorized maintainers can request deployment or rollback.	Unauthorized model update request returns 401 or 403 before file movement.
D4 Private Backend Subnet	Reduces direct access to Jetson update endpoints, but does not replace artifact verification.	Client VLAN cannot write to Jetson model directories or service ports.
D9 Signed Bitstream + PS Boundary	Provides the equivalent integrity principle for Zynq bitstreams and PS-to-PL trust boundaries.	Model and bitstream comparisons reference both D7 and D9 signatures.

Research Notes: What To Measure Experimentally

A4 is a strong candidate for reproducibility and provenance experiments.

Integrity gate effectiveness

Measure whether every altered byte in ONNX, label map, calibration, preprocessing config, or model card causes deployment refusal. Record false accepts and false rejects.

Deployment overhead

Measure digest time, signature verification time, manifest parsing time, TensorRT rebuild delay, and total update downtime on the Jetson.

Behavior drift

Run a fixed canary suite before and after each verified update. Track confidence distribution shift, class confusion changes, and backend-specific latency changes.

Rollback reliability

Measure how quickly the system returns to the previous verified model after a failed update and whether result logs preserve both attempted and active artifact identities.

Key Takeaways

A model update is a security event, not just a file copy.

A4 changes persistent model behavior by altering the artifact before runtime activation.
The Jetson should verify a signed manifest and exact artifact digest before loading or rebuilding TensorRT state.
The complete model bundle includes weights, graph, preprocessing, label map, calibration, metadata, and deployment policy.
Every benchmark and inference result should be attributable to a verified model artifact identity.