技能 数据科学 隐私政策配置与脱敏处理

隐私政策配置与脱敏处理

v20260803
configuring-privacy-policies
本技能用于配置多种符合法规的隐私保护策略,支持对敏感数据进行脱敏处理。用户可以根据HIPAA、GDPR、PIPEDA等不同地域和场景的法律要求,选择合适的策略,确保数据脱敏、替换或假名化,从而满足数据共享和研究的合规需求。
获取技能
400 次下载
概览

Configuring privacy policies

A policy profile is a named bundle of de-identification decisions: which action (mask/redact/replace/keep) applies to each label, how aggressively detectors arbitrate, whether the mandatory safety sweep runs, and whether a reversible mapping is produced. OpenMed ships seven profiles. Pass one by name to deidentify(policy=...) and you get a compliance-aligned default without hand-wiring 50+ per-label actions. Everything runs on-device.

When to use this skill

Use it to pick the right policy= for a regulatory context, to understand what a profile actually changes, or to go beyond the bundle — keeping quasi- identifiers for research, or registering a custom surrogate generator (e.g. your own MRN format).

Quick start

import openmed

note = "Jane Roe, DOB 1979-04-11, lives in Cambridge MA 02139. SSN 123-45-6789."

# HIPAA Safe Harbor: mask every identifier class.
safe = openmed.deidentify(note, policy="hipaa_safe_harbor")

# GDPR pseudonymization: replace with fakes AND keep a reversible mapping.
gdpr = openmed.deidentify(note, policy="gdpr_pseudonymization")
mapping = gdpr.mapping          # present because the profile sets keep_mapping=True

# Research limited dataset: mask direct identifiers, KEEP quasi-identifiers
# (dates, age, ZIP, geography) so the data stays analytically useful.
lds = openmed.deidentify(note, policy="research_limited_dataset")

The seven bundled profiles

Each profile lives in openmed/core/policies/<name>.json. Summary of what each actually configures:

Profile Default action Quasi-identifiers Mapping Safety sweep Use case
hipaa_safe_harbor mask all masked none mandatory HIPAA §164.514(b)(2) Safe Harbor — strip all 18 identifier classes
hipaa_expert_review_assist redact redacted; clinical concepts kept none optional Assist Expert Determination (§164.514(b)(1)); keeps microbiology/clinical terms for a statistician to assess residual risk
gdpr_pseudonymization replace replaced; clinical kept kept + reversible mandatory GDPR Art. 4(5) pseudonymization — reversible under controlled key
canada_pipeda replace (IDs masked) replaced kept + reversible mandatory PIPEDA-aligned; like GDPR but masks ID_NUM/SSN outright
research_limited_dataset mask direct ids keeps dates, age, ZIP, geography, org, job none mandatory HIPAA Limited Data Set (§164.514(e)) — usable for research with a DUA
clinical_minimal_redaction mask direct ids keeps quasi-identifiers none optional Internal clinical use where readability matters; lighter cascade
strict_no_leak mask everything masked; even clinical concepts masked none mandatory Maximum-recall, union arbitration, all cascade tiers — zero-leakage posture

Key dimensions to reason about:

  • default_actionmask ([NAME]), redact, replace (fake value), or keep. Set per label in the profile's actions map.
  • policy_label_actions — coarse action by class: DIRECT_IDENTIFIER / QUASI_IDENTIFIER / CLINICAL_CONCEPT. Research and minimal-redaction profiles keep quasi-identifiers; strict-no-leak masks even clinical concepts.
  • keep_mapping / reversible_id — only GDPR and PIPEDA produce a reversible mapping. Treat that mapping as PHI.
  • safety_sweep_mandatory — deterministic structured-ID sweep (SSN, MRN- like, emails) that runs regardless of model confidence. Off only for the two "minimal/assist" profiles.
  • arbitration_mode / forced_cascade_tiersstrict_no_leak uses high_recall_union across tiers R0–R3 (most aggressive); minimal redaction uses only R0–R1.

Choosing: map regulation → profile

  • Publish or share data with no DUA, UShipaa_safe_harbor.
  • Statistician will certify low risk (keep clinical signal)hipaa_expert_review_assist, then human Expert Determination.
  • EU subjects, need reversibility under a keygdpr_pseudonymization.
  • Canadian subjectscanada_pipeda.
  • Research cohort needing dates/age/geographyresearch_limited_dataset (requires a Data Use Agreement).
  • Internal clinical workflow, readability firstclinical_minimal_redaction.
  • Adversarial / zero-tolerance leakagestrict_no_leak.

Customizing beyond the bundle

When a profile is close but not exact, drive the engine directly with Anonymizer / AnonymizerConfig, or register custom generators.

from openmed import (
    Anonymizer, AnonymizerConfig,
    register_label_generator, register_clinical_provider,
)

# 1) Per-instance config (language, locale, deterministic surrogates):
anon = Anonymizer(AnonymizerConfig(lang="en", consistent=True, seed=7))
fake_name = anon.surrogate("John Doe", "PERSON")     # type-matched surrogate

# 2) Override the surrogate for one canonical label (e.g. your MRN format).
#    Generators take (faker, original, *, locale) and return a string.
def hospital_mrn(faker, original, *, locale):
    return f"H{faker.numerify('#######')}"

register_label_generator("ID_NUM", hospital_mrn)     # global, all new Anonymizers

# 3) Add a custom Faker provider (e.g. proprietary identifier formats).
register_clinical_provider(MyClinicalProvider)        # a faker BaseProvider class

Use register_label_generator(canonical_label, fn) to swap one label's surrogate; use register_clinical_provider(provider) to add whole Faker providers. For per-call scoping, pass providers via AnonymizerConfig.custom_providers instead of the global registry. Validate custom labels against openmed.CANONICAL_LABELS.

Hand-off to / from OpenMed

  • Apply a policy: openmed.deidentify(text, policy="<name>") — see deidentifying-clinical-text.
  • Surrogate strategy: generating-synthetic-surrogates for method="replace" with consistent/seed/locale and custom providers.
  • Verify coverage: auditing-deidentification-runs (audit=True) and auditing-safe-harbor-checklist (18 identifier categories).
  • Other surfaces: MCP openmed_deidentify and REST POST /pii/deidentify accept the same policy argument.

Edge cases & gotchas

  • Profiles are configuration, not a guarantee. A profile that keeps quasi- identifiers (research/minimal) does not meet Safe Harbor — pair it with a Data Use Agreement or Expert Determination.
  • Reversible profiles produce a re-identifying mapping. GDPR/PIPEDA mappings are as sensitive as the raw PHI; store them encrypted and separately.
  • register_label_generator is global and persists for the process. It mutates a shared registry; prefer AnonymizerConfig.custom_providers for isolated, per-run behavior.
  • Surrogates must not collide with real values. Keep generated identifiers out of the real ID space; see generating-synthetic-surrogates.
  • Permissive licensing only. Do not bundle UMLS/SNOMED/CPT/MIMIC/i2b2/n2c2 into custom providers; call restricted terminologies out-of-process.

Standards & references

信息
Category 数据科学
Name configuring-privacy-policies
版本 v20260803
大小 8.35KB
更新时间 2026-08-04
语言