Segmenting clinical sections

A clinical note is not flat text — it is a sequence of named sections (Chief
Complaint, HPI, Past Medical History, Medications, Allergies, Assessment & Plan).
The same phrase means different things in different sections: "diabetes" in
PMH is historical context, "diabetes" in Assessment & Plan is an active problem,
and "penicillin" under Allergies is an adverse-reaction flag, not a current
medication. Splitting the note into canonical sections before NER or
de-identification gives every downstream OpenMed step the context it needs to be
more precise — and lets you process sensitive sections under stricter policies.

When to use

• You have a free-text note, H&P, progress note, or discharge summary and are

about to run NER (extracting-clinical-entities) or de-identification.

• The user wants section detection, header parsing, LOINC section mapping, or

per-section processing (e.g. "redact the Social History section harder").

• Downstream NER is over- or under-firing because it can't tell historical PMH

mentions from active A&P problems.

Quick start

import re
import openmed

# Synthetic note.
note = """CHIEF COMPLAINT: chest pain.
HPI: 54M with 2 hours of substernal pressure.
PAST MEDICAL HISTORY: type 2 diabetes, prior MI 2019.
MEDICATIONS: metformin 500 mg BID.
ALLERGIES: penicillin (rash).
ASSESSMENT AND PLAN: acute coronary syndrome; start aspirin, admit."""

# Map common header variants -> canonical section + LOINC document-section code.
SECTION_MAP = {
    "chief complaint": ("Chief Complaint", "10154-3"),
    "hpi": ("History of Present Illness", "10164-2"),
    "history of present illness": ("History of Present Illness", "10164-2"),
    "past medical history": ("Past Medical History", "11348-0"),
    "medications": ("Medications", "10160-0"),
    "allergies": ("Allergies", "48765-2"),
    "assessment and plan": ("Assessment and Plan", "51847-2"),
}

HEADER_RE = re.compile(r"^(?P<h>[A-Z][A-Za-z /&]+):", re.MULTILINE)

# Split note into (canonical_label, loinc, body) chunks at each header.
chunks, matches = [], list(HEADER_RE.finditer(note))
for i, m in enumerate(matches):
    raw = m.group("h").strip().lower()
    label, loinc = SECTION_MAP.get(raw, (m.group("h").strip(), None))
    body_start = m.end()
    body_end = matches[i + 1].start() if i + 1 < len(matches) else len(note)
    chunks.append({"section": label, "loinc": loinc,
                   "text": note[body_start:body_end].strip()})

# Run NER per section — pass the section label downstream as context.
for c in chunks:
    ents = openmed.analyze_text(c["text"], model_name="disease_detection_superclinical",
                                output_format="dict")
    c["entities"] = ents

Each chunk now carries its canonical section label and LOINC code, so downstream
context resolution can treat PMH findings as historical and A&P findings as
active.

Workflow

1. Detect section headers. Use header heuristics: a line that is a known

header phrase, often uppercase, ending in a colon, at line start. Maintain a
synonym map (HPI ↔ History of Present Illness, PMH ↔ Past Medical History,
A&P ↔ Assessment and Plan) so variants normalize to one canonical label.

1. Normalize to canonical labels and LOINC codes. Map each detected header to

a canonical section name and a LOINC document-section code (e.g. HPI →
10164-2, PMH → 11348-0, Medications → 10160-0, Allergies → 48765-2,
A&P → 51847-2). Unknown headers keep their literal text and a null code.

1. Chunk the note into (section, loinc, body) spans between consecutive

headers, preserving original character offsets if you need to map results back.

1. Process per section. Run analyze_text / deidentify on each chunk and

carry the section label forward. This is where precision is won: section-aware
negation (PMH = historical) and section-specific de-id policy
(Social History / Family History often warrant stricter redaction).

1. Reassemble with provenance. Tag each downstream entity with its source

section so the problem-list and context layers can use it.

Hand-off to / from OpenMed

• To extracting-clinical-entities: feed each section chunk into

openmed.analyze_text and attach the section label to every entity — section
context measurably sharpens entity precision and downstream status assignment.

• To deidentifying-clinical-text: run openmed.deidentify per section so

high-risk sections (Social/Family History) can use a stricter policy profile
than the body.

• To resolving-clinical-context: the section label is a strong prior — PMH

biases temporality toward historical, A&P toward recent/active. Pass it as part
of the modifier window.

• To reconciling-problem-lists: section provenance (PMH vs. A&P) is a key

signal for active-vs-resolved reconciliation.

Edge cases & gotchas

• Header variants are endless. "PMHx," "Past Med Hx," "PMH/PSH," inline

headers without a colon, and run-on notes all appear. Keep the synonym map
data-driven and fall back gracefully to the literal header for unknowns.

• Don't drop unsectioned text. Notes often start with un-headed preamble or

have free text between sections. Capture it as an "unknown/other" chunk rather
than discarding it, or you lose entities.

• LOINC is a binding, not a parser. LOINC document-section codes label the

section; they do not detect it. Mapping is your responsibility and is
user-supplied terminology — do not bundle LOINC content; reference codes only.

• Preserve offsets if you will re-merge entities into the original note for

de-id; chunking loses position unless you track it.

• Local-first. All segmentation and per-section processing runs on-device.

Standards & references

• LOINC document-section codes (clinical document ontology):

https://loinc.org/

• Denny et al., SecTag — a clinical note section tagger:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2655775/

• HL7 C-CDA section templates (canonical clinical document sections):

https://www.hl7.org/ccdasearch/