Data Analysis & Evidence (asq-data-analysis)
When to trigger
- You have data but the path from data to theory is opaque
- Qualitative: your quotes are decorative, not evidentiary; coding is undocumented
- Quantitative: main results exist but robustness/alternative explanations are thin
- Reviewers ask "how did you get from your data to these constructs?"
Branch A — Qualitative analysis (the data-to-theory link)
ASQ expects readers to see how raw data became theory — its guidelines stress that helping readers understand how the research was performed and ensuring the trustworthiness of published work are explicit aims (verify at journals.sagepub.com/author-instructions/asq). Qualitative rigor is judged on its own terms here, not held to a quantitative yardstick. Make the analytic ladder visible.
-
Transparent coding. Describe first-order codes (informant terms), second-order themes (researcher constructs), and aggregate dimensions — the Gioia-style data structure — or an equivalent (Eisenhardt cross-case, Langley process bracketing). State who coded, how disagreements were resolved, and how iteration proceeded.
-
Data-to-theory table. Provide a table linking representative raw evidence → codes → constructs, so the inference is auditable (see
asq-tables-figures).
-
Power quotes vs. proof quotes. Use a few vivid "power quotes" in the body; place corroborating "proof quotes" in tables/appendix. Quotes must carry the claim, not illustrate it after the fact.
-
Evidence for each construct. Every theoretical construct should be backed by patterned evidence across informants/cases, with counts or prevalence where appropriate.
-
Negative cases. Report disconfirming instances and how they refined the theory.
-
Process display. For process theory, show the temporal/event structure (timeline, phase model, visual mapping) — as Barley (1986, ASQ) did in tracing how CT scanners restructured radiology departments over time.
Branch B — Quantitative analysis
-
Main models match the design (FE/RE, event-history, multilevel, network models); report clearly with appropriate standard errors (clustering at the right level).
-
Robustness that targets the theory's threats: alternative measures, alternative samples, alternative specifications, endogeneity checks, and modern staggered-DiD diagnostics if relevant.
-
Mechanism evidence. Don't stop at the reduced-form relationship — provide mediation/moderation or supplementary tests that probe why.
-
Effect interpretation. Report and interpret magnitudes in organizational terms, not just significance stars.
-
Alternative explanations are tested, not waved away.
Either branch — the "so what" of the evidence
- Tie every analytic result back to the mechanism and the surprise.
- Distinguish what the data can and cannot establish — overclaiming is a fast path to rejection.
- Prepare the exhibits jointly with
asq-tables-figures.
Execution bridge (StatsPAI / Stata MCP)
Run the battery, don't just enumerate it. Full map:
execution-with-mcp. ASQ wants a clean causal or well-identified observational design behind an organizational-theory contribution; reduced-form estimation fits the chain below, interpretive work does not.
-
Many outcomes / specifications:
romano_wolf (step-down FWER) or
benjamini_hochberg — report the adjusted threshold.
-
OVB sensitivity:
oster_delta / sensemakr.
-
Inference:
wild_cluster_bootstrap (few clusters), twoway_cluster / conley;
multilevel data → cluster at the right level.
-
Re-fit off one handle:
audit_result(result_id) lists the missing checks and the
exact suggest_function for each.
-
Exhibits:
etable / did_summary_to_latex from the handle — no retyped numbers.
Keep the decisive checks in the body and the exhaustive battery in the appendix. See the
executed chain in the JF execution walkthrough.
Checklist
Anti-patterns
- "Anecdotal" qualitative work: a few cherry-picked quotes with no coding transparency
- Quotes that illustrate a pre-set conclusion rather than generating/supporting it
- Quantitative robustness theater: many tables that never address the real threat
- Reporting significance with no interpretation of organizational magnitude
- Stopping at the X→Y relationship without evidence on the mechanism
- Overclaiming causality or generalizability beyond the design
Output format
【Branch】qualitative / quantitative
【Data-to-theory link】data structure / mechanism tests done
【Key evidence】power quotes or main estimates
【Robustness/trustworthiness】checks completed + gaps
【What evidence cannot show】explicit limits
【Next step】asq-contribution-framing
