Quantitative Research Skill

Complete workflow for designing, executing, analyzing, and reporting quantitative research. Produces four separate deliverable files.

Workflow Overview

Phase 1: Research Design      -> RQs, hypotheses, variables, design
Phase 2: Sampling Plan        -> Target user profiles, sample size, power
Phase 3: Instrument Design    -> Questionnaire (Deliverable 1)
Phase 4: Data Collection      -> Dataset (Deliverable 2)
Phase 5: Statistical Analysis -> Analysis Results (Deliverable 3)
Phase 6: Report Writing       -> Research Report (Deliverable 4)

Four Deliverables (each exported as a separate file):
1. questionnaire.md — Complete structured questionnaire with scales
2. data.csv — Clean dataset with all variables
3. analysis_results.md — Full statistical analysis output
4. report.md — APA-style research report

Execute phases sequentially. Each phase feeds into the next.

Phase 1: Research Design

Goal: Transform a research topic into a testable experimental/survey design.

Steps

1. Define research questions: Convert broad topic into 1-3 specific, testable RQs
2. Develop hypotheses: For each RQ, state H1 (alternative) and H0 (null)
   - Include: main effects, interaction effects, mediation, moderation
3. Operationalize variables:
   - IV(s): name, levels, how manipulated/measured
   - DV(s): name, how measured, expected range
   - Mediators: name, measurement approach
   - Moderators: name, measurement approach
   - Controls: covariates to hold constant
4. Choose design:
   - Between-subjects (independent groups)
   - Within-subjects (repeated measures)
   - Mixed factorial
   - Survey/correlational
5. Plan analysis: Map each hypothesis to a specific statistical test

Design Selection Guide

Output

• Research design document (RQs, hypotheses, variables, design matrix)
• Analysis plan table (each hypothesis -> test -> criterion)

Phase 2: Sampling Plan

Goal: Define who to recruit and how many.

Steps

1. Define target population: Who has the characteristics needed?
2. Create user personas: 6-10 detailed profiles representing variation
3. Determine sample size:
   - Rule of thumb: 30+ per cell for between-subjects
   - Power analysis: G*Power for precise calculation (power=0.80, alpha=0.05)
   - Minimum total: n=200 for survey, n=6/cell for experiment
4. Plan randomization: How to assign participants to conditions?
5. Plan compensation: Incentive structure if applicable

Sample Size Guidelines

Output

• 6-10 participant personas
• Sample size justification (with power analysis)
• Randomization plan

Phase 3: Instrument Design

Goal: Create valid, reliable measurement tools. → Deliverable 1: questionnaire.md

Steps

1. Select scales: Use established scales when available (check Cronbach's alpha > 0.70)
2. Adapt scales: Modify existing scales to fit context, preserve core items
3. Create manipulation materials: Design stimuli for each experimental condition
4. Build questionnaire structure:
   - Section 1: Manipulation/instructions
   - Section 2: Manipulation check items
   - Section 3: DV measurement items
   - Section 4: Mediator items
   - Section 5: Moderator items
   - Section 6: Control variable items
   - Section 7: Demographics
5. Use Likert scales: 7-point strongly recommended (1-7), 5-point acceptable
6. Pilot test: Run with 20-30 participants, check for floor/ceiling effects

Scale Quality Checklist

• [ ] Each construct has 3-6 items
• [ ] Mix of positively and negatively worded items
• [ ] Reverse-coded items clearly marked
• [ ] Clear instructions for each section
• [ ] Estimated completion time < 15 minutes

Deliverable 1: questionnaire.md

Export a standalone questionnaire file containing:
- Study title and estimated completion time
- All experimental conditions with full text
- All scale items with source attribution
- All manipulation check items
- All demographic questions
- Randomization instructions

See references/templates.md for questionnaire template.

Phase 4: Data Collection

Goal: Obtain clean, complete data. → Deliverable 2: data.csv

Steps

1. Generate/simulate data: If conducting simulated research:
   - Set population means based on theory/prior research
   - Add realistic variance (SD 0.7-1.2 for 7-point scales)
   - Ensure manipulation checks show large effects (d > 2.0)
   - Build in hypothesized effects at expected magnitude
2. Real data collection: If fielding survey:
   - Use Qualtrics/SurveyMonkey/Google Forms
   - Set attention checks (e.g., "Select strongly agree")
   - Set completeness requirements
3. Clean data: Remove speeders, attention check failures, incomplete responses
4. Document exclusions: Report how many and why

Data Structure Standards

The CSV must include these columns:
- participant_id: Unique identifier (1 to N)
- Condition indicators: e.g., ai_use, transparency, originality
- Raw item scores: One column per item (e.g., CP1, CP2, CW1, AS1)
- Composite scores: Mean scores for each construct
- Demographics: gender, age, education, work_exp, industry, position, ai_freq

Data Simulation Standards

• Total n matches sample size plan
• Equal or near-equal cell sizes
• Manipulation checks show large effects (Cohen's d > 2.0)
• Realistic inter-item correlations (r = 0.30-0.70)
• Some missing data (5-10%) if realistic
• Demographics match target population distributions

Deliverable 2: data.csv

Export a clean CSV file with:
- Header row with variable names
- One row per participant
- All raw item scores and composite variables
- Condition assignment variables
- Demographic variables
- Missing values coded as NA

Phase 5: Statistical Analysis

Goal: Test all hypotheses rigorously. → Deliverable 3: analysis_results.md

Analysis Pipeline

Execute in this order:

Step 1: Descriptive Statistics
- Sample characteristics (demographics)
- Variable means, SDs, ranges
- Correlation matrix

Step 2: Manipulation Checks
- t-tests or ANOVA on manipulation check items
- Cohen's d for effect size
- Must be significant (p < .001) and large (d > 2.0)

Step 3: Reliability Analysis
- Cronbach's alpha for all multi-item scales
- Alpha > 0.70 acceptable, > 0.80 good, > 0.90 excellent

Step 4: Main Effect Tests
- Independent samples t-test (2 groups)
- One-way ANOVA (3+ groups)
- Report: M, SD, t/F, p, Cohen's d / eta-squared

Step 5: Interaction Tests
- Multi-way ANOVA for factorial designs
- Moderated regression for continuous moderators
- Report: F, p, eta-squared, simple effects

Step 6: Mediation Analysis
- Baron & Kenny 4-step method
- Bootstrap indirect effect (5000 samples)
- Report: indirect effect, 95% CI, proportion mediated

Step 7: Moderation Analysis
- Center predictor and moderator
- Compute interaction term
- Run moderated regression
- Report: beta, p, interaction plot

Step 8: Follow-up Tests
- Post-hoc comparisons (Tukey HSD, Bonferroni)
- Simple effects analysis for interactions
- Contrast analysis for planned comparisons

Deliverable 3: analysis_results.md

Export a standalone analysis file containing ALL statistical output:
- Section 1: Descriptive statistics tables
- Section 2: Manipulation check results (with effect sizes)
- Section 3: Reliability analysis (Cronbach's alpha table)
- Section 4: Main effects (t-tests/ANOVA tables)
- Section 5: Interaction effects (ANOVA tables + simple effects)
- Section 6: Mediation analysis (path coefficients + bootstrap CIs)
- Section 7: Moderation analysis (regression tables + interaction plots description)
- Section 8: Hypothesis testing summary (H1-Hn: supported/not supported)
- Section 9: Figures (bar charts, interaction plots, mediation diagrams)

See references/templates.md for analysis output table templates.

Phase 6: Report Writing

Goal: Produce APA-style report. → Deliverable 4: report.md

Report Structure

1. Abstract (150-250 words)
2. Introduction (2-3 pages) — literature review + hypotheses
3. Method (1-2 pages) — participants, design, materials, procedure
4. Results (2-4 pages) — key findings (refer to full analysis in Deliverable 3)
5. Discussion (2-3 pages) — contributions, implications, limitations
6. References
7. Appendix — note that full questionnaire, data, and analysis are separate files

Deliverable 4: report.md

Export a standalone APA-style report that:
- Presents key findings (not all raw output — refer to Deliverable 3)
- Interprets results in context of hypotheses
- Discusses theoretical and practical implications
- Acknowledges limitations
- Cites all four deliverable files in appendix

Four Deliverables Summary

All four files are independent deliverables. The report references but does not duplicate the questionnaire, data, or analysis results.

Key Principles

1. Design first, analyze second: Never change hypotheses after seeing data
2. Report everything: Report non-significant results too
3. Effect size is king: Always report effect sizes, not just p-values
4. Check assumptions: Normality, homogeneity, independence before parametric tests
5. Four files, four purposes: Questionnaire for replication, data for transparency, analysis for rigor, report for communication
6. Separate concerns: Each file has one job — do not duplicate content across files