Collecting Quantitative Data

Based on Creswell, Educational Research
Chapter 5

Five Steps in the Data Collection Process

Five Steps in Quantitative Data Collection

1. Identify participants (unit of analysis, population, sample)
2. Obtain permissions (IRB, sites, participants, parents)
3. Decide what information to collect (variables, data types)
4. Locate, select, and assess instruments (reliability, validity)
5. Administer data collection (standardization, ethics)

The process involves more than simply gathering information; it includes interrelated steps.

Step 1: Identify Participants

Unit of Analysis

The level at which data needs to be gathered:

• Individual (students, teachers, parents)
• Family
• School
• School district

Example: In studying adolescent aggression's impact on school climate, measure aggression from individuals but school climate from entire schools.

Population and Sample

Key Definitions:

• Population: A group with the same characteristic (e.g., all teachers)
• Target Population (Sampling Frame): Group the researcher can identify and study
• Sample: Subgroup selected from the target population

Representative Sample: Individuals selected such that they are typical of the population, enabling generalizations.

Probability Sampling Methods

Most rigorous form - sample is representative of population:

• Simple Random Sampling: Every individual has equal probability of selection (use random numbers table)
• Systematic Sampling: Select every nth individual (e.g., every 5th person on list)
• Stratified Sampling: Divide population by characteristic, then sample from each subgroup proportionally
• Multistage Cluster Sampling: Sample in two or more stages for large populations

Nonprobability Sampling

Select individuals because they are available and convenient:

• Convenience Sampling: Participants who are willing and available
  • Cannot claim representativeness
  • Still provides useful information
• Snowball Sampling: Ask participants to identify others
  • Recruits large numbers
  • Don't know exact sample composition
  • May not be representative

Sample Size Guidelines

General rule: Select as large a sample as possible to minimize sampling error.

Rough Estimates:

• Experiments: ~15 participants per group
• Correlational Studies: ~30 participants
• Survey Studies: ~350 individuals (varies by factors)

Sampling Error: Difference between sample estimate and true population score. Larger samples reduce this error.

Step 2: Obtain Permissions

Required From:

• Institutions or organizations (school district)
• Specific sites (secondary school)
• Participants or group of participants
• Parents of participants (for minors)
• Campus Institutional Review Board (IRB)

Best practice: Request permission formally in a letter including purpose, time required, activities, benefits, and anonymity provisions.

Informed Consent

Participants must sign before participating. Form should include:

• Purpose of the study
• Procedures and time required
• Risks and benefits
• Right to withdraw at any time
• Voluntary participation
• Anonymity/confidentiality assurances
• Investigator contact information

Institutional Review Board (IRB)

Three Ethical Principles:

1. Respect for Persons: Autonomy, voluntary consent, right to privacy
2. Beneficence: Weigh benefits vs. risks; minimize harm
3. Justice: Equitable participant selection

Sensitive Populations (children, mentally incompetent, prisoners, pregnant women) require heightened review and protections.

Step 3: Types of Data to Collect

Four Major Types:

1. Performance Measures - Achievement, intelligence, aptitude tests
2. Attitudinal Measures - Surveys, Likert scales assessing feelings
3. Behavioral Observations - Checklists, direct recording of actions
4. Factual Information - Public records, attendance, grades

Performance Measures

Assess individual's ability on standardized tests:

• Achievement Tests: Iowa Test of Basic Skills (norm-referenced)
• Criterion-Referenced: GED Test, Metropolitan Achievement Test
• Intelligence Tests: Wechsler Intelligence Scale
• Aptitude Tests: Stanford-Binet IQ Scale
• Interest Inventories: Strong Interest Inventory
• Personality Assessment: Minnesota Multiphasic Personality Inventory

Attitudinal Measures

Measure feelings toward educational topics:

• Use unbiased questions
• Encourage honest answers
• Often use Likert scales

Example: "Student Adaptation to College Questionnaire" (SACQ) with 4 scales: Academic Adjustment, Social Adjustment, Emotional Adjustment, Goal Commitment

Limitation: Does not provide direct evidence of actual behavior.

Behavioral Observations

Record specific behaviors using checklists:

• Advantage: Measures actual behavior, not just perceptions
• Disadvantage: Time-consuming, difficult to score
• Requires observer training for consistency

Example: MIDI (Measurement of Inappropriate and Disruptive Interactions) - records context, inappropriate behavior type, and extent of misbehavior.

Factual Information

Numeric data from public records:

• Grade reports
• School attendance records
• Student demographic data
• Census information
• Suspension records

Note: Must scrutinize public documents carefully - availability does not guarantee accuracy. Some documents (health records) are protected by federal regulations.

Step 4: Locate & Assess Instruments

Three Options:

1. Use existing instrument (easiest)
2. Locate and modify (requires permission)
3. Develop your own (most difficult)

Search Strategies:

• Published journal articles
• ERIC database
• Mental Measurements Yearbook (MMY)
• Tests in Print (TIP)

Reliability

Scores are stable and consistent across administrations.

Types of Reliability:

• Test-Retest: Same test, different times (correlation should be .6 or higher)
• Alternate Forms: Two equivalent versions
• Interrater: Multiple observers score similarly
• Internal Consistency: Cronbach's alpha (for continuous variables)
• Split-Half: Kuder-Richardson (KR-20, KR-21)

Validity

Evidence that test scores measure intended construct for proposed use.

Five Sources of Validity Evidence:

1. Test Content: Expert review of items
2. Response Processes: Fit between construct and responses
3. Internal Structure: Statistical relationships among items
4. Relations to Other Variables: Correlations with external criteria
5. Consequences of Testing: Intended and unintended outcomes

Scales of Measurement

Categorical Scales:

• Nominal: Categories without order (gender, yes/no)
• Ordinal: Ranked order (1st, 2nd, 3rd; "highly important" to "not important")

Continuous Scales:

• Interval/Quasi-Interval: Equal distances (Likert scale: strongly agree to strongly disagree)
• Ratio: True zero + equal intervals (height, income)

Important: Scale type determines statistical test (parametric vs. nonparametric).

Step 5: Administer Data Collection

Standardization:

• Use uniform procedures for all participants
• Prevents bias and ensures comparability
• Train all data collectors
• Written procedures help maintain consistency

Ethical Practices:

• Protect anonymity (assign numbers)
• Maintain confidentiality
• Respect voluntary participation
• Minimize disruption to sites
• Allow withdrawal at any time

Key Takeaways

1. Quantitative data collection is a systematic 5-step process
2. Rigorous sampling strengthens generalizability
3. Ethical permissions and IRB approval are mandatory
4. Instrument quality requires reliability and validity evidence
5. Standardized administration ensures data integrity
6. Every decision must align with research questions

Remember: The process involves more than simply gathering information - it includes interrelated steps that build upon each other.

1 / 21

Share This