The Chi-squared (χ²) test is a statistical method used to determine whether there is a significant association between two categorical variables or whether observed data fits an expected distribution. In categorical data analysis, the chi-square test compares observed frequencies with expected frequencies under a given hypothesis.
For example, Entity 1: People’s favorite colors and Entity 2: Their preference for ice cream.
- Null Hypothesis (H₀): Favorite color and ice cream preference are independent (no relationship).
- Alternative Hypothesis (H₁): They are dependent (a relationship exists).
By comparing observed survey data with expected frequencies (if no relationship existed), the Chi-Square test calculates a test statistic (χ²). If this value is large enough, we reject H₀, concluding that color preference does influence ice cream choice and vice versa.
Formula For Chi-Square Test
\chi^2 = \sum \frac{ (O_i - E_i)² }{ E_i }
Symbols are broken down as follows:
- Oi: Observed frequency
- Ei: Expected frequency
Categorical Variables
Categorical variables classify data into distinct, non-numeric groups (e.g., colors, fruit types).
Key Characteristics:
- Distinct Groups: No overlap (e.g., hair color: blonde, brunette).
- Non-Numerical: No arithmetic meaning (e.g., "apple" ≠ "orange" numerically).
- Limited Options: Fixed categories (e.g., traffic lights: red, yellow, green).
Example: "Do you prefer tea, coffee, or juice?" → Categories: tea/coffee/juice.
Steps for Chi-Square Test
Step 1: Define Hypothesis
- Null Hypothesis (H₀): The variables are independent.
- Alternative Hypothesis (H₁): The variables are dependent.
Step 2: Gather and Organize Data
Collect data on two categorical variables:
Sex (male or female)
Favorite ice cream flavor (chocolate, vanilla, or strawberry)
Chocolate | Vanilla | Strawberry | Total | |
|---|---|---|---|---|
Male | 20 | 15 | 10 | 45 |
Female | 25 | 20 | 30 | 75 |
Total | 45 | 35 | 40 | 120 |
Step 3: Calculate Expected Frequencies
- Get Observed Frequency: In any specific cell, the expected frequency can be described as the number of occurrences that would be expected if the two variables were independent.
- Expected Frequency Calculation: This involves multiplying the sums of rows and columns in proportion, then dividing by the total number of observations in a table.
Observed frequency
E_{ij}=\frac{(Row Total)×(Column Total)}{Grand Total}
- Male and chocolate:
\frac{45×45}{120} = 16.875 - Male and Vanilla:
\frac{45×35}{120} = 13.125 - Male and Strawberry:
\frac{45×40}{120}=15.0 - Female and chocolate:
\frac{75×45}{120} = 28.125 - Female and Vanilla:
\frac{75×35}{120} = 21.875 - Female and Strawberry:
\frac{75×40}{120}=25.0
Step 4: Perform Chi-Square Test
Use Chi-Square Formula:
\chi^2 = \sum \frac{ (O_i - E_i)² }{ E_i }
Step 5: Determine Degrees of Freedom (df)
df = (number of rows - 1) × (number of columns - 1)
Step 6: Calculation of p-value
- If a predefined table is provided, use it
- Otherwise, we generally take the significance level as α = 0.05
General formula :
p = P\left(\chi^2_{df} \geq \chi^2_{\text{observed}}\right)
where ;
ν represents the degrees of freedom.
Calculation:
Here, χ² = 4.86 with df=2:p = e^{-\chi^2/2}
Substituting χ2 = 4.86 :p = e^{-4.86/2} \approx 0.088
Critical value at α = 0.05 is 5.991.
The p-value (0.088) is greater than 0.05; therefore, we fail to reject the null hypothesis, as the result is not statistically significant.
Step 7: Interpret Results
- If the p-value is less than a certain significance level (e.g., 0.05), then we reject the null hypothesis, which is commonly denoted by α. Thus, it means that category variables highly correlate with each other.
- When a p-value is above α, it implies that we cannot reject the null hypothesis; hence, there is insufficient evidence for establishing the relationship between these variables.
No significant evidence supports the claim that men prefer vanilla or women prefer chocolate (p>0.05).
Addressing Assumptions and Considerations
- Chi-square tests suppose that the observations are independent of one another; they are distinct.
- Each cell in the table should have a minimum of five values in it for better results. Otherwise, think about Fisher’s exact test as an alternative measure if a table cell has fewer than five numbers in it.
- Chi-square tests do not indicate a causal relationship, but they identify an association between variables.
Goodness - Of - Fit
A goodness-of-fit test checks if a hypothesized model matches observed data. For example, testing whether a die is fair.
Key Aspects:
- Purpose: Check how well observed data fits expected data
- Data Types: Categorical only
- Applications: Compare observed vs. expected frequencies.
- Benefits: Identifies model-data mismatch.
Applications of Chi-Square Test in Computer Science
A/B Testing & Feature Evaluation
- Compare user engagement (e.g., clicks, conversions) between two website versions (A vs. B).
- Chi-test is used to test if observed metrics (e.g., "Click" vs. "No Click") differ significantly between groups.
- Example: Observed: Version A: 120 clicks / 1,000 views; Version B: 150 clicks / 1,000 views. Chi-Square: Checks if the difference is statistically significant (not due to chance).
Machine Learning (Feature Selection)
- Identify categorical features correlated with target variables.
- Test if independence between features (e.g., "Browser Type" vs. "Purchase Decision") using the Chi-square test.
- Example: χ² p-value < 0.05 → "Browser Type" significantly affects purchases.
Database Query Optimization
- Assess if data is evenly distributed across partitions.
- Chi-square is used to test if actual row counts per partition match the expected uniform distribution.
- Example: Uneven distribution (χ² significance) suggests a poor sharding strategy.
Natural Language Processing (NLP)
- Evaluate word frequency distributions in texts.
- Compare observed word counts (e.g., "error" in logs) to the expected Poisson distribution.
- Example: Detects overused terms in spam emails (χ² highlights deviations from normal usage).