The application of a one-way analysis of variance

11 ANALYSIS OF VARIANCE

CHAPTER LEARNING OBJECTIVES

1. Explain the application of a one-way analysis of variance (ANOVA) model.
2. Define the concepts of between and within total variance.
3. Calculate and interpret a test for two or more sample cases with means.
4. Determine the significance of an F-ratio test statistic.
5. Interpret output for ANOVA.

Many research questions require us to look at multiple samples or groups, at least more than two at a time. We may be interested in studying the relationship between ethnic identity (white, African American, Asian American, Latino) and church attendance, the influence of one’s social class (lower, working, middle, and upper) on President Donald Trump’s job approval ratings, or the effect of educational attainment (less than high school, high school graduate, some college, and college graduate) on household income. Note that each of these examples requires a comparison between multiple demographic or ethnic groups, more than the two-group comparisons we reviewed in Chapter 8. While it would be easy to confine our analyses between two groups, our social world is much more complex and diverse.

Let’s say that we’re interested in examining educational attainment—on average, how many years of education do Americans achieve? For 2018, the U.S. Census reported that 89.8% of adults (25 years and older) completed at least a high school degree, and 35% of all adults attained at least a bachelor’s degree.¹ Special attention has been paid to the educational achievement of Latino students. Data from the U.S. Census, as well as from the U.S. Department of Education, confirm that Latino students continue to have lower levels of educational achievement than other racial or ethnic groups.

In Chapter 8, we introduced statistical techniques to assess the difference between two sample means or proportions. In Table 8.2, we compared the difference in educational attainment for blacks and whites. But what if we wanted to examine more than two racial or ethnic groups?

Suppose we collect a random sample of 23 men and women, grouped them into four demographic categories, and included their educational attainment in Table 11.1. With the t-test statistic we discussed in Chapter 8, we could analyze only two samples at a time. We would have to analyze the mean educational attainment of whites and blacks, whites and Asians, whites and Latinos, blacks and Asians, and so on. (Confirm that we would have to analyze six different pairs.) In the end, we would have a series of t-test statistic calculations, and we still wouldn’t be able to answer our original question: Is there a difference in educational attainment among all four demographic groups?

Table 11.1 Educational Attainment (Measured in Years) for Four Racial/Ethnic Groups

Source: U.S. Census Bureau, Educational Attainment in the United States: 2018, Table 1.

LEARNING CHECK 11.1

Identify the independent and dependent variables in Table 11.1.

There is a statistical technique that will allow us to examine all the four groups or samples simultaneously. This technique is called analysis of variance (ANOVA). ANOVA follows the same five-step model of hypothesis testing that we used with t test for means and Z test for proportions (in Chapter 8) and chi-square (in Chapter 10). In this chapter, we review the calculations for ANOVA and discuss two applications of ANOVA from the research literature.

Analysis of variance (ANOVA): An inferential statistics technique designed to test for the significant relationship between two variables in two or more groups or samples.

UNDERSTANDING ANALYSIS OF VARIANCE

Recall that the t test examines the difference between two means, ¯¯¯Y1−¯¯¯Y2Y¯1−Y¯2 , while the null hypothesis assumed that there was no difference between them: μ₁ = μ₂. Rejecting the null hypothesis meant that there was a significant difference between the two mean scores (or the populations from which the samples were drawn).

The logic of ANOVA is the same but extending to two or more groups. For the data presented in Table 11.1, ANOVA will allow us to examine the variation among four means (¯¯¯Y1,¯¯¯Y2,¯¯¯Y3,¯¯¯Y4)(Y¯1,Y¯2,Y¯3,Y¯4) , and the null hypothesis can be stated as follows: μ₁ = μ₂ = μ₃ = μ₄. Rejecting the null hypothesis for ANOVA indicates that there is a significant variation among the four samples (or the four populations from which the samples were drawn) and that at least one of the sample means is significantly different from the others. In our example, it suggests that years of education (dependent variable) do vary by group membership (independent variable).

When ANOVA procedures are applied to data with one dependent and one independent variable, it is called a one-way ANOVA.
One-way ANOVA: Analysis of variance application with one dependent variable and one independent variable.

The means, standard deviations, and variances for the samples have been calculated and are shown in Table 11.2. Note that the four mean educational years are not identical, with Asians having the highest educational attainment. Also, based on the standard deviations, we can tell that the samples are relatively homogeneous with deviations within 1.51 to 2.07 years of the mean. We already know that there is a difference between the samples, but the question remains: Is this difference significant? Do the samples reflect a relationship between demographic group membership and educational attainment in the general population?

LEARNING CHECK 11.2

We’ve calculated the mean and standard deviation scores for each group in Table 11.2. Compute each mean (Chapter 3) and standard deviation (Chapter 4) and confirm that our statistics are correct.

To determine whether the differences are significant, ANOVA examines the differences between our four samples, as well as the differences within a single sample. The differences can also be referred to as variance or variation, which is why ANOVA is the analysis of variance. What is the difference between one sample’s mean score and the overall mean? What is the variation of individual scores within one sample? Are all the scores alike (no variation), or is there a broad variation in scores? ANOVA allows us to determine whether the variance between samples is larger than the variance within the samples. If the variance is larger between samples than the variance within samples, we know that educational attainment varies significantly across the samples. It would support the notion that group membership explains the variation in educational attainment.

Table 11.2 Years of Education, Means, Standard Deviations, and Variances for Four Racial/Ethnic Groups

THE STRUCTURE OF HYPOTHESIS TESTING WITH ANOVA

The Assumptions

ANOVA requires several assumptions regarding the method of sampling, the level of measurement, the shape of the population distribution, and the homogeneity of variance.

Independent random samples are used. Our choice of sample members from one population has no effect on the choice of sample members from the second, third, or fourth population. For example, the selection of Asians has no effect on the selection of any other sample.

The dependent variable, years of education, is an interval-ratio level of measurement. Some researchers also apply ANOVA to ordinal-level measurements.

The population is normally distributed. Although we cannot confirm whether the populations are normal, given that our N is so small, we must assume that the population is normally distributed to proceed with our analysis.

The population variances are equal. Based on our calculations in Table 11.2, we see that the sample variances, although not identical, are relatively homogeneous.²

Stating the Research and the Null Hypotheses and Setting Alpha

The research hypothesis (H₁) proposes that at least one of the means is different. We do not identify which one(s) will be different, or larger or smaller; we only predict that a difference does exist.

H₁: At least one mean is different from the others.

ANOVA is a test of the null hypothesis of no difference between any of the means. Since we’re working with four samples, we include four μ_s in our null hypothesis.

H₀: μ₁ = μ₂ = μ₃ = μ₄

As we did in other models of hypothesis testing, we’ll have to set our alpha. Alpha is the level of probability at which we’ll reject our null hypothesis. For this example, we’ll set alpha at .05.

The Concepts of Between and Within Total Variance

A word of caution before we proceed: Since we’re working with four different samples and a total of 23 respondents, we’ll have a lot of calculations. It’s important to be consistent with your notations (don’t mix up numbers for the different samples) and be careful with your calculations.

Our primary set of calculations has to do with the two types of variance: (1) between-group variance and (2) within-group variance. The estimate of each variance has two parts, the sum of squares and degrees of freedom (df).

The between-group sum of squares (SSB) measures the difference in average years of education between our four groups. Sum of squares is the abbreviation for “sum of squared deviations.” For SSB, what we’re measuring is the sum of squared deviations between each sample mean to the overall mean score. The formula for the SSB can be presented as follows:

Between-group sum of squares (SSB): The sum of squared deviations between each sample mean to the overall mean score.

(11.1)

SSB=∑nk(¯¯¯Yk−¯¯¯Y)2SSB=∑nk(Y¯k−Y¯)2

where

• n_k = the number of cases in a sample (k represents the number of different samples)
• ¯¯¯YkY¯k = the mean of a sample
• ¯¯¯YY¯ = the overall mean

SSB can also be understood as the amount of variation in the dependent variable (years of education) that can be attributed to or explained by the independent variable (the four demographic groups).

Within-group sum of squares (SSW) measures the variation of scores within a single sample or, as in our example, the variation in years of education within one group. SSW is also referred to as the amount of unexplained variance, since this is what remains after we consider the effect of the specified independent variable. The formula for SSW measures the sum of squared deviations within each group, between each individual score with its sample mean.

Within-group sum of squares (SSW): Sum of squared deviations within each group, calculated between each individual score and the sample mean.

(11.2)

SSW=∑(Yi−¯¯¯Yk)2SSW=∑(Yi−Y¯k)2

where

• Y_i = each individual score in a sample
• ¯¯¯YkY¯k = the mean of a sample

Even with our small sample size, if we were to use Formula 11.2, we’d have a tedious and cumbersome set of calculations. Instead, we suggest using the following computational formula for within-group variation or SSW:

(11.3)

SSW=∑Y2i−∑(∑Yk)2nkSSW=∑Yi2−∑(∑Yk)2nk

where

• Y2iYi2 = the squared scores from each sample
• ΣYkΣYk = the sum of the scores of each sample
• n_k = the number of cases in a sample

Together, the explained (SSB) and unexplained (SSW) variances compose the amount of total variation in scores. The total sum of squares (SST) can be represented by

Total sum of squares (SST): The total variation in scores, calculated by adding SSB (between-group sum of squares) and SSW (within-group sum of squares).

(11.4)

SST=∑(Yi−¯¯¯Y)2=SSB+SSWSST=∑(Yi−Y¯)2=SSB+SSW

where

• Y_i = each individual score
• ¯¯¯YY¯ = the overall mean

The second part of estimating the between-group and within-group variances is calculating the degrees of freedom. Degrees of freedom are also discussed in Chapters 8 and 10. For ANOVA, we have to calculate two degrees of freedom. For SSB, the degrees of freedom are determined by

df_b = k – 1 (11.5)

where k is the number of samples.

For SSW, the degrees of freedom are determined by

df_w = N – k (11.6)

where

• N = total number of cases
• k = number of samples

Finally, we can estimate the between-group variance by calculating mean square between. Simply stated, mean squares are averages computed by dividing each sum of squares by its corresponding degrees of freedom. Mean square between can be represented by

Mean square between: Sum of squares between divided by its corresponding degrees of freedom.

Mean square between = SSB / df_b (11.7)

and the within-group variance or mean square within can be represented by

Mean square within: Sum of squares within divided by its corresponding degrees of freedom.

Mean square within = SSW / df_w (11.8)

The F Statistic

Together, the mean square between (Formula 11.7) and mean square within (Formula 11.8) compose the F ratio obtained or F statistic. Developed by R. A. Fisher, the F statistic is the ratio of between-group variance to within-group variance and is determined by Formula 11.9:

F ratio or F statistic: The test statistic for ANOVA, calculated by the ratio of mean square to mean square within.

(11.9)

F=MeansquarebetweenMeansquarewithin=SSB/dfbSSW/dfwF=MeansquarebetweenMeansquarewithin=SSB/dfbSSW/dfw

We know that a larger F-obtained statistic means that there is more between-group variance than within-group variance, increasing the chances of rejecting our null hypothesis. In Table 11.3, we present additional calculations to compute F.

Table 11.3 Computational Worksheet for ANOVA

Let’s calculate between-group sum of squares and degrees of freedom based on Formulas 11.1 and 11.5. The calculation for SSB is

∑nk(Y−Y)2=6(13−13.96)2+5(13.6−13.96)2+6(16.67−13.96)2+6(12.5−13.96)2∑nk(Y−Y)2=6(13−13.96)2+5(13.6−13.96)2+6(16.67−13.96)2+6(12.5−13.96)2

= 5.53 + .65 + 44.06 + 12.79

= 63.03

The degrees of freedom for SSB is k – 1 or 4 – 1 = 3. Based on Formula 11.7, the mean square between is

Mean square between = 63.03 / 3 = 21.01

The within-group sum of squares and degrees of freedom are based on Formulas 11.3 and 11.6. The calculation for SSW is

∑Y2i−∑(ΣYk)2nk=(1,028+934+1,688+955)−(7826+6825+10026+7526)∑Yi2−∑(ΣYk)2nk=(1,028+934+1,688+955)−(7826+6825+10026+7526)

= 4,605 – (1,014 + 924.8 + 1,666.67 + 937.50)

= 4,605 – 4,542.97

= 62.03

The degrees of freedom for SSW is N – k = 23 – 4 = 19. Based on Formula 11.8, the mean square within is

Mean square within = 62.03 / 19 = 3.26

Finally, our calculation of F is based on Formula 11.9:

F ratio = 21.01 / 3.26 = 6.44

A Closer Look 11.1Decomposition of SST

According to Formula 11.4, sum of squares total (SST) is equal to

SST=∑(Yi−¯¯¯Y)2=SSB+SSWSST=∑(Yi−Y¯)2=SSB+SSW

You can see that the between sum of squares (explained variance) and within sum of squares (unexplained variance) account for the total variance (SST) in a particular dependent variable. How does that apply to a single case in our educational attainment example? Let’s take the first black respondent in Table 11.1 with 16 years of education.

This respondent’s total deviation (corresponding to SST) is based on the difference between the years of education from the overall mean (Formula 11.4). The individual mean is quite a bit higher than the overall mean education of 8 years. The difference of the individual from the overall mean is 2.04 years (16 – 13.96). Between-group deviation (corresponding to SSB) can be determined by measuring the difference between the group average from the overall mean (Formula 11.1). The deviation between the group average and overall average for blacks is –.36 years (13.60 – 13.96). Finally, the within-group deviation (corresponding to SSW, Formula 11.2) is based on the difference between the first black person’s years of education and the group average for blacks: 2.40 years (16 – 13.60). So for the first black person in our sample, SSB + SSW = SST or 2.40 + –.36 = 2.04. In a complete ANOVA problem, we’re computing these two sources of deviation (SSB and SSW) to obtain SST (Formula 11.4) for everyone in the sample.

Making a Decision

To determine the probability of calculating an F statistic of 6.44, we rely on Appendix E, the distribution of the F statistic. Appendix E lists the corresponding values of the F distribution for various degrees of freedom and two levels of significance, .05 and .01.

Since we set alpha at .05, we’ll refer to the table marked “p = .05.” Note that Appendix E includes two dfs. These refer to our degrees of freedom, df₁ = df_b and df₂ = df_w.

Because of the two degrees of freedom, we’ll have to determine the probability of our F obtained differently than we did with t test or chi-square. For this ANOVA example, we’ll have to determine the corresponding F, also called the F critical, when df_b = 3 and df_w = 19, and α = .05.

Based on Appendix E, the F critical (as in Appendix E) is 3.13, while our F obtained (the one that we calculated) is 6.44. Since our F obtained is greater than the F critical (6.44 > 3.13), we know that its probability is <.05, extending into the shaded area. (If our F obtained was <3.13, we could determine that its probability was greater than our alpha of .05, in the unshaded area of the F-distribution curve. Refer to Figure 11.1.) We can reject the null hypothesis of no difference and conclude that there is a significant difference in educational attainment between the four groups.

F critical: F-test statistic that corresponds to the alpha level, df_w, and df_b.

F obtained: The F-test statistic that is calculated.

THE FIVE STEPS IN HYPOTHESIS TESTING: A SUMMARY

To summarize, we’ve calculated an ANOVA test examining the difference between four demographic groups and their average years of education.

1. Making Assumptions:
• Independent random samples are used.
• The dependent variable, years of education, is an interval-ratio level of measurement.
• The population is normally distributed.
• The population variances are equal.
2. Stating the Research and Null Hypothesis and Selecting Alpha:
• H₁: At least one mean is different from the others.

H₀: μ₁ = μ₂ = μ₃ = μ₄

α = .05

Description

Figure 11.1 Comparing F Obtained Versus F Critical

1. 
   Selecting the Sampling Distribution and Specifying the Test Statistic: The F distribution and F statistic are used to test the significance of the difference between the four sample means.
2. Computing the Test Statistic: We need to calculate the between-group and within-group variation (sum of squares and degrees of freedom). We estimate SSB = 21.01 (df_b = 3) and SSW = 3.26 (df_w = 19).

Based on Formula 11.9,

F = 21.01 / 3.26 = 6.44

1. Making a Decision and Interpreting the Results: We reject the null hypothesis of no difference. Our F obtained of 6.44 is greater than the F critical of 3.13. The probability of 6.44 is <.05. F doesn’t advise us about which groups are different, only that educational attainment does differ significantly by demographic group members. Based on the sample data, we know that the only group to achieve a college education average was Asians (16.67 years). If we were to rank the other group means, second highest educational attainment was among blacks (13.60), followed by whites (13.00) and Latinos (12.50).

LEARNING CHECK 11.3

If alpha were changed to .01, would our final decision change?

STATISTICS IN PRACTICE: THE ETHICAL CONSUMER

Consumers increasingly avoid products that are perceived as having a negative social or environmental impact. By purchasing fair labor, organic, or environmentally safe products, consumers express their politics through their purchases, a practice that has been referred to as ethical consumerism. The GSS 2014 included a series of questions on what it takes to be a good citizen. Respondents were asked how important it was to “choose products for political, ethical or environmental reasons, even if they cost a bit more.” Answers were ranked on a 7-point scale: 1 = not at all important to 7 = very important. We use SPSS and the GSS 2014 data to examine the relationship between responses to this question and political party identification (Democrat, Independent, or Republican). We will set alpha at .05 to assess our results. SPSS output is presented as Figure 11.2.

The ANOVA output includes two tables: (1) Descriptives and (2) ANOVA. In the Descriptives table, the N, mean, and standard deviation are reported for each group and the entire sample, along with the 95% confidence interval for each mean.

The F obtained is reported in the ANOVA table, along with its level of significance (or probability). For these data, F obtained is 1.766 with a significance of .172. Since the level of significance is greater than our alpha (.172 > .05), we fail to reject the null hypothesis. Although the means are different (Democrats had the highest mean of 5.11), these differences are not significant.

Description

Figure 11.2 SPSS ANOVA Output: Political Party Identification and Choosing Products for Political Reasons

A Closer Look 11.2Assessing the Relationship Between Variables

Based on our five-step model of F, we’ve determined that there is a significant difference between the four demographic groups in their educational attainment. We rejected the null hypothesis and concluded that the years of education (our dependent variable) do vary by group membership (our independent variable). But can we say anything about how strong the relationship is between the variables?

The correlation ratio or eta square (η²) allows us to make a statement about the strength of the relationship or the effect size. Eta square is determined by the following:

(11.10)

η2=SSBSSTη2=SSBSST

The ratio of SSB to SST (SSB + SSW) represents the proportion of variance that is explained by the group (or independent) variable. Eta square indicates the strength of the relationship between the independent and dependent variables, ranging in value from 0 to 1.0. As eta square approaches 0, the relationship between the variables is weaker, and as eta square approaches 1, the relationship between the variables is stronger.

Based on our ANOVA example,

η2=63.0363.03+62.03=.50η2=63.0363.03+62.03=.50

We can state that 50% of the variation in educational attainment can be attributed to demographic group membership. Or we can say that 50% of the variation in the dependent variable (educational attainment) can be explained by the independent variable (group membership). So how strong is this relationship? We can base our determination of the strength on A Closer Look 10.2 from Chapter 10, the same scale that we used to assess gamma. We conclude that there is a very strong relationship between group membership and educational attainment.

LEARNING CHECK 11.4

Calculate eta square for the model presented in Figure 11.2.

READING THE RESEARCH LITERATURE: COLLEGE SATISFACTION AMONG LATINO STUDENTS

Like bivariate, t test, or chi-square analyses, ANOVA can help us understand how the categories of experience—race, age, class, and/or gender—shape our social lives. ANOVA allows us to investigate a variety of social categories by comparing the differences between them. We conclude this chapter with an example of how ANOVA is presented and interpreted in the social science literature.

Young Kim, Liz Rennick, and Marla Franco (2014)³ investigated the levels of college engagement and satisfaction among Latino undergraduate students attending highly selective colleges and universities in comparison with other racial/ethnic groups. The Latino student population is often characterized as academically unprepared for college; however, recent research has indicated that Latino students demonstrate a stronger drive to success and a higher level of academic effort compared to their peers. The 2014 research by Kim et al. is among the few studies to consider the experience of Latino students at selective colleges and universities.

Their research is based on the 2010 University of California Undergraduate Experience Survey. Kim and their colleagues limited their analyses to the outcomes of 33,415 junior and senior students. All factors were measured on a 6-point Likert-type scale: 1 = very poor to 6 = excellent. Selected results are presented in Table 11.4.⁴

The table presents ANOVA results for four models regarding the student satisfaction with the educational experience. The mean and standard deviation for each racial/ethnic group are reported in the table along with the obtained F ratio. The sample sizes for each racial/ethnic student group are not reported. All models are significant at the .001 level. The researchers briefly summarize their findings:

In terms of satisfaction with the educational experience, Latino students rated their satisfaction with quality of instruction and courses in the major, advising and out-of-class contact, and library support as equal to or higher than all other racial/ethnic groups. Sense of belonging and overall college satisfaction was the only area where White students reported higher levels of satisfaction than Latino students. (p. 253)⁵

Table 11.4 Patterns of College Experiences by Student Race/Ethnicity

Source: Adapted from Young Kim, Liz Rennick, and Marla Franco, “Latino College Students at Highly Selective Institutions: A Comparison of Their College Experiences and Outcomes to Other Racial/Ethnic Groups,” Journal of Hispanic Higher Education 13, no. 4 (2014): 254.

*** p < .001.

They state, “We may argue that the college experiences of Latino students at highly selective research institutions are favorable and meaningful, which may contribute to development in desired educational outcomes at these types of institutions” (p. 258).

LEARNING CHECK 11.5

For the “Sense of Belonging and Satisfaction” ANOVA model, what additional information would you need to determine the F critical?

Data at Work
Kevin Hemminger: Sales Support Manager/ Graduate Program in Research Methods and Statistics

Photo courtesy of Kevin Hemminger

While working as a sales support manager for a travel management company, Kevin decided to return to school to pursue a degree in criminal justice. Although originally he intended to become a probation officer for the juvenile court system, his career trajectory went into a different direction after taking a statistics course. “Due in great part to an incredible professor who made the subject enjoyable and interesting, I decided to pursue a PhD in Research Methods and Statistics in order to teach research methods and statistics. . . . My hope is to teach on either the graduate level, or to gifted high school students preparing for college.”

Kevin has incorporated what he’s learned about methods and statistics into his sales support work. “Being a very customer-focused company in a very customer-centric field, the idea of surveying customers is critical and being able to incorporate a knowledgeable administration of surveys has been integral to my team’s success. Knowing research best practices and understanding what constitutes good, valid research has made a tremendous impact on the value of the data I provide to my leaders. My knowledge in the field has also given me a confidence level in speaking about quantitative data that I never had before. I work with enormous data sets and spreadsheets on a daily basis and understanding even the most basic concepts such as mean, median, mode, standard deviation, and variance has allowed me to lead projects and present findings in a much more intelligent way.”

He recently completed his master’s degree and is currently working on his doctorate. Kevin offers these fine words of encouragement for anyone considering data and statistical work: “If you’re thinking about pursuing a career involving statistics, my recommendation is to dive full throttle into a study or project you’re incredibly passionate about. Watching your study move from an idea, to a literature review, to a hypothesis, data collection, and then data analysis is one of the best ways to confirm your interest—and, by far, the best way to experience the power behind statistics as a discipline. If you get excited watching your data come in, and even more excited wondering [what] you’re going to do with that data, I think you’ve confirmed there is no question you’d enjoy the field! If you’re learning stats concepts and all you can think about is how you’d teach that concept to others—it’s over! You’re perfect for the field.”

Main Points

• Analysis of variance (ANOVA) procedures allows us to examine the variation in means in more than two samples. To determine whether the difference in mean scores is significant, ANOVA examines the differences between multiple samples, as well as the differences within a single sample.
• One-way ANOVA is a procedure using one dependent variable and one independent variable. The five-step hypothesis testing model is applied to one-way ANOVA.
• The test statistic for ANOVA is F. The F statistic is the ratio of between-group variance to within-group variance.

KEY TERMS

analysis of variance (ANOVA) 374

between-group sum of squares (SSB) 377

F critical 381

F obtained 381

F ratio or F statistic 378

mean square between 378

mean square within 378

one-way ANOVA 374

total sum of squares (SST) 377

within-group sum of squares (SSW) 377

Digital Resources

Get the tools you need to sharpen your study skills. SAGE Edge offers a robust online environment featuring an impressive array of free tools and resources. Access practice quizzes, eFlashcards, video, and multimedia at edge.sagepub.com/frankfort9e.

SPSS Demonstration [GSS18SSDS-A]

Computing Analysis of Variance Models