section name header

Learning Objectives ⬇

This chapter is designed to help you understand the purpose of advanced statistical tests of differences between means (difference tests), when they may be used, and how to interpret the results. Because these advanced techniques require specialized skills to carry out, you may want to consult with a statistician when using these approaches. This chapter will prepare you to:

Key Terms ⬆ ⬇

Introduction ⬆ ⬇

We have discussed tests of mean difference such as t-tests and analyses of variance (ANOVA) that may be used when there is one categorical independent variable (the grouping variable) and one continuous dependent variable. However, in practice, you may be called upon to interpret findings from studies in which there is more than one independent or dependent variable, or you may be interested in analyzing more than one independent and/or dependent variable. For example, starting smoking is associated with several factors, including exposure to smoking in a family environment, access to tobacco products, and cognitive capacity, which means there are three independent variables: exposure, access, and cognition. Similarly, we may believe that interventions to help people quit smoking must also help them maintain their current weight—so there are two dependent variables, smoking cessation and weight. We may also be interested in differences over multiple time points or multiple dependent and independent variables simultaneously.

CASE STUDY
Data from Keating, S. R., & McCurry, M. (2019). Text messaging as an intervention for weight loss in emerging adults. Journal of the American Association of Nurse Practitioners, 31(9), 527μ536.

Overweight and obesity in young adults constitute a growing healthcare concern because it is associated with increased chronic disease, healthcare costs, and poorer quality of life. Keating and McCurry (2019) reported findings of a study to assess the effects of a text-messaging intervention on weight loss motivation, stages of change, and body mass index (BMI). The researchers randomly assigned participants to either the control group that received online access to a weight loss information site or the intervention group that received the same site access plus daily weight loss text messages. Motivation, change readiness, and BMI were self-reported and collected at baseline, 4 weeks, and 8 weeks.

Of interest to our discussion, Keating and McCurry (2019) used a repeated measures analysis of variance (ANOVA) to examine group differences in motivation level, stages of change, and BMI over the three points in time. They discovered no interaction effect between group (intervention and control) and weight loss motivation, stages of change, and BMI, nor was there any difference between the control and intervention groups. However, the repeated measures ANOVA revealed that in both groups there was an increase in weight loss motivation (F (2, 92) = 12.20, p = .000, partial eta squared = 0.117) and stages of change (F (2, 92) = 10.59, p = .000, partial eta squared = 0.103), and a decrease in BMI over time (F (1.72, 93) = 26.36, p = .000, partial eta squared = 0.221). Although the results suggest that the text-messaging intervention was no more or less effective than access to the online weight loss information site, we should carefully interpret these results. Recall that both groups reported a change in weight loss motivation, stages of change, and BMI over time. It may be that a more robust intervention with more or other resources would be effective. Repeating this novel study with a larger group, in different demographic groups, at a different time of year, or direct measurement of BMI could yield stronger results.

In these cases, you may run separate ANOVAs for each combination of variables, but you again run into the problem of inflated Type I error that we discussed in the previous chapter. A better alternative is to choose a more advanced ANOVA that allows you to test multiple hypotheses simultaneously and test for interaction effects among multiple independent variables. These tests require that dependent variables be continuous or measured at the interval or ratio level, while independent variable(s) must be categorical. In this chapter, we will discuss several advanced statistical tests that compare group means of continuous variable(s).

The Keating and McCurry (2019) study discussed here is a good example of the use of advanced ANOVA techniques. Nurses are often interested in complex human behaviors and other phenomena that cannot be explained or predicted by a single variable. Advanced ANOVAs make it possible to examine the complexity using a powerful test.

Choosing the Right Statistical Test ⬆ ⬇

Choosing the right statistical test depends upon the proposed research questions and consideration of factors such as the number of variables and the level of measurement for both independent and dependent variables. Elsewhere, we worked on examples where there was only one independent variable. However, we are often interested in research that examines the effect of more than one independent variable on the dependent variable, and the addition of independent variables beyond two interjects a new consideration—interaction effects. For example, you may be interested in investigating the effect of race (measured in three groups of White, Black, or Asian) as well as gender (male, female, or transgender) on exercise. When there is more than one independent variable in group comparison tests, you can examine potential interaction effects between/among these independent variables, as they may work together to create group differences. For example, Asian men may have more negative attitudes toward exercise than White women, or vice versa. We will discuss interaction effects in greater detail in a later section.

The number of dependent variables will also influence the selection of tests. When group differences on the mean are examined for a single continuous (measured at the interval or ratio level) dependent variable, the design is called univariate, and tests such as analysis of covariance (ANCOVA), repeated measures analysis of variance, and factorial analysis of variance can be used. However, when there are two or more continuous dependent variables, the design is called multivariate, and tests such as multivariate analysis of variance (MANOVA) or multivariate analysis of covariance (MANCOVA) should be used to answer the proposed research question. You can differentiate between univariate and multivariate tests because the number of dependent variables is usually reported in an article or clearly stated in a hypothesis.

Ancova ⬆ ⬇

In a previous chapter, we saw how one-way ANOVA allowed us to examine group differences on a continuous dependent variable. However, in many studies, there may be another continuous variable that may affect the dependent variable, but is not a variable of interest. These variables are known as covariates and can be included in ANOVA to control for their effect in order to examine the true influence of an independent variable on the dependent variable. For example, we may still be interested in examining the effect of the amount of exercise on a health problem index, but we know that body weight will also affect the health problem index score. We can get a better understanding of the influence of weight if we measure the weight and enter it as a covariate in an ANCOVA design. Such a procedure will allow us to control for the effect of weight and discern the true effect of the amount of exercise on the health problem index.

When we identify covariates that influence the dependent variable and carefully control for them in an ANCOVA, we achieve an important goal: we can explain the variability that we could not explain without the covariates. The unexplained variability is reduced, and the design allows us to examine more accurately the true effect of an independent variable. Similar to a regression analysis, ANCOVA allows us to compute the percentage of variance attributed to the independent variable and the covariates—this process is called “partitioning.” The partitioning of the variability is shown in Figure 12-1.

Partitioning of variability in analysis of covariance (ANCOVA).

A diagram shows the total variability in ANCOVA, analysis of covariance, partitioned into Variability explained by I V, Variability explained by covariate, and Variability that cannot be explained.

Assumptions

ANCOVA has all of the assumptions of ANOVA, plus an additional two assumptions. These are (1) independence between the covariate and independent variable and (2) homogeneity of regression slope.

Independence between the covariate and independent variable makes sense as we try to reduce the variability that is not explained by the independent variable by explaining it with the covariates. If the covariate and the independent variable are dependent (overlapping), the variability that is computed will be difficult to interpret and it will be unclear whether it was explained by the independent variable.

Homogeneity of regression slope means that the relationship between the covariate and the dependent variable stays the same. For example, the health problem index should increase as the weight increases across all the workout groups.

Doing and Interpreting ANCOVA

First, we need to set up hypotheses. These look similar to those of ANOVA design, except the means are adjusted for the covariate:

H0: There is no difference among group means after adjusting for the covariate.

Ha: At least two group means differ after adjusting for the covariate.

or

H0: μ1 = μ2 = μ3 after adjusting for the covariate.

Ha: μj≠μk for some j and k after adjusting for the covariate.

To conduct ANCOVA in IBM SPSS Statistics software (SPSS), you will open ExerciseCov.sav and go to Analyze > Generalized linear models > Univariate, as shown in Figure 12-2. The data here are those we used in the multiple comparison example where the amount of exercise affected the health problem index. In the Univariate dialogue box, you will move the independent variable, Exercise, into “Fixed Factor(s)”; the dependent variable, Health, into “Dependent Variable”; and the covariate, Weight, into “Covariate(s)” by clicking the corresponding “arrow” buttons in the middle, as shown in Figure 12-3. There are six buttons in the box, but only the commonly used buttons are discussed here. The “Contrasts” and “Post Hoc” buttons are used when further investigations are needed among more than two groups in the factor with ANOVA results indicating substantial differences in group means, as discussed in the earlier section on planned contrasts and post hoc tests. The “Options” button gives us several choices that may help us interpret the results of the ANCOVA, and these are shown in Figure 12-4. Please review the example output shown in Table 12-1.

Selecting ANCOVA under “General Linear Models” in SPSS.

A screenshot in S P S S shows the selection of the Analyze menu, with General linear model command chosen, from which the Univariate option is selected. The data in the worksheet shows columns of numerical data.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Defining variables in ANCOVA in SPSS.

A screenshot in S P S S Editor defines variables in ANCOVA. The numerical data in the worksheet lists Exercise, Health, and Weight values.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Useful options for ANCOVA in SPSS.

A screenshot in S P S S Editor shows the Univariate Options dialog box. The numerical data in the worksheet lists Exercise, Health, and Weight values.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation. Courtesy of IBM SPSS Statistics.

Table 12-1 Example Output of ANCOVA in SPSS

Between-Subjects Factors

Value Label

N

Amount of exercise

1

None

20

2

1 day per week

20

3

3 days per week

20

4

5 days per week

20

Tests of Between-Subjects Effects Dependent Variable: Health Index

Source

Type III Sum of Squares

df

Mean Square

F

Sig.

Corrected model

169.937a

4

42.484

1.645

.172

Intercept

112.647

1

112.647

4.363

.040

Weight

143.437

1

143.437

5.555

.021

Exercise

163.406

3

54.469

2.110

.106

Error

1,936.446

75

25.819

Total

3,740.195

80

Corrected total

2,106.382

79

aR-square (R2) = .081 (adjusted R2 = .032)

You will see that the ANCOVA results table is very similar to that of a one-way ANOVA, but there is an additional line representing the covariate—weight. The result indicates that weight substantially influences the health problem index with a small p-value of .007. However, you will find it interesting that the amount of exercise does not have a substantial effect on the health problem index with the covariate in the design. Do you remember that the amount of exercise made a substantial difference on average health problem index value in one-way ANOVA? When we add the covariate, weight, to our analysis, we find that weight is substantially influencing the health problem index and cancelling out the effect of exercise. If we had neglected to include the covariate of weight, we would have misinterpreted the influence of exercise on the health problem index.

Reporting ANCOVA

When reporting ANCOVA results, you should report the size of the F-statistics, along with associated degrees of freedom and associated p-value. For an effect size for ANCOVA, omega squared (ω2) can be calculated as it is with one-way ANOVA. Here, ω2 is only useful when the sample sizes for all groups are the same. Another type of effect size, partial eta squared (η2), is useful when sample sizes are not equal, and we can compute this in SPSS.

The following is a sample report from our example:

The covariate, weight, was substantially influencing this sample’s health problem index, F (1, 75) = 5.55, p = .021. However, the amount of exercise per week did not influence the health problem index after controlling for the effect of weight, F (3, 75) = 2.11, p = .106, ω2 = .04.

Further group comparisons of the amount of exercise per week are not considered, as the overall test did not find substantial differences on the average health problem index.

Factorial Analaysis of Variance (Anova) ⬆ ⬇

One-way ANOVA allows us to examine the effect of one independent grouping variable on a dependent variable, which is why the design is called one-way ANOVA. However, it is often the case that an investigator wants to examine the effect of more than one independent grouping variable on a dependent variable. For example, we may be interested in examining the effect of both the amount of exercise and the number of servings of soy milk per day on the health problem index. In this case, factorial ANOVA would be a good choice to examine the joint effect of the two independent variables. The design is called a two-way factorial ANOVA when there are two independent variables. You can see that with large sample sizes, there is the potential for three-, four-, and five-way factorial ANOVA, but this text will only consider two-way factorial ANOVA. In factorial ANOVA, each level of each factor (independent variable) is crossed with each level of another factor so that we can examine the interaction effect between factors/independent variables. In our present example, we would examine the mean differences between one serving of soy milk per day and no exercise per week, one serving of soy milk and 1 day of exercise per week, one serving of soy milk and 3 days of exercise per week, then two servings of soy milk and 1 day of exercise per week, and so on (you get the idea). (See Table 12-2.) An interaction effect is said to be present when the differences explained by one factor depend upon those by the other factor.

Table 12-2 Example Permutations of a Two-Way Factorial ANOVA

Factor A (Amount of Exercise)

None (A1)

1 Day per Week (A2)

3 Days per Week (A3)

5 Days per Week (A4)

Factor B (servings of soy milk)

None (B1)

A1B1

A2B1

A3B1

A4B1

1 cup of milk (B2)

A1B2

A2B2

A3B2

A4B2

2 cups of milk (B3)

A1B3

A2B3

A3B3

A4B3

3 cups of milk (B4)

A1B4

A2B4

A3B4

A4B4

Doing and Interpreting Factorial ANOVA

First, we need to set up hypotheses. Because we have two independent factors, we need hypotheses for each of the factors and another for the interaction effects between the two factors:

H01: There is no difference among group means in Factor A.

Ha1: At least two group means in Factor A differ.

or

H01: μ1 = μ2 = μ3

Ha1: μj≠μk for some j and k

H02: There is no difference among group means in Factor B.

Ha2: At least two group means in Factor B differ.

H03: There is no interaction effect between Factor A and B.

Ha3: There is an interaction effect between Factor A and B.

The test statistic for each hypothesis in factorial ANOVA can be found by the same equation:

where differences between groups are divided into three components: (1) differences between groups explained by Factor A, (2) differences between groups explained by Factor B, and (3) differences between groups explained by the interaction of Factors A and B. We evaluate the associated p-value with this statistic and make a decision (i.e., reject the null hypothesis when the p-value associated with the computed statistic is small or not reject when the p-value associated with the computed statistic is large). Of course, we will support this statistical result with a measure of effect size and a corresponding interval estimate (i.e., confidence interval) as a measure of importance.

Results are reported as main effect and interaction effect. The main effect is the group difference between each of the independent (grouping) variables on the dependent variables. The interaction effect reflects the interaction of grouping variables on the dependent variables. Be sure to look at the interaction effect first, as the main effect of an independent variable compares means of its groups without considering another independent variable—that is, the results of the main effect cannot be trusted when the effect of one factor is associated with the effect of another factor. In that case, you need to conduct a simple effect analysis, which looks at the effect of one variable at each level of the other variable to see if the results of a given main effect happen at all levels of the other variable. In contrast, the main effect results can be interpreted, as they are when a substantial interaction effect does not exist.

To conduct a two-way factorial ANOVA in Excel, you will open ExerciseFA.xlsx and note that the data should be formatted differently, as shown in Figure 12-5. Go to Data > Data Analysis, as shown in Figure 12-6, and choose “ANOVA: Two-Factor With Replication” (Figure 12-7), as we have four groups of exercise and individuals within each group are doing more than one thing (i.e., drinking four different amounts of milk). In the “ANOVA: Two-Factor With Replication” dialogue box, you will provide A1:E21 as Input Range and insert “5” as Rows per sample (Figure 12-8). Clicking “OK” will then produce the output of requested regression analysis. The example output is shown in Figure 12-9.

Formatting the data for analysis in Excel.

An Excel screenshot shows the formatting of data of conduct a two-way factorial ANOVA.

Courtesy of Microsoft Excel © Microsoft 2020.

Finding Data Analysis ToolPak in Excel.

An Excel screenshot shows the Data Analysis ToolPak add-in, in the Analysis group under Data menu.

Courtesy of Microsoft Excel © Microsoft 2020.

Selecting “ANOVA: Two-Factor With Replication” in Excel.

An Excel screenshot shows selection of the option, ANOVA: two-factor with replication, within the Data Analysis Tools. The worksheet lists the five columns, Exercise, No milk, 1 cup of milk, 2 cups of milk, 3 cups of milk, with numerical data.

Courtesy of Microsoft Excel © Microsoft 2020.

Defining data ranges and selecting options for “ANOVA: Two-Factor With Replication” in Excel.

An Excel screenshot shows the ANOVA: two-factor with replication dialog box with fields to define data. The worksheet lists the five columns, Exercise, No milk, 1 cup of milk, 2 cups of milk, 3 cups of milk, with numerical data.

Courtesy of Microsoft Excel © Microsoft 2020.

Example output for “ANOVA: Two-Factor With Replication” in Excel.

An Excel screenshot shows an output of the ANOVA: Two-factor with replication.

Courtesy of Microsoft Excel © Microsoft 2020.

To conduct a two-way factorial ANOVA in SPSS, you will open ExerciseFA.sav and go to Analyze > General Linear Model > Univariate, as shown in Figure 12-10. The variables shown in Figure 12-11 represent the amount of exercise and the number of servings of soy milk consumed per day as independent variables, and the health problem index as a dependent variable. In the Univariate dialogue box, you will move the independent variables into “Fixed Factor(s)” and a dependent variable into “Dependent Variable” by clicking corresponding arrow buttons in the middle, as shown in Figure 12-11. There are six buttons in the box, but only the commonly used buttons are discussed here. The “Contrasts” and “Post Hoc” buttons are used when further investigations are needed among more than two groups in the factor with an overall group difference from ANOVA results, as discussed in the earlier section on planned contrasts and post hoc tests. The “Plots” button can be used to help us interpret the interaction effect visually and can be created as shown in Figure 12-12. The “Options” button gives us several choices that may help us interpret the results of factorial ANOVA; these are shown in Figure 12-13. Clicking “OK” will then produce the output of the requested factorial ANOVA analysis. An example output is shown in Table 12-3.

Selecting factorial ANOVA under “General Linear Model” in SPSS.

There are three columns of numerical data: Exercise, Milk, and Health. The values of Milk range between 1 and 4, with 1 indicating No milk, 2 indicating 1 cup of milk, 3 indicating 2 cups of milk, and 4 indicating 3 cups of milk.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Defining variables in factorial ANOVA in SPSS.

A screenshot in S P S S Editor defines variables in factorial ANOVA. The numerical data in the worksheet lists Exercise, Milk, and Health values.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Creating an interaction plot in factorial ANOVA in SPSS.

A screenshot in S P S S Editor shows the steps involved in creation of an interaction plot in factorial ANOVA.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Useful options for factorial ANOVA in SPSS.

A screenshot in S P S S Editor shows the Univariate Options dialog box. The numerical data in the worksheet lists Exercise, Milk, and Health values.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Table 12-3 Example Output of Factorial ANOVA in SPSS

Between-Subjects Factors

Value Label

N

Amount of exercise

1

None

20

2

1 day per week

20

3

3 days per week

20

4

5 days per week

20

1

None

20

Amount of milk drunk per day

2

1 cup of milk

20

3

2 cups of milk

20

4

3 cups of milk

20

Descriptive Statistics Dependent Variable: Health Index

Amount of Exercise

Amount of Milk Consumed

Mean

Std. Deviation

N

None

None

1.2480

.93229

5

1 cup of milk

1.1060

.27907

5

2 cups of milk

1.4360

1.24422

5

3 cups of milk

2.0240

.73330

5

Total

1.4535

.87585

20

1 day per week

None

3.6140

2.27765

5

1 cup of milk

3.9600

2.85788

5

2 cups of milk

5.2260

3.22941

5

3 cups of milk

10.2200

5.05440

5

Total

5.7550

4.21423

20

3 days per week

None

3.4300

1.86317

5

1 cup of milk

1.1700

.62678

5

2 cups of milk

4.0660

2.46468

5

3 cups of milk

11.4260

5.16824

5

Total

5.0230

4.82911

20

Descriptive Statistics Dependent Variable: Health Index

Amount of Exercise

Amount of Milk Consumed

Mean

Std. Deviation

N

5 days per week

None

6.8500

6.32134

5

1 cup of milk

2.6020

2.06655

5

2 cups of milk

5.6820

5.30650

5

3 cups of milk

9.1080

7.21566

5

Total

6.0605

5.65636

20

Total

None

3.7855

3.82430

5

1 cup of milk

2.2095

2.04190

5

2 cups of milk

4.1025

3.54806

5

3 cups of milk

8.1945

6.01191

5

Total

4.5730

4.60305

20

Levene’s Test of Equality of Error Variancesa Dependent Variable: Health Index

F

df1

df2

Sig.

4.508

15

64

.000

aDesign: Intercept + Exercise + Milk + Exercise × Milk

Tests the null hypothesis that the error variance of the dependent variable is equal across groups.

Tests of Between-Subjects Effects Dependent Variable: Health Index

Source

Type III Sum of Squares

df

Mean Square

F

Sig.

Corrected model

819.988a

15

54.666

4.097

.000

Intercept

1,672.986

1

1,672.986

125.395

.000

Exercise

270.871

3

90.290

6.767

.000

Milk

390.858

3

130.286

9.765

.000

Exercise × milk

158.258

9

17.584

1.318

.245

Error

853.873

64

13.342

Total

3,346.847

80

Corrected total

1,673.861

79

aR-square (R2) = .490 (adjusted R2 = .370)

A line graph shows the estimated marginal means of health.

You will see that Levene’s test for the assumption of homogeneity of variance is violated, but ANOVA designs tend to be robust when the sample sizes across the groups are the same. From the interaction plot (Profile Plots), we can see that the lines look relatively parallel to each other. When the lines in the interaction plot are parallel, we say that there is no interaction effect between the two factors; if the lines cross, there is an interaction effect. In this case, there is no interaction between servings of soy milk per day and level of exercise.

In the “Test of Between-Subjects Effects” table, the two main factors (exercise and amount of soy milk consumed) were found to have substantial effects on the health index, but the interaction between them was not substantial. Because one factor is not dependent upon the other factor from its nonsubstantial effect (i.e., one factor does not work with the other factor to make changes on means of the dependent variable), the main effects can be interpreted as they are. Both the amount of exercise and the amount of milk consumed per day have a substantial effect on the health problem index, but we still do not know how the groups (levels of exercise and amount of milk consumed) differ. Therefore, Bonferroni tests on both factors should be done as post hoc tests; these results are shown in Table 12-4. From the outputs, we can see that people who exercise 0 days per week have a substantially lower health index than people who exercise at least 1 day per week. We can also see that people who drank 3 cups of soy milk have a substantially higher health index than those who drank less than 3 cups of soy milk.

Table 12-4 Post Hoc Test Results of Factorial ANOVA in SPSS

Multiple Comparison Health Index, Bonferroni

Amount of Exercise (I)

Amount of Exercise (J)

Mean Difference (I μ J)

Std. Error

Sig.

95% Confidence Interval

Lower Bound

Upper Bound

None

1 day per week

μ4.3015*

1.15507

.002

μ7.4464

μ1.1566

3 days per week

μ3.5695*

1.15507

.018

μ6.7144

μ.4264

5 days per week

μ4.6070*

1.15507

.001

μ7.7519

μ1.4621

1 day per week

None

4.3015*

1.15507

.002

1.1566

7.4464

3 days per week

.7320

1.15507

1.000

μ2.4129

3.8769

5 days per week

μ.3055

1.15507

1.000

μ3.4504

2.8394

3 days per week

None

3.5695*

1.15507

.018

.4246

6.7144

1 day per week

μ.7320

1.15507

1.000

μ3.8769

2.4129

5 days per week

μ1.0375

1.15507

1.000

μ4.1824

2.1074

5 days per week

None

4.6070*

1.15507

.001

1.4621

7.7519

1 day per week

.3055

1.15507

1.000

μ2.8394

3.4504

3 days per week

1.0375

1.15507

1.000

μ2.1074

4.1824

*The mean difference is significant at the .05 level.

Based on observed means.

The error term is mean square (error) = 13.342.

Multiple Comparisons Health Index, Bonferroni

Amount of Milk Drunk per Day (I)

Amount of Milk Drunk per Day (J)

Mean Difference (Iμ J)

Std. Error

Sig.

95% Confidence Interval

Lower Bound

Upper Bound

None

1 cup of milk

1.5760

1.15507

1.000

μ1.5689

4.7209

2 cups of milk

μ.3170

1.15507

1.000

μ3.4619

2.8279

3 cups of milk

μ4.4090*

1.15507

.002

μ7.5539

μ1.2641

1 cup of milk

None

μ1.5760

1.15507

1.000

μ4.7209

1.5689

2 cups of milk

μ1.8930

1.15507

.637

μ5.0379

1.2519

3 cups of milk

μ5.9850*

1.15507

.000

μ9.1299

μ2.8401

2 cups of milk

None

.3170

1.15507

1.000

μ2.8279

3.4619

1 cup of milk

1.8930

1.15507

.637

μ1.2519

5.0379

3 cups of milk

μ4.0920*

1.15507

.004

μ7.2369

μ.9471

3 cups of milk

None

4.4090*

1.15507

.002

1.2641

7.5539

1 cup of milk

5.9850*

1.15507

.000

2.8401

9.1299

2 cups of milk

4.0920*

1.15507

.004

.9471

7.2369

*The mean difference is significant at the .05 level.

Based on observed means.

The error term is mean square (error) = 13.342.

Reporting Two-Way Factorial ANOVA

Reporting two-way factorial ANOVA results is similar to that in one-way ANOVA. You should report the size of F-statistics along with associated degrees of freedom and associated p-value for both the main effect and interaction effect. The computation of effect size for any factorial ANOVA design is more complicated and cumbersome than for one-way ANOVA, but we can obtain and report estimates of effect size, partialη2, as shown in Figure 12-13. We recommend that you compute and interpret other types of effect size in consultation with a statistician.

The following is a sample report from our example:

  • There was no interaction effect between the amount of exercise per week and the number of servings of soy milk per day on the health index, F (9, 64) = 1.32, p = .245, η2 = .156.
  • There was a substantial effect of the amount of exercise per week on the health index, F (3, 64) = 3.78, p = .015, η2 =.241. The Bonferroni test revealed that the group that did not exercise had a substantially lower health index than those who exercised 1 day per week (p =.002, 95% CI [μ7.45, μ1.16]), those who exercised 3 days per week (p =.018, 95% CI [μ6.71, μ0.43]), and those who exercised 5 days per week (p =.001, 95% CI [μ7.75, μ1.46]).
  • There was also a substantial effect of the number of servings of soy milk drunk per day on health index: F (3, 64) = 14.40, p = .000, η2 =.314. The Bonferroni test revealed that the group who drank 3 cups of soy milk per day had a substantially higher health index than those who drank no milk (p =.002, 95% CI [1.26, 7.55]), those who drank 1 cup of milk (p =.000, 95% CI [2.84, 9.13]), and those who drank 2 cups of milk (p =.004, 95% CI [0.94, 7.24]).

Repeated Measures Anova ⬆ ⬇

Introduction

Until now, we have discussed ANOVA designs where several independent group means are compared. However, there are situations where three or more group means come from the same group of participants, similar to what we discussed in dependent samples t-test. Repeated measures ANOVA is simply an extended design of the dependent samples t-test where there are more than two measurements in the same group of participants. Recall that repeated measures design can take two different forms. The first is when the same variable is measured multiple times with the same group of participants to see changes over time, such as in our case study (Keating and McCurry 2019); and the second is when multiple treatments are given to the same group of participants to compare responses to each treatment. Repeated measures ANOVA can also be used when sample members are matched based on some important characteristics. For example, male and female nursing home residents may be matched when a researcher is interested in examining the level of sleep disturbance, since they reside in the same environment.

Repeated measures ANOVA design provides the following advantages:

  • It is suitable when some variables are to be measured repeatedly over time (i.e., longitudinal design).
  • It is an economical design compared with independent sample group comparison because the required sample size is smaller than when you have more than one group.
  • Because it analyzes only one group, the variability is lower than in a multigroup design, and therefore the validity of results will be higher.

However, there are disadvantages:

  • Repeated measurements may not be possible because of dropout, death, and so on.
  • There may be a maturation effect where the subjects change over the course of the study.
  • There may be other effects that reduce the validity of results, such as subjects’ scores converging on the average with multiple measurements.
Assumptions

Repeated measures ANOVA assumptions are very similar to those required by one-way ANOVA, such as normality and homogeneity of variance. Similar to other ANOVA designs, repeated measures ANOVA is robust to the violation of the normality and homogeneity of variance, especially when group sample sizes are the same. Note that the assumption of homogeneity of variance only applies when there are groups that are compared along with repeated measures. However, the last assumption of independence is automatically violated because the measurements are coming from the same group of participants. Instead, the assumption of relations between/among repeated measures, the assumption of sphericity, is added and should be tested. The assumption of sphericity means that the variances of the differences, as well as the correlations among the repeated measures, are all equal.

Doing and Interpreting Repeated Measures ANOVA

First, we need to set up hypotheses:

H0: There is no difference among repeated measure means.

H1: At least two repeated measure means differ.

or

H01: μ1 = μ2 = μ3

Ha1: μj≠μk for some j and k.

The test statistic each hypothesis in repeated measures ANOVA can be found by the following equation:

Once the statistic is computed, we evaluate whether the associated p-value is small enough to rule out chance; in other words, is the p-value small or large enough to indicate evidence for an effect? As discussed earlier, it is important to support the p-value with a measure of effect size, along with a corresponding interval estimate (i.e., confidence interval), as a measure of importance.

To conduct a repeated measures ANOVA in Excel, you will open Falls.xlsx (the data shown in the figure are the number of falls across four times: baseline, 3 months after, 6 months after, and 9 months after implementing a newly developed fall prevention intervention) and go to Data > Data Analysis, as shown in Figure 12-14. In the Data Analysis window, you will note that “ANOVA: Two-Factor Without Replication” is on the list (Figure 12-15), as only one group doing more than one thing (i.e., getting measured on four different occasions). In the “ANOVA: Two-Factor Without Replication” dialogue box, you will provide A1:D51 as Input Range with “Labels” checked (Figure 12-16). Clicking “OK” will then produce the output of the requested regression analysis. The example output is shown in Figure 12-17.

Finding the Data Analysis ToolPak in Excel.

An Excel screenshot shows the Data Analysis ToolPak add-in, in the Analysis group under Data menu.

Courtesy of Microsoft Excel © Microsoft 2020.

Selecting “ANOVA: Two-Factor Without Replication” in Excel.

An Excel screenshot shows selection of the option, ANOVA: two-factor without replication, within the Data Analysis Tools. The worksheet lists the four columns, baseline, 3 month after, 6 month after, and 9 month after, with numerical data.

Courtesy of Microsoft Excel © Microsoft 2020.

Defining data ranges and selecting options for “ANOVA: Two-Factor Without Replication” in Excel.

An Excel screenshot shows the ANOVA: two-factor without replication dialog box with fields to define data. The worksheet lists the four columns, baseline, 3 month after, 6 month after, and 9 month after, with numerical data.

Courtesy of Microsoft Excel © Microsoft 2020.

Example output for “ANOVA: Two-Factor Without Replication” in Excel.

An Excel screenshot displaying the output for ANOVA: Two-factor without replication.

Courtesy of Microsoft Excel © Microsoft 2020.

To conduct a repeated measures ANOVA in SPSS, you will open Falls.sav and go to Analyze > General Linear Model > Repeated Measures, as shown in Figure 12-18. This will bring up the Repeated Measures Define Factor(s) dialogue box; here, the name of the within-subject factor and the number of repeated measures should be identified as shown in Figure 12-19. In this example, the data are the number of falls across four times: baseline, 3 months after, 6 months after, and 9 months after implementing a newly developed fall prevention intervention. Type “Falls” for name and “4” for level since there are four measurements, and then click “Define.” In the Repeated Measures dialogue box, you will see that four levels with question marks are already shown under “Within-Subjects Variable(s),” so now you need to tell SPSS what those levels are. Move the corresponding levels starting with the first measurement, in this case “baseline,” until every level is moved in order by clicking an arrow button in the middle, as shown in Figure 12-20. There are six buttons in the box, but only the commonly used buttons are discussed here. The “Contrasts” and “Post Hoc” buttons are used when further investigations are needed among more than two groups in the factor with an overall groups difference from ANOVA results, as discussed earlier. However, we have only one group in this example, so they are skipped here. However, the overall significance test in repeated measures ANOVA only says that there is a substantial difference between time measurements. It does not tell which measurements differ from the other measurements. The “EM Means” button provides a basic set of the post hoc tests in case the overall test indicates that there is a difference among the repeated measures (Figure 12-21), and the “Options” button provides several options that may help us interpret the results of repeated measures ANOVA (Figure 12-22). Clicking “OK” will then produce the output of the requested analysis. Here, you will see Bonferroni pairwise comparison results that will help us to figure out which measurements actually differ from the other measurements. Example output is shown in Table 12-5.

Selecting repeated measures ANOVA under “General Linear Model” in SPSS.

A screenshot in S P S S shows the selection of the Analyze menu, with General linear model command chosen, from which the repeated measures ANOVA option is selected. The data in the worksheet shows columns of numerical data.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Defining the name and level of repeated measures in repeated measures ANOVA in SPSS.

A screenshot in S P S S Editor defines the variables repeated measures ANOVA. The data in the worksheet shows columns of numerical data.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Defining variables in repeated measures ANOVA in SPSS.

A screenshot in S P S S Editor defines variables in repeated measures ANOVA. The data in the worksheet shows columns of numerical data.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Post hoc tests for repeated measures ANOVA in SPSS.

A screenshot in S P S S Editor shows the Repeated Measures Estimated Marginal Means dialog box. The data in the worksheet shows columns of numerical data.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Useful options for repeated measures ANOVA in SPSS.

A screenshot in S P S S Editor shows the Repeated Measures options dialog box. The data in the worksheet shows columns of numerical data.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Table 12-5 Example Output of Repeated Measures ANOVA in SPSS

Within-Subjects Factors Measure: MEASURE_1

Fall

Dependent Variable

1

Baseline

2

ThreeM

3

SixM

4

NineM

Descriptive Statistics

Mean

Std. Deviation

N

Baseline

9.42

.499

50

ThreeM

7.96

.880

50

SixM

6.54

.503

50

NineM

4.70

1.035

50

Multivariate Testsa

Effect

Value

F

Hypothesis df

Error df

Sig.

Fall

Pillai’s trace

.960

378.556b

3.000

47.000

.000

Wilks’ lambda

0.40

378.556b

3.000

47.000

.000

Hotelling’s trace

24.163

378.556b

3.000

47.000

.000

Roy’s largest root

24.163

378.556b

3.000

47.000

.000

aDesign: intercept

bExact statistic

Within subjects design: fall

Mauchly’s Test of Sphericitya

Within-­Subjects Effect

Mauchly’s W

Approx. Chi-Square

df

Sig.

Epsilonb

GreenhouseμGeisser

HuynhμFeldt

Lower Bound

.646

20.837

5

.001

.776

.817

.333

aDesign: intercept.

bMay be used to adjust the degrees of freedom for the averaged tests of significance.

bDesign: intercept.

Tests the null hypothesis that the error covariance matrix of the orthonormalized, transformed dependent variables is proportional to an identity matrix.

Corrected tests are displayed in the Tests of Within-Subjects Effects section.

Within subjects design: fall.

Mauchly’s Test of Sphericitya

Source

Type III Sum of Squares

Df

Mean Square

f

Sig.

Partial Eta Squared

Fall

Sphericity assumed

609.175

3

203.058

350.862

.000

.877

GreenhouseμGeisser

609.175

2.329

261.578

350.862

.000

.877

HuynhμFeldt

609.175

2.452

248.430

350.862

.000

.877

Lower bound

609.175

1.000

609.175

350.862

.000

.877

Error (fall)

Sphericity assumed

85.075

147

GreenhouseμGeisser

85.075

114.113

HuynhμFeldt

85.075

120.153

Lower bound

85.075

49.000

You will see that the group sample sizes are the same (in the Descriptive Statistics section) with no missing data (as seen in Table 12-5),and the assumption of sphericity is violated with a small p-value of .001. If this assumption is not violated, sphericity-assumed statistics could be interpreted and reported from the “Tests of Within-Subjects Effects” section of the table. However, this result will be biased when the assumption of sphericity is violated. Although this may sound complex, the “Tests of Within-Subjects Effects” section also generates other statistics that adjust for this violation, and one of these can be reported instead. There are different recommendations with regard to which one to use, but the GreenhouseμGeisser statistics are useful when the estimate of epsilon in the Mauchly’s test table is less than 0.75, and the HuynhμFeldt statistics are when it is larger than 0.75. In our example, epsilon is larger than 0.75, so HuynhμFeldt statistics should be used. Therefore, these repeated measures substantially differ, and each time measurement substantially differs from each other from Bonferroni pairwise comparisons, as shown in Table 12-6.

Table 12-6 Bonferroni Pairwise Comparisons

Pairwise Comparisons Measure: MEASURE_1

Fall (I)

Fall (J)

Mean Difference (I-J)

Std. Error

Sig.a

95% Confidence Interval for Differencea

Lower Bound

Upper Bound

1

2

1.460b

.128

.000

1.107

1.813

3

2.880b

.106

.000

2.590

3.170

4

4.720b

.164

.000

4.268

5.172

2

1

μ1.460b

.128

.000

μ1.813

μ1.107

3

1.420b

.140

.000

1.034

1.806

4

3.260b

.195

.000

2.723

3.797

3

1

μ2.880b

.106

.000

μ3.170

μ2.590

2

μ1.420b

.140

.000

μ1.806

μ1.034

4

1.840b

.163

.000

1.393

2.287

4

1

μ4.720b

.164

.000

μ5.172

μ4.268

2

μ3.260b

.195

.000

μ3.797

μ2.723

3

μ1.840b

.163

.000

μ2.287

μ1.393

aAdjustment for multiple comparisons: Bonferroni test.

bThe mean difference is significant at the .05 level.

Based on estimated marginal means.

Reporting Repeated Measures Factorial ANOVA

Reporting repeated measures ANOVA results should sound familiar to you, as it is similar to that in a one-way ANOVA, but the result of Mauchly’s test for sphericity should be the first part of reporting since it will determine what statistics to report. You should then report the size of F-statistics, along with associated degrees of freedom and the associated p-value. The computation of effect size for any repeated measures ANOVA design is fairly complicated and more cumbersome than that for one-way ANOVA, but we can obtain and report estimates of effect size, partialη2, as shown in Figure 12-13. For other types of effect size, we recommend that you compute and interpret these.

The following is a sample report from our example:

Mauchly’s test indicated that the assumption of sphericity has been violated, χ2 (5) = 20.84, p = .001; therefore, the HuynhμFeldt correction was used (ε = .82). The results show that the number of falls substantially changed over time, F (2.45, 120.15) = 350.86, p = .000, η2 = .877. The Bonferroni post hoc test revealed that the number of falls was substantially lower in 9 months after the fall prevention program than those at 6 months after (p =.000, 95% CI [μ2.29, μ1.39]) and 3 months after (p =.000, 95% CI [μ3.80, μ2.72]), and at baseline (p =.000, 95% CI [μ5.17, μ4.27]). The number of falls 6 months after the fall prevention program was substantially lower than those 3 months after (p =.000, 95% CI [μ1.81, μ1.03]), and at baseline (p =.000, 95% CI [μ3.17, μ2.59]); and the number of falls 3 months after the prevention program was also substantially lower than that of baseline (p =.000, 95% CI [μ1.81, μ1.11]).

Manova ⬆ ⬇

Introduction

MANOVA is another extension of ANOVA where group differences on more than one dependent variable are investigated. For example, we might be interested in studying the effect of exercise frequency on weight and bone density. We could group participants into low-, moderate-, and high-exercise-frequency groups, and then examine that effect on the two dependent variables: weight and bone density. You may think of conducting multiple one-way ANOVAs, but similar to multiple t-tests, the Type I error will be inflated and you will not be able to capture information about the relationship among those dependent variables. MANOVA has the power to detect significant group differences along a combination of dependent variables, and so it is a better approach than several one-way ANOVAs in this situation.

Multivariate procedures are very useful when you have multiple dependent variables to investigate simultaneously. However, you should keep in mind that the design will be much more complicated because there is more than one dependent variable. You may want to limit the number of dependent variables based on the manageability and alignment with the theory guiding the research. Too many dependent variables will confuse the results and make it difficult to convey the meaning of the study.

Assumptions

All of the ANOVA assumptions are also required for MANOVA, but in a multivariate way. Multivariate normality is assumed in MANOVA, but there is no way of checking this assumption in SPSS. However, you can at least assume multivariate normality if you find that each dependent variable is normally distributed as in ANOVA. However, homogeneity of variance in MANOVA means something a little different and needs a different approach. Because there is more than one dependent variable, there are covariances between dependent variables as well as variance within each dependent variable. Levene’s test will only be able to test for equality of variability, but there is a multivariate statistic that can test for equality of varianceμcovariance, named Box’s M test. The theory is the same as Levene’s test, in that it assumes the equality of varianceμcovariance among the variables and the assumption is said to be violated when the test produces a small p-value such as .001. As with ANOVA, MANOVA is also relatively robust to the violation of assumptions when the sample sizes across groups are the same.

Doing and Interpreting MANOVA

First, we need to set up hypotheses. The data contains nurses’ education level as an independent variable and the number of patient falls, functional ability, and quality of life as dependent variables. Note that we are testing whether the mean vectors, any subscripted elements array, are arranged in rows or columns (i.e., three means arranged in columns in this example), not a single mean. Therefore, the hypothesis of MANOVA is:

H0: The mean vectors of all groups are equal.

Ha: At least one mean vector is not equal to the others.

Some of the most commonly reported MANOVA statistics include Pillai’s trace, Wilks’ lambda, Hotelling’s trace, and Roy’s largest root. While Wilks’ lambda is the most preferred statistic for MANOVA, others may work better in situations where assumptions are violated. Detailed discussions on these statistics are beyond the scope of our text, and we recommend that you consult a statistician.

Once the statistic is computed, we examine the p-value to determine if the observed difference between the repeated measure means is substantial. As discussed earlier, it is important to support the statistical results with a measure of effect size, along with a corresponding interval estimate (i.e., confidence interval), as a measure of importance.

If the results of MANOVA indicate an overall group difference, it tells us that the groups differ on the combination of dependent variables, but it does not specify on which dependent variables the groups differ. There are several ways of following up the results of MANOVA with substantial group differences, including the following:

  • One-way ANOVAs: Substantial group differences from MANOVA can be followed by a series of one-way ANOVAs for each dependent variable, with an adjustment for Type I error, to see on which dependent variable the groups differ.
  • Discriminant analysis: This procedure is the exact opposite of MANOVA. It uses dependent variables in MANOVA as predictor variables and the groups as the dependent variable. Its goal is to predict group membership with predictor variables, and it allows assessing the relative importance of each dependent variable in predicting group membership.
  • RoyμBargman stepdown analysis: This procedure assesses the importance of each dependent variable using the most important dependent variable as a covariate in steps.

Because both discriminant analysis and RoyμBargman stepdown analysis are advanced techniques beyond the scope of this text, only a series of one-way ANOVAs with Bonferroni adjustment for Type I error will be discussed as a follow-up test of MANOVA in this text.

To conduct MANOVA in SPSS, you will open FallFuncQOL.sav and go to Analyze > General Linear Model > Multivariate, as shown in Figure 12-23. In the Multivariate dialogue box, move the independent variables into “Fixed Factor(s)” and the dependent variables into “Dependent Variables” by clicking the corresponding arrow buttons in the middle (see Figure 12-24). Notice the space for dependent variables is wider than that found in univariate general linear models. There are six buttons in the box, but only commonly used buttons are discussed here. The “Contrasts” and “Post Hoc” buttons are used when further investigations are needed among more than two groups in the factor with an overall group difference from ANOVA results, as discussed earlier in this chapter. However, we do not have to use these buttons since we have only two groups. The “Options” button provides several options that may help us interpret the results of MANOVA; these are shown in Figure 12-25. Clicking “OK” will then produce the output of the requested MANOVA analysis. An example output is shown in Table 12-7.

Selecting MANOVA under “General Linear Model” in SPSS.

A screenshot in S P S S shows the selection of the Analyze menu, with General linear model command chosen, from which the Multivariate option is selected. The data in the worksheet shows columns of numerical data.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Defining variables in MANOVA in SPSS.

A screenshot in S P S S Editor defines variables in MANOVA. The data in the worksheet shows columns of numerical data.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Useful options for MANOVA in SPSS.

A screenshot in S P S S Editor shows the Options for MANOVA dialog box. The data in the worksheet shows columns of numerical data.

Reprint Courtesy of International Business Machines Corporation, © International Business Machines Corporation. “IBM SPSS Statistics software (“SPSS”)”. IBM®, the IBM logo, ibm.com, and SPSS are trademarks or registered trademarks of International Business Machines Corporation.

Table 12-7 Example Output of MANOVA in SPSS

Multivariate Testsa

Effect

Value

F

Hypothesis df

Error df

Sig.

Partial Eta Squared

Noncent. Parameter

Observed Powerb

Intercept

Pillai’s trace

.978

145.679c

3.000

96.000

.000

.978

436.036

1.000

Wilks’ lambda

.022

145.679c

3.000

96.000

.000

.978

436.036

1.000

Hotelling’s trace

45.459

145.679c

3.000

96.000

.000

.978

436.036

1.000

Roy’s largest root

45.459

145.679c

3.000

96.000

.000

.978

436.036

1.000

Education

Pillai’s trace

.621

52.336c

3.000

96.000

.000

.621

157.009

1.000

Wilks’ lambda

.379

52.336c

3.000

96.000

.000

.621

157.009

1.000

Hotelling’s trace

1.636

52.336c

3.000

96.000

.000

.621

157.009

1.000

Roy’s largest root

1.636

52.336c

3.000

96.000

.000

.621

157.009

1.000

aDesign: intercept + education

bComputer using alpha = .05

cExact statistic

Tests of Between-Subjects Effects

Source

Dependent Variable

Type III sum of squares

df

Mean square

F

Sig.

Partial Eta Squared

Corrected Model

Number of Fall

200.255a

1

200.255

68.203

.000

.410

Functional Ability

117.749b

1

117.749

49.154

.000

.334

Quality of Life

140.701c

1

140.701

54.600

.000

.358

Intercept

Number of Fall

4290.495

1

4290.495

1461.252

.000

.937

Functional Ability

4251.429

1

4251.429

1774.744

.000

.948

Quality of Life

4059.301

1

4059.301

1575.247

.000

.941

Education

Number of Fall

200.255

1

200.255

68.203

.000

.410

Functional Ability

117.749

1

117.749

49.154

.000

.334

Quality of Life

140.701

1

140.701

54.600

.000

.358

Error

Number of Fall

287.745

98

2.936

Functional Ability

234.761

98

2.396

Quality of Life

252.539

98

2.577

Total

Number of Fall

4584.000

100

Functional Ability

4487.000

100

Quality of Life

4312.000

100

Corrected Total

Number of Fall

488.000

99

Functional Ability

352.510

99

Quality of Life

393.240

99

R Squared = 410 (Adjusted R squared = .404)

R Squared = .334 (Adjusted R Squared = .327)

R Squared = .358 (Adjusted R Squared = .351)

The output indicates that the independent variable, nurses’ education level, has a substantial effect on the combined dependent variables of number of falls, functional ability, and quality of life, with a small p-value of .000. Note that the output reports four different statistics for the effect of the independent variable: Pillai’s trace, Wilks’ lambda, Hotelling’s trace, and Roy’s largest root. These statistics will report the same results in most situations with relative robustness to the violation of multivariate normality, as in our output, but there may be some cases where not all statistics agree on the significance of the independent variables. If the statistics are dissimilar, the decision can be made based on which set of dependent variables the group differences occur, but refer to previous research for further discussion (Olson, 1974, 1976; Stevens, 1980). Separate univariate ANOVAs performed as a follow-up test indicated that nurses’ education level had a substantial effect on all dependent variables. From Table 12-7, we can see that nurses’ education level had a substantial effect on the number of falls, functional ability, and quality of life, with a small p-value of .000.

Reporting MANOVA

Reporting MANOVA results is a bit different from reporting a one-way ANOVA results because it uses different statistics. You should include one of the four multivariate statistics that are reported in the output along with the size of F-statistics, associated degrees of freedom, and the associated p-value. The computation of effect size for a MANOVA design is fairly complicated and more cumbersome than that for one-way ANOVA, but we can obtain and report estimates of effect size, partialη2, as shown in Figure 12-25. We recommend that you compute and interpret in consultation with a statistician for the other types of effect sizes.

The follow-up ANOVAs’ results are reported as shown previously.

The following is a sample report from our example:

There was a substantial effect of nurses’ education level on the combined dependent variables, including number of falls, functional ability, and quality of life, Λ = 0.38, F (3, 96) = 52.34, p = .000, η2 = .621. Follow-up separate univariate ANOVAs revealed that nurses’ education level had a substantial effect on all of the dependent variables: number of falls, F (1, 98) = 68.20, p = .000, η2 = .410, functional ability, F (1, 98) = 49.15, p = .000, η2 = .334, and quality of life, F (1, 98) = 54.60, p = .000, η2 = .358.

Note that we have reported on Wilks’ lambda, but you could choose the others. You will have to ensure that you use the corresponding symbol to report (i.e., V for Pillai’s trace, T for Hotelling’s trace, and Θ for Roy’s largest root).

Nurse investigators occasionally encounter studies and analyses that include an extension of MANOVA, multivariate analysis of covariance (MANCOVA), and factorial MANOVA. While this discussion is beyond the scope of our text, you should understand that MANCOVA is used to control for the effect of covariates on the combination of dependent variables and factorial MANOVA is used to investigate both the main and interaction effect of independent variables on a combination of dependent variables.

Summary ⬆ ⬇

In this chapter, we have seen how to carry out advanced ANOVA analyses to make comparisons between means. Occasionally, it will be important to control other variables that we think may influence the dependent variable, and we can accomplish this by employing ANCOVA. Other variations of ANOVA include factorial and repeated measures, which are specialized applications of one-way ANOVA that are applicable in situations when there are multiple independent variables or the same variable is being measured over time. MANOVA allows us to analyze the effects of the grouping variable(s) on more than one dependent variable.

All these techniques share similar assumptions related to normality, independence, and homogeneity of variance. Once we have examined the effect of independent variable(s) on the dependent variable(s), we also need to conduct follow-up statistical tests that tell us more specifically what groups are creating the effect.

Tests of differences between means are most useful in descriptive comparative and experimental designs in which differences between groups are of interest.

Critical Thinking Questions ⬆ ⬇

  1. What are the differences between one-way ANOVA, factorial ANOVA, repeated measures ANOVA, ANCOVA, and multivariate ANOVA?
  2. Why is ANCOVA a more powerful design than one-way ANOVA when there is a third variable that is affecting a relationship between the independent and dependent variables?
  3. How do substantial interaction effects influence the interpretation of main effects in factorial ANOVA?

Self-Quiz ⬆ ⬇

  1. Suppose that you plan to compare systolic blood pressure between three age groups (50μ65 years, 66μ80 years, and 80 years and older), and evidence from the literature suggests that age may influence blood pressure. What statistical test will be most appropriate?
    1. one-way ANOVA
    2. analysis of covariance
    3. independent t-test
    4. MANOVA
  2. Suppose that you find that there is an interaction effect of gender and ethnicity on risk for falls in a nursing home. What statistic will help you determine which variable is contributes the greatest variance?
    1. simple effect analysis
    2. Bonferroni test
    3. Levine’s test for homogeneity
  3. Suppose that you find that there is a substantial effect of the independent variable on the set of four dependent variables. Which tests can you conduct to follow up the substantial results?
    1. a series of ANOVAs
    2. RoyμBargman stepdown analysis
    3. discriminant analysis
    4. All of these are correct.

Reference ⬆

Keating, S. R., &McCurry, M. (2019). Text messaging as an intervention for weight loss in emerging adults. Journal of the American Association of Nurse Practitioners, 31(9), 527μ536.

Olson, C. L. (1974). Comparative robustness of six tests in multivariate analysis of variance. Journal of the American Statistical Association, 69, 894μ908.

Olson, C. L. (1976). On choosing a test statistic in multivariate analysis of variance. Psychological Bulletin, 83, 579μ586.

Stevens, J. P. (1980). Power of the multivariate analysis of variance tests. Psychological Bulletin, 88, 728μ737.