Factorial Between-Subjects Analysis of Variance (ANOVA)

This chapter will help you to:

1. State the goal of the factorial ANOVA
2. Identify the various effects tested in a factorial ANOVA
3. State the null hypotheses for each effect in a factorial ANOVA
4. Interpret interaction plots related to factorial designs
5. Explain the two types of post hoc tests in a factorial ANOVA
6. Use the univariate GLM procedure to run a factorial between-subjects ANOVA
7. Interpret the results of the GLM procedure for a factorial between-subjects ANOVA
8. Present the results of a factorial between-subjects ANOVA using APA style guidelines.

The Factorial ANOVA

We are now at a point to really appreciate the strength of the ANOVA. In the last lesson (within-subjects ANOVA), we saw how we could partition the variance in DV scores due to the participants. We removed that variance from consideration because we were not interested in the effect of the participant. We can use that same approach, however, for other variables. This is exactly what we do in a “factorial” design; investigate the effects of more than one predictor variable.

A factorial design has multiple predictor variables

Factorial Designs

The world is complex. That is, there are many simultaneously impactful factors at work for any given phenomenon. If want to predict who will do well in college, we may want to know more than a student’s gender. Perhaps we may better predict GPA if we add in employment status. In a factorial design, we measure the dependent variable under the combination of levels from more than one independent or predictor variable.

In this lesson, we’ll be examining the factorial between-sujects ANOVA. As you recall, this implies that each participant recieves only one level of each IV. In a fully balanced design, every participant gets exposed to one level of each IV. In unbalanced design, some participants may not receive the manipulation for all IVs.

In a factorial between-subjects design each participant is exposed to just one level of each independent variable.

Relevant Research Questions

Whenever we are wanting to know the impact of multiple categorical predictor variables on a *continuous dependent variable, the factorial ANOVA may a good option.

Assumptions

There are assumptions about the data that must be verified for the conclusions of the factorial between-subjects ANOVA to be valid.

1. Normal distribution of DV scores within each combination of IV levels.

This requires that we check the distribution of scores after breaking the participants into groups according to the combination of IV levels. As we’ll see in the example, this means checking for normality in each cell.

2. Equal variance of DV scores across the IV combination groups.

This requires us to calculated and compare the variance for each cell.

Null Hypotheses

As with our other ANOVA, our null hypothesis is that there is no impact of the IV on DV. The difference here is that we now have multiple IVs so we will have multiple null hypotheses.

As such, for any given effect we test, we assume that the group means are equal to one another and thus that all samples are derived from the same population.

\[H_{0_{A}}:M_{A_1} = M_{A_2} =\dots = M_{A_j} = \mu_0\\ H_{0_{B}}:M_{B_1} = M_{B_2} =\dots = M_{B_k}= \mu_0\\ H_{0_{AxB}}:M_{A_1B_1} = M_{A_1B_2} = M_{A_2B_1} = \dots = M_{A_jB_k} = \mu_0\]

Let’s work with an example to make the factorial ANOVA a little more concrete.

Example

If you recall from the one-way between-subjects ANOVA lesson, we had an example about the effect of different treatments on strawberry sweetness. In that example, we had one variable (i.e., treatment) that had three levels (i.e., none, sugar water, and MiracleGro). Let’s expand that split plot design by adding another factor that might influence strawberry sweetnes: soil.

Dependent Variable: Sweetness Rating (0 - 10)

Independent Variable 1: Treatment (Water vs. Sugar Water vs. MiracleGro)

Independent Variable 2: Soil (Natural vs. Enriched)

Naming Convention

Let’s take a minute to introduce a new naming convention. An ANOVA with only one independent variable are referred to as a one-way ANOVA. An ANOVA with two independent variables may be referred to as a two-way ANOVA. An ANOVA with four independent variables can be called a four-way ANOVA.

We can be a little more informative in our naming by introducing the number of levels of each IV in the name.

For example, rather than just describing an ANOVA as a two-way ANOVA, we might call it a 3x4 ANOVA. This would tell us that the design have two independent variables. The first IV has three levels and the second IV has four levels.

Here is the breakdown.

The number of terms tells you the number of independent variables.

An AxBxCxD ANOVA would have 4 independent variables. An AxB ANOVA would have 2.

The value of each term tells you the number of levels for that independent variable.

A 2x4x3 ANOVA has an IV with 2 levels, another IV with 4 levels, and another IV with 3 levels.

Table 1 represents a 3x2 split plot design with our two variables.

Table 1

Representation of Two-Factor Split Plot Design

	Water	Sugar Water	MiracleGro
Natural	1 2 1	0 0 1	5 6 7
Enriched	2 3 3	0 1 1	9 8 10

You may see the similarity with the one-way within-subjects ANOVA at this point in how we have a grouping by rows instead of just by columns. There are some important differences, however. We now have more than one effect we’ll want to test.

Effects

In factorial designs, we will have two kinds of effects: main effects and interaction effects.

Main Effects

We are familiar with main effects as they are similar to what we tested in the one-way ANOVAs. A main effect is the impact of changing the levels of just one IV on the DV. That means that a main effect will only consider the impact of one IV at a time on the DV. You will have one main effect to test for each IV.

Main effects are the impact of one IV on the DV, regardless of the other IVs.

In our example, we would test for main effect of treatment and a main effect of soil.

The main effects are a comparison of the marginal means for each IV to the grand mean. Table 2 includes the marginal means for both treatment and soil.

Table 2

Marginal Means for Main Effects

	Water	Sugar Water	MiracleGro	Marginal Mean
Natural	1 2 1	0 0 1	5 6 7	2.56
Enriched	2 3 3	0 1 1	9 8 10	4
Marginal Mean	2	0.5	7.33	GM = 3.28

Note. Yellow highlighted cells relate to the main effect of Soil. Green highlighted cells relate to the main effect of Treatment.

Interaction Effects

Although it is convenient to test two variables in the same sample, the more imporant benefit of the factorial ANOVA is the ability to test for an interaction effect of two or more variable. You may have heard of interaction effects in relation to medications. Usually, you hear about the negative side effects that can occur when combining medications. The key point is that either may be a safe drug when taken alone but when the drugs are acting simultaneously, a new effect emerges. As such, interaction effects are the changes that occur in the dependent variable because of the combination of multiple independent variables.

Interaction effects are the combined impact of IVs on the DV.

There are two types of interaction effects. The first is called a crossover interaction because the pattern of the means changes directions as the other IV changes levels. Figure 1 demonstrates the crossover interaction

Figure 1

Crossover Interaction

The other type of interaction is an ordinal interaction. Rather than switching directions, the difference in means across the levels in one IV increases or decreases as the levels of the other IV change. Figure 2 represents an ordinal interaction.

Figure 2

Ordinal Interaction

In both cases, we can judge if an interaction effect is likely by the divergence of our lines from parallel.

In our example, we would test for one interaction effect, the treatment x soil (read as “treatment by soil”) interaction. This effect is tested by comparing the cell means (i.e., the mean sweetness of strawberries grown with water in natural soil) to the grand mean. Table 3 adds the cell means to the marginal means of Table 2.

Table 3

Cell Means for Interaction Effect

	Water	Sugar Water	MiracleGro	Marginal Mean
Natural	1 2 1 – M=1.33	0 0 1 – M=0.33	5 6 7 – M=6	2.56
Enriched	2 3 3 – M=2.67	0 1 1 – M=0.67	9 8 10 – M=8.67	4
Marginal Mean	2	0.5	7.33	GM = 3.28

Note. Pink highlighted cells relate to the interaction effect of Soil x Treatment.

Sources of Variance

The ANOVA approach is all about partitioning the variability in the DV scores. We had previously divided the variance into the effect variance and the error variance (we also separated the variance due to participants for the within-subjects design). The factorial ANOVA uses the same approach to partition variance into main effect variance, interaction effect variance, and error variance.

We discussed the sources for the main and interaction effects in the previous section. Where is the error? Remember that the “error” variance is the left over variance in the dependent variable that we cannot account for. The main effects account for variability across rows and columns. The interaction effect accounts for variability across cells. What is left? The variability within cells (i.e., the difference among values and the cell means)! Table 4 highlights the error variance.

Table 4

Within Cell Variance as Error Variance

	Water	Sugar Water	MiracleGro	Marginal Mean
Natural	1 2 1 – M=1.33	0 0 1 – M=0.33	5 6 7 – M=6	2.56
Enriched	2 3 3 – M=2.67	0 1 1 – M=0.67	9 8 10 – M=8.67	4
Marginal Mean	2	0.5	7.33	GM = 3.28

Note. Blue highlighted values relate to the calculation of error variance.

You can all of these sources of variance represented for our example in Table 5 in the next section.

Analysis Procedure

Check Assumptions

The first step in the factorial ANOVA is to verify that the assumptions hold.

You can check the assumption regarding normality by producing histograms or Q-Q plots for each combination of factor levels (see walkthrough below).

You can check the assumption of homogeneity of variance by producing a simple error bar chart and checking Leven’s test for the equality of variances (see walkthrough below).

Test Main and Interaction Effects

The GLM procedure in SPSS will produce a test of between subjects effect table that is an ANOVA table.

Table 5 representes an ANOVA table for our example of the impact of soil and treatment on strawberry sweetness

Table 5

ANOVA Table for Sweetness Example

Source	Sum of Squares	df	Mean Square	F	Sig.
Treatment	154.78	2	77.39	174.125	<.001
Soil	9.39	1	9.39	21.125	<.001
Treatment x Soil	4.11	2	2.06	4.625	.032
Error	5.33	12	0.44	–	–
Total	173.61	17	–	–	–

As we had done with previous ANOVA, we will check the “Sig.” column for values less than .05 (our alpha-level). If the Sig. value (i.e., the p-value) is less than .05, we reject the null hypothesis for that effect. We will need to do this for *each effect in our table.

In our example, each main effect and the interaction effect are statistically significant.

Recall that a significant F-value tells us that there is a difference among means, somehwere, but not which means are reliably different. As such, we will likely need post-hoc analyses to elucidate the pattern of means. This is true EXCEPT FOR IVs WITH ONLY 2 LEVELS. If one of your IVs only has two levels, then you know that the two sample means are reliably different from the ANOVA and you do not need to perform a separate post hoc analysis.

Post Hoc Analyses

Order of Post Hoc Analysis

There are three significant effects (one for treatment, one for soil, and one for the interaction of treatment by soil). Which should we investigate further first?

If you have a significant interaction effect, start with that interaction effect. Interaction effects will tell you the most about how the IVs effect the DV. Furthermore, in some cases, the main effects main contradict the interaction effect. As such, the more nuanced story (i.e., the interaction effect) is the better story to tell.

Order of Post Hoc Analyses

1. Significant interactions should be investigated first because they have the most amount of detail regarding how the IVs impact the DV.

2. If the interaction is not significant, investigate the significant main effects.

3. If no effects are statistically significant, do not perform (or interpret) post hoc analyses.

Types of Post Hoc Analyses

Before we get to how to perform post hoc analyses, let’s revisit why we want to perform post hoc analyses. We want to know about the pattern of means. We want to know how changing the level of IVs impacts the DV scores. This is especially important for the interaction effect.

Simple Effects Test

If we have a statistically significant interaction effect, it is telling us the impact of one IV on the DV changes as we change the levels of the other IV. As such, we may want to examine the effect of one IV within the levels of the other IV. That is, we take the data the was collected under one condition of one IV and test for the effect of the other IV on the DV. This approach is called the simple effects test.

The simple effects test ivnestigates the effect of one IV on the DV within each level of another IV.

In our example, we will test the effect of treatment in the “natural” soil condition and then separately in the “enriched” soil condition using one-way ANOVA. Tables 6 and 7 are the results of the simple effects tests within each type of soil.

Table 6

Simple Effects Test of Treatment within Natural Soil

Source	Sum of Squares	df	Mean Square	F	Sig.
Treatment	54.89	2	27.44	49.40	<.001
Error	3.33	6	0.556	–	–
Total	58.22	8	–	–	–

Table 7

Simple Effects Test of Treatment within Enriched Soil

Source	Sum of Squares	df	Mean Square	F	Sig.
Treatment	104	2	52.00	156	<.001
Error	2	6	0.33	–	–
Total	106	8	–	–	–

With the effect of treatment being statistically significant in both conditions of soil (Fs > 49, ps < .001), we can move into the next post hoc analysis: pairwise comparisons.

Pairwise Comparison: Tukey HSD

The Tukey HSD is the appropriate analysis for a follow up to the simple effects test because the simple effects test was just a one-way between-subjects ANOVA.

Remember that the Tukey HSD test is preferable to an independent sample t-test because it controls for multiple tests so that we maintain an overall alpha level equal to 0.05. That is, it prevents us from inflating our type I error rate.

Tables 8 and 9 present the results fo the Tukey HSD tests for treatement in the natural and enriched soil conditions, respectively.

Table 8

Tukey HSD Test of Treatment within Natural Soil

Comparison	Difference	95% CI LL	95% CI UL	Sig.
Sugar - MiracleGro	-5.67	-7.53	-3.80	<.001
Water - MiracleGro	-4.67	-6.63	-2.80	<.001
Water - Sugar	1.00	-0.87	2.87	.300

Table 9

Tukey HSD Test of Treatment within Enriched Soil

Comparison	Difference	95% CI LL	95% CI UL	Sig.
Sugar - MiracleGro	-8	-9.45	-6.55	<.001
Water - MiracleGro	-6	-7.45	-4.55	<.001
Water - Sugar	2.00	0.55	3.47	.013

It seems that we’ve found the important change across soil conditions. Whereas the differences among the treatments are all statistically significant (ps < .05) in the enriched soil condition, the difference btween water and sugar was not reliably different in the natural soil condition (p = .300).

It can be hard to interpret interaction effects through post hoc analyses alone. An interaction plot can help to reinforce what is expressed in these analyses. Figure 3 represents the interaction between soil and treatment on strawberry sweetness

Figure 3

Interaction Plot of Soil and Treatment on Sweetness

As we saw across tables 8 and 9, the difference in sweetness for strawberries grown in natural soil were significanct for MiracleGro and Sugar or Water but not between Sguar and Water. In contrast, all treatments yieleding significantly different sweetness ratings in the enriched soil.

Depending on the guiding hypotheses, you may want to perform your pairwise comparisons differently. Perhaps I am interested in the best overall approach to growing sweet strawberries. That is, perhaps I won’t to directly compare all conditions. If this is the case, we would want to put all combinations of the IVs into a Tukey HSD test. Table 10 presents the results of such a test.

Table 10

Tukey HSD Using All Pairwise Comparisons

Comparison	Difference	95% CI LL	95% CI UL	Sig.
Sugar(Enriched)-MiracleGro(Enriched)	-8.000	-9.828	-6.172	0.000
Water(Enriched)-MiracleGro(Enriched)	-6.000	-7.828	-4.172	0.000
MiracleGro(Natural)-MiracleGro(Enriched)	-2.667	-4.495	-0.838	0.004
Sugar(Natural)-MiracleGro(Enriched)	-8.333	-10.162	-6.505	0.000
Water(Natural)-MiracleGro(Enriched)	-7.333	-9.162	-5.505	0.000
Water(Enriched)-Sugar(Enriched)	2.000	0.172	3.828	0.029
MiracleGro(Natural)-Sugar(Enriched)	5.333	3.505	7.162	0.000
Sugar(Natural)-Sugar:(Enriched)	-0.333	-2.162	1.495	0.988
Water(Natural)-Sugar(Enriched)	0.667	-1.162	2.495	0.817
MiracleGro(Natural)-Water(Enriched)	3.333	1.505	5.162	0.001
Sugar(Natural)-Water(Enriched)	-2.333	-4.162	-0.505	0.010
Water(Natural)-Water(Enriched)	-1.333	-3.162	0.495	0.214
Sugar(Natural)-MiracleGro(Natural)	-5.667	-7.495	-3.838	0.000
Water(Natural)-MiracleGro(Natural)	-4.667	-6.495	-2.838	0.000
Water(Natural)-Sugar(Natural)	1.000	-0.828	2.828	0.480

As we expected from Figure 3, apply MiracleGro to strawberry plants in enriched soil lead to reliably sweeter strawberries than any other soil x treatment condition. We can tell this because all Tukey HSD post hoc tests involving the MiracleGro in enriched soil were statistically significant.

Using SPSS: GLM for the Factorial Between-Subjects ANOVA

Now that we have the necessary background, we can have SPSS perform a factorial between-subjects ANOVA, calculate pairwise comparisons, and produce plots.

The Data Set

Please download the “Motivation.sav” file from the “Datasets” folder in Canvas. Figure 4 is a section of the data view.

Figure 4

Data View of Motivation.sav

It seems that “reward” and “model” are nominal variables and “motivation” is a scale variable. Let’s find out more by checking the variable view. Figure 5 contains the relevant information from the variable view.

Figure 5

Variable View of Motivation.sav

We see that “external reward” has two levels (no reward vs. reward) and that “motivated model” also has to levels (model absent vs. model present). We also see that motivation is a reported score by the participant.

The Research Question

Given the data file and variables, we can surmise that the research question is: “Do the offer of an external reward and the presence of a motivated model affect reported motivation?”

This seems appropriate for a factorial between-subjects ANOVA because we have categorical independent variables and a continuous dependent variable. We will, however, also need to check our assumptions regarding normality and homogeneity of variance.

Checking Assumptions

Normality of DV for Groups

We’ll check for normality of the DV within each group by splitting the file by both IVs, then check our descriptive statistics.

Step 1 Split File

The assumption about normality of the DV needs to be checked the interaction level because we will be using means calculated for the DV at the intersection of the IVs. If we are using means, we require normality.

To get to the interaction level, we need to split our file by both IV. Go to the “Data” menu, then select “Split File.”

Choose “Compare groups” in the “Split File” window. You can then drag each IV (i.e., “reward” and “model”) to the “Groups based on” box (see Figure 6).

Figure 6

Split File Window

Click the “Paste” button to generate the syntax.

Step 2 Descriptive Statistics

We’ll now ask SPSS to provide us with some descriptive statistics related to our distributions to allow us to judge normality.

Go to the “Analyze” menu, hover over “Descriptive Statistics” then choose “Frequencies”.

Select our dependent variable “Reported Motivation” and move it to the “Variables(s)” box (see Figure 7).

Figure 7

Selecting Dependent Variable for Frequencies

Click the “Statistics” button to select the descriptives for calculation.

We’ll want to select mean and median for central tendency and select skewnesss and kurtosis from the “Characterize Posterior Distribution” area (see Figure 8). Click “Continue” to return to the main “Frequencies” window.

Figure 8

Selecting Descriptive Statistics

Turn off the “Display frequency tables” option in the main “Frequencies” window then click the “Paste” button to generate the syntax.

Step 4 Run the Syntax

Navigate to the syntax editor.

Add a new line below the existing syntax and type “SPLIT FILE OFF.” (be sure to put that “.” at the end of the statement). Your syntax should look like that in Figure 9.

Figure 9

Split File and Frequencies Syntax

Select the syntax and press the “run” buton.

Step 5 Interpret the Output

Navigate to the output window to see the statistics. Figure 10 shows the descriptive statistics.

Figure 10

Descriptive Statistics for Motivation Split by Reward and Model

Notice how closely the mean and median align within each grouping. You should also note that the skewness and kurtosis values are within acceptable ranges. As such, we can conclude that our samples of reported motivation are normal.

Equal Variance Across Groups

Although we will get a proper Levene’s test for equality of variances in the GLM procedure, we can approximate it visually using a simple error bar chart. Recall that we are looking for equal variance across groups so we want to produce a chart that represents variance for each group. We can then look for equal sizes across those representations.

Step 1 Open Chart Builder

Navigate to the “Chart Builder” through the “Graphs” menu.

Step 2 Select Cluster Error Bar from Gallery

Click on the “Bar” option and then double click the “Cluster Error Bar” template (see Figure 11).

Figure 11

Selecting Cluster Error Bar Chart

Step 3 Set Variables

This will be an interaction plot of sorts because we will want to include both IVs to compare the variance of the DV. Drag “motivation” to the y-axis. Drag “model” to the x-axis and “reward” to the “Cluster on X: set color” box in the top right of the preview window (see Figure 12)

Figure 12

Setting Variables in Clustered Error Bar Chart

Step 4 Set Error Bars

By default, the error bars are set to 95% CI. Although variance is incorporated in the calculation of these values, we would be better served by setting the error bars to 1 standard devation.

Click on any of the dots in the chart preview window. This will change the “Element Properties” window to reflect properties for the points. Select the “Standard deviation” option under “Error Bars Represent” then change the multiplier to 1 (see Figure 13).

Figure 13

Setting Error Bar Properties

Click the “Paste” button to generate the syntax.

Step 5 Run the Syntax

Navigate to the syntax editor then select and run the syntax associated with the figure (see Figure 14).

Figure 14

Syntax Associated with Clustered Error Bar Chart

Step 6 Interpret the Figure

If the output window did not open, navigate to it using the “window” menu. You should see a chart like that depicted in Figure 15.

Figure 15

Clustered Error Bar with 1 Standard Deviation Error Bars

We have a simple decision rule regarding homogeneity of variance. We should be concerned if any error bar is twice as large as any other error bar. In our example, it seems that the “Reward” with “Model Present” motivation scores may be less variable than the “Reward” with “Model Absent” motivation scores. We will need to check Levene’s test for any violations.

Setting Up the General Linear Model

It is time for us to set up our general linear model to test our factorial between-subjects ANOVA.

Step 1 Select the Univariate GLM

Although we have multiple IVs, we still only have one DV so we will choose the “Univariate…” option under the “General Linear Model” submenu in the “Analyze” menu.

Step 2 Assign Variables

The “Univariate” window will look familiar as we had used this for independent samples t-test and the one-way between-subjects ANOVA. In fact, whenever you have a fully between-subjects design (i.e., no within-subjects or repeated measures), you can use this GLM procedure.

Drag “motivation” to the “Dependent Variable” box. Then drag the two IVs (i.e., “reward” and “model”) to the “Fixed Factor(s)” box (see Figure 16).

Figure 16

Assigning Variables in Univariate GLM

Step 3 Create Line Charts

We’re going to ask SPSS to make three charts for our results. The first two will be line charts for our main effects. The third will be a line chart for the interaction effect.

Click on the “Plots” button to open the “Univariate: Profile Plots” window.

Main Effects Plots

To create the main effects plots, follow the procedure that we’ve followed in previous lessons.

Drag “reward” to the “Horizontal Axis” box. Click the “Add” button.

Repeat these steps for “model.”

Interaction Effect Plot

To create the interaction plot, you’ll want one IV in the “x-axis”Horizontal Axis" box and the other in the “Separate Lines” box. I’ve chosen “reward” and “model,” respectively. Click the “Add” button.

Setting Properties for Plots

With the data set for each plot, we’ll want to tell SPSS how to display the figures.

Click the “Line Chart” option under “Chart Type.” Lastly, Be sure to select “Include Error Bars. The default error bar option of”Confidence Interval (95.0%)" is what we want. See Figure 17 for the completed “Profile Plots” window.

Figure 17

Main and Interaction Effect Plots Set Up

Click the “Continue” button to return to the main “Univariate” window.

Step 4 Getting Means and Confidence Intervals

Plots are great for quick comparisons but we’ll want the actual values for our write-up. Let’s ask SPSS for all of the marginal and cell means (with confidence intervals) through the Estimated Marginal Means window. Click on the “EM Means” button.

We’ll want to drag over all of the effects (except OVERALL) from the “Factor(s) and Factor Interactions” box to the “Display Means for” box.

“Reward” will give us the marginal means of motivation scores for when the reward is present and when the reward is absent, averaging across the levels of “model.”

“Model” will give us the marginal means for the two levels of “model” while averaging across the levels of reward.

"Reward*model" will give us the cell means (i.e., the mean motivation score for each combination of factor levels).

Figure 18 show what the “Univariate: Estimated Marginal Means” window should look like.

Figure 18

Completed Estimated Marginal Means Window

Click “Continue” to return to the main “Univariate” window.

Step 5 Extra Options: Homogeneity Tests

Our clustered error bar chart suggested a possible violation for the assumption of equality of variance. We’ll ask SPSS for a formal Levene’s test by clicking the “Options” button then selecting the “Homogeneity test” option (see Figure 19).

Figure 19

Selecting Homogeneity Test Option

Click “Continue” to return to the main window.

Step 6 Post Hoc Tests

I’ve saved the post hoc tests for last because they should be the last step in your analytical procedure, if they are required. They may not be required for two reasons. You SHOULD NOT perform post hoc analyses if the effect in the ANOVA is not statistically significant. You do not need to perfom post hoc analyses for main effects that only have two levels.

In our example, we have a 2x2 ANOVA so we do not need any post hoc analyses for our main effects. We can simply view the estimated marginal means to report differences.

Main Effects

If your main effect does have more than two factors, you can follow the post hoc procedure as outlined in the one-way between-subjects ANOVA. That is, you would click the “Post Hoc” button, drag the IV (the one with a significant effect and more than two levels) from the “Factor(s)” area to the “Post Hoc Tests for” area. You’ll then select the “Tukey” option from the tests in the area below the variables area.

Interaction Effects

If you have a significant interaction effect, you need to break your factorial ANVOA into simpler models. I typically like to highlight what is driving the interaction effect. That is, I want to emphasize what is reliably impacting the change in the DV scores versus what is not.

To decide which follow up models to test, you either need to have some a priori hypothesis (reasoned before the data collection and analysis) or exploratory hypothesis (derived from the results of analyses). I suggest we look at our pattern of means in the interaction plot before deciding how we procedure with the post hoc analyses.

Click the “Paste” button in the main “Univariate” window to generate the syntax.

Step 7 Run the Model.

Navigate to the syntax window then select and run the GLM syntax as shown in Figure 20.

Figure 20

Syntax for the Univariate GLM

Interpreting the Output

Navigate once again to the output viewer. We’ll start interpreting our output by finishing the verification of our assumption of homogeneity of variance.

Levene’s Test for Equality of Variances

Scroll to Levene’s Test for the Equality of Variances (see Figure 21).

Figure 21

Levene’s Test for Equality of variances

When we check the “Sig.” value for the test “based on mean,” we find that it is larger than .05. As such, we do not have a violation of the assumption of homogeneity of variance.

We can now procedure to interpret the test of between-subjects effects table.

Test of Between-Subjects Effects

Figure 22 reproduces the test of between-subjets effects table.

Figure 22

Test of Between-Subjects Effects

I’ve drawn a rectangle around the two main effects and interaction effects. A quick glance at the “Sig.” column tells us that each of these effects is statistically significant.

We’ll want to interpret the interaction effect, however, because it contains the most detail about how all the levels of the IV impact the DV.

Line Charts

Scroll down to the last line chart in the output. It should be an interaction plot that looks like the one in Figure 23.

Figure 23

Interaction Plot of Reward and Model on Motivation

Interpreting an interaction plot can sometimes be tricky to do in a concise manner but you should always aim to be clear. That is, even if it takes two or three sentences, you want to describe how the levels of the IVs combine to change the DV.

Here are some things to note about this interaction plot.

1. Motivation scores are higher in the “reward” condition than in the “no reward” condition for both the “model present” and “model absent” conditions.

2. Motivation scores are higher for the “model present” condition than the “model absent” condition for both the “no reward” and “reward” conditions.

3. The difference in motivation scores between the “model present” and “model absent” conditions is larger for the “reward” than the “no reward” condition.

These three pieces of information (all derived from the same figure) tell us that there is a main effect of “reward”, a main effect of “model”, and an ordinal interaction between model and reward on motivation scores.

We might summarize the interaction effect like this:

‘A significant ordinal interaction revealed that the difference in motivation scores between the “model present” and “model absent” conditions is larger for the “reward” than the “no reward” condition. More specifically, whereas motivaitonal scores were higher for “model present” conditionx, the effect of the model was enhanced when coupled with an external reward.’

Post Hoc Analyses

We do not need post hoc analyses of the main effects because they each only have two levels. As such, with a significant F-value for the main effects, we know that there is a reliable difference between the two levels within each IV.

We could perform a post hoc pairwise comparison for any of the cell means (e.g., Reward and Model Present - No Reward and Model Absent). Unfortunately, the process for comparing across cells like this requires coding contrasts using syntax, which is beyond the scope of this course.

The good news is that we can utilize the supplement to null hypothesis significance testing (NHST). We can compare the 95% CI for the cell means to determine which are reliable different from others.

Using Confidence Intervals

Scroll to the third table in the “Estimated Marginal Means” section of the output. This table reports the means and 95% Confidence Interval bounds for motivation scores within each combination of the levels of the IVs (See Figure 24).

Figure 24

Cell Means and Confidence Interval Bounds for Interaction Effect

For any pairwise comparison we wish to make, we simply look for non-overlapping intervals between those rows. Let’s compare the “No Reward” + “Model Absent” condition to the “Reward” + “Model Present” Condition. Table 11 is a revised version of the table in Figure 24.

Table 11

Comparing 95% CI of Two Conditions

External Reward	Motivated Model	Mean	95% CI LL	95% CI UL
No Reward	Model Absent	34.55	32.315	36.785
Reward	Model Present	80.80	78.565	83.035

Across these two conditions, we see that the lower bound of the “Reward” + “Model Present” condition is higher than the upper bound of the “No Reward” + “Model Absent” condition. As such, we can conclude that these are statistically reliably different.

In fact, when we look at table in Figure 24, we see that all of the conditions had reliably different motivations scores because none of the confidence intervals overlap.

With our analyses and interpretation complete. It is time to write up the results.

Presenting the Results in APA Format

Our work now is to present the results in a clear and concise way to our readers. We need to describe and present the results from the factorial between-subjects ANOVA and we need to interpret the interaction effect using the line chart and the cell means with confidence intervals.

Styling the Line Chart

Let’s start with the line chart for our interaction effect. This is the first line chart that has had a grouping factor so we will have to ensure that we properly deal with the legend of the extra IV.

We need to do the following to get our line chart within APA guidelines.

1. Remove figure title.
2. Remove “Error bars: 95% CI” from below the figure.
3. Remove grid lines.
4. Update y-axis title to include DV.
5. Drag legend inside of chart area.
6. Add figure number above figure (in text).
7. Add figure title above figure (in text).
8. Add note below figure (in text) about error bars.

Figure 25 represents the final APA-styled figure.

Figure 25

Interaction Plot of Model and Reward on Motivation

Note. Error bars represent 95% CI.

Writing Up the Statistical Results

When writing up the statistical results, we always want to provide and understandable interpretation with the information from our results that lead to that interpretation. As such, I suggest we follow the formulat of Test + Interpretation + Statistical Summary.

ANOVA

The test of between-subjects effects is reporduced to aid the write-up.

Figure 22

Test of Between-Subjects Effects

‘A factorial between-subjects ANOVA was implemented through a univariate general linear model. The model revealed significant effects for reward (F[1,76]=807.057, p<.001), for model (F[1,76]=164.142, p<.001), and for the interaction of reward and model (F[1,76]=53.737, p<.001).’

Interaction Effect

When describing interaction effects, it is often beneficial to refer a figure or table that contains the details of that interaction. In this write up, we will refer to both the interaction plot and to the means table.

‘Figure 25 is a line chart representing the interaction effect. It suggests an ordinal interaction in which the difference in motivation scores between the “model present” and “model absent” conditions is larger for the “reward” than the “no reward” condition. More specifically, whereas motivaitonal scores were higher for “model present” condition, the effect of the model was enhanced when coupled with an external reward. Table 12 reveals no overlapping confidence intervals and thus that each sample of motivational scores were reliably different.’

Table 12

Cell Means and Confidence Intervals

External Reward	Motivated Model	Mean	Std. Error	95% Confidence Interval
				Lower Bound	Upper Bound
No Reward	Model Absent	34.550	1.122	32.315	36.785
	Model Present	40.700	1.122	38.465	42.935
Reward	Model Absent	58.200	1.122	55.965	60.435
	Model Present	80.800	1.122	78.565	83.035

When you are writing up results in the class worksheets, you do not need to reproduce a table (unless otherwise requested). You can just refer to the table in the output. For example, you could have written something like

‘The table of cell means and confidence intervals reveals no overlapping confidence intervals and thus that each sample of motivational scores were reliably different.’

Putting It Together

A factorial between-subjects ANOVA was implemented through a univariate general linear model. The model revealed significant effects for reward (F[1,76]=807.057, p<.001), for model (F[1,76]=164.142, p<.001), and for the interaction of reward and model (F[1,76]=53.737, p<.001). Figure 25 is a line chart representing the interaction effect. It suggests an ordinal interaction in which the difference in motivation scores between the “model present” and “model absent” conditions is larger for the “reward” than the “no reward” condition. More specifically, whereas motivaitonal scores were higher for “model present” condition, the effect of the model was enhanced when coupled with an external reward. Table 12 reveals no overlapping confidence intervals and thus that each sample of motivational scores were reliably different.

Summary

In this lesson, we’ve:

1. Extended the ANOVA approach to multiple variables,
2. Described interaction effects,
3. Explored post hoc analyses,
4. Set up and run a general linear model in SPSS for factorial between-subjects ANOVA,
5. Interpreted the results of the GLM, and
6. Shared the results in APA format.

In the next lesson, we’ll generalize the within-subjects ANOVA for factorial designs.