Degrees of Freedom

Your life is made up of constraints on your time. When you go to work, how much time you spend studying, and the amount of sleep you need are all examples of constraints placed on you. You can think about how free you are in terms of how many constraints are placed upon you.

In statistics, there are constraints as well. The Chi Squared Tests use degrees of freedom to describe how free a test is based on the constraints placed on it. Read on to figure out how free the Chi Squared Test really is!

Degrees of freedom meaning

Many tests use degrees of freedom, but here you will see degrees of freedom as it relates to Chi Squared Tests. In general, the degrees of freedom is a way to measure how many test statistics you have calculated from the data. The more test statistics you have calculated using your sample, the less freedom you have to make choices with your data. Of course, there is a more formal way to describe these constraints as well.

Let's look at an example to see what that means in practice.

The number of constraints will also depend on the number of parameters you need to describe a distribution, and whether or not you know what these parameters are.

Next, let's look at how the constraints relate to degrees of freedom.

Degrees of freedom formula

For most cases, the formula

degrees of freedom = number of observed frequencies - number of constraints

can be used. If you go back to the example with the four sided die above, there was one constraint. The number of observed frequencies is \(4\) (the number of sides on the die. So the degrees of freedom would be \(4-1 = 3\).

There is a more general formula for the degrees of freedom:

degrees of freedom = number of cells (after combining) - number of constraints.

你可能想知道什么是细胞及其原因you might combine it. Let's look at an example.

You will usually only combine adjoining cells in your tables of data. Next, let's look at the official definition of degrees of freedom with the Chi-Squared distribution.

Degrees of freedom definition

If you have a random variable \(X\) and want to do an approximation for the statistic \(X^2\), you would use the \(\chi^2\) family of distributions. This is written as

\[\begin{align} X^2 &= \sum \frac{(O_t - E_t)^2}{E_t} \\ &= \sum \frac{O_t ^2}{E_t} -N \\ & \sim \chi^2, \end{align}\]

where \(O_t\) is the observed frequency, \(E_t\) is the expected frequency, and \(N\) is the total number of observations. Remember that the Chi-Squared tests are only a good approximation if none of the expected frequencies is below \(5\).

The \(\chi^2\) distributions are actually a family of distributions that depend on the degrees of freedom. The degrees of freedom for this kind of distribution are written using the variable \(\nu\). Since you may need to combine cells when using \(\chi^2\) distributions, you would use the definition below.

There will be cases where cells won't be combined, and in that case, you can simplify things a bit. If you go back to the four sided die example, there are \(4\) possibilities that could come up on the die, and these are the expected values. So for this example \(\nu = 4 - 1 = 3\) even if you are using a Chi-Squared distribution to model it.

To be sure you know how many degrees of freedom you have when using the Chi-Squared distribution, it is written as a subscript: \(\chi^2_\nu \).

Degrees of freedom table

Once you know that you are using a Chi-Squared distribution with \(\nu\) degrees of freedom, you will need to use a degrees of freedom table so that you can do hypothesis tests. Here is a section out of a Chi-Squared table.

Table 3. Chi-Squared table.

The first column of the table contains the degrees of freedom, and the first row of the table are areas to the right of the critical value.

The notation for a critical value of \(\chi^2_\nu\) which is exceeded with probability \(a\%\) is \(\chi^2_\nu(a\%)\) or\(\chi^2_\nu(a/100)\).

Let's take an example using the Chi-Squared table.

There is a second use for the table, as demonstrated in the next example.

Of course, a table can't list all of the possible values. If you need a value which is not in the table, there are many different statistics packages or calculators that can give you Chi-Squared table values.

Degrees of freedom t-test

The degrees of freedom in a \(t\)-test is calculated depending on if you are using paired samples or not. For more information on these topics, see the articlesT-distributionand Paired t-test.