- Types of degrees of freedom
- In a mechanical case
- In a set of random values
- Examples
- Variance and degrees of freedom
- In the Chi square distribution
- In the hypothesis test (With solved example)
- References
The degrees of freedom in statistics are the number of independent components of a random vector. If the vector has n components and there are p linear equations relating its components, then the degree of freedom is np.
The concept of degrees of freedom also appears in theoretical mechanics, where roughly they are equivalent to the dimension of space where the particle moves, minus the number of bonds.
Figure 1. A pendulum moves in two dimensions, but it only has one degree of freedom because it is forced to move in an arc of radius L. Source: F. Zapata.
This article will discuss the concept of degrees of freedom applied to statistics, but a mechanical example is easier to visualize in geometric form.
Types of degrees of freedom
Depending on the context in which it is applied, the way to calculate the number of degrees of freedom may vary, but the underlying idea is always the same: total dimensions less number of restrictions.
In a mechanical case
Let us consider an oscillating particle tied to a string (a pendulum) that moves in the vertical xy plane (2 dimensions). However, the particle is forced to move on the circumference of radius equal to the length of the chord.
Since the particle can only move on that curve, the number of degrees of freedom is 1. This can be seen in figure 1.
The way to calculate the number of degrees of freedom is by taking the difference of the number of dimensions minus the number of constraints:
degrees of freedom: = 2 (dimensions) - 1 (ligature) = 1
Another explanation that allows us to arrive at the result is the following:
-We know that the position in two dimensions is represented by a point of coordinates (x, y).
-But since the point must comply with the equation of the circumference (x 2 + y 2 = L 2) for a given value of the variable x, the variable y is determined by said equation or restriction.
In this way, only one of the variables is independent and the system has one (1) degree of freedom.
In a set of random values
To illustrate what the concept means, suppose the vector
x = (x 1, x 2,…, x n)
Representing the sample of n normally distributed random values. In this case the random vector x has n independent components and therefore x is said to have n degrees of freedom.
Let us now construct the vector r of the residuals
r = (x 1 -
Where
So the sum
(x 1 -
It is an equation that represents a restriction (or binding) in the elements of the vector r of the residues, since if n-1 components of the vector r are known, the restriction equation determines the unknown component.
Therefore the vector r of dimension n with the restriction:
∑ (x i -
It has (n - 1) degrees of freedom.
Again it is applied that the calculation of the number of degrees of freedom is:
degrees of freedom: = n (dimensions) - 1 (constraints) = n-1
Examples
Variance and degrees of freedom
The variance s 2 is defined as the mean of the square of the deviations (or residuals) of the sample of n data:
s 2 = (r • r) / (n-1)
where r is the vector of the residuals r = (x1 -
s 2 = ∑ (x i -
In any case, it should be noted that when calculating the average of the square of the residuals, it is divided by (n-1) and not by n, since as discussed in the previous section, the number of degrees of freedom of the vector r is (n-1).
If for the calculation of the variance it were divided by n instead of (n-1), the result would have a bias that is very significant for values of n less than 50.
In the literature, the variance formula also appears with the divisor n instead of (n-1), when it comes to the variance of a population.
But the set of the random variable of the residuals, represented by the vector r, although it has dimension n, only has (n-1) degrees of freedom. However, if the number of data is large enough (n> 500), both formulas converge to the same result.
Calculators and spreadsheets provide both versions of the variance and the standard deviation (which is the square root of the variance).
Our recommendation, in view of the analysis presented here, is to always choose the version with (n-1) each time it is required to calculate the variance or standard deviation, to avoid biased results.
In the Chi square distribution
Some probability distributions in continuous random variable depend on a parameter called degree of freedom, this is the case of the Chi square distribution (χ 2).
The name of this parameter comes precisely from the degrees of freedom of the underlying random vector to which this distribution applies.
Suppose we have g populations, from which samples of size n are taken:
X 1 = (x1 1, x1 2,…..x1 n)
X2 = (x2 1, x2 2,…..x2 n)
….
X j = (xj 1, xj 2,…..xj n)
….
Xg = (xg 1, xg 2,…..xg n)
A population j that has mean
The standardized or normalized variable zj i is defined as:
zj i = (xj i -
And the vector Zj is defined like this:
Zj = (zj 1, zj 2,…, zj i,…, zj n) and follows the standardized normal distribution N (0,1).
So the variable:
Q = ((z1 1 ^ 2 + z2 1 ^ 2 +…. + Zg 1 ^ 2),…., (Z1 n ^ 2 + z2 n ^ 2 +…. + Zg n ^ 2))
follows the χ 2 (g) distribution called the chi-square distribution with degree of freedom g.
In the hypothesis test (With solved example)
When you want to test hypotheses based on a certain set of random data, you need to know the number of degrees of freedom g in order to apply the Chi-square test.
Figure 2. Is there a relationship between the preference of ice cream FLAVOR and the customer's GENDER? Source: F. Zapata.
As an example, the data collected on the preferences of chocolate or strawberry ice cream among men and women in a certain ice cream parlor will be analyzed. The frequency with which men and women choose strawberry or chocolate is summarized in Figure 2.
First, the table of expected frequencies is calculated, which is prepared by multiplying the total of rows by the total of columns, divided by total data. The result is shown in the following figure:
Figure 3. Calculation of expected frequencies based on the observed frequencies (values in blue in figure 2). Source: F. Zapata.
Then the Chi square is calculated (from the data) using the following formula:
χ 2 = ∑ (F o - F e) 2 / F e
Where F o are the observed frequencies (Figure 2) and F e are the expected frequencies (Figure 3). The summation goes over all the rows and columns, which in our example give four terms.
After doing the operations you get:
χ 2 = 0.2043.
Now it is necessary to compare with the theoretical Chi square, which depends on the number of degrees of freedom g.
In our case, this number is determined as follows:
g = (# rows - 1) (#columns - 1) = (2 - 1) (2 - 1) = 1 * 1 = 1.
It turns out that the number of degrees of freedom g in this example is 1.
If you want to check or reject the null hypothesis (H0: there is no correlation between TASTE and GENDER) with a level of significance of 1%, the theoretical Chi-square value is calculated with degree of freedom g = 1.
The value is sought that makes the accumulated frequency (1 - 0.01) = 0.99, that is, 99%. This value (which can be obtained from the tables) is 6,636.
As the theoretical Chi exceeds the calculated one, then the null hypothesis is verified.
In other words, with the data collected, no relationship is observed between the variables TASTE and GENDER.
References
- Minitab. What are the degrees of freedom? Recovered from: support.minitab.com.
- Moore, David. (2009) Basic applied statistics. Antoni Bosch editor.
- Leigh, Jennifer. How to calculate degrees of freedom in statistical models. Recovered from: geniolandia.com
- Wikipedia. Degree of freedom (statistics). Recovered from: es.wikipedia.com
- Wikipedia. Degree of freedom (physical). Recovered from: es.wikipedia.com