t-test

Explanations > Social Research > Analysis > t-test

Description | Example | Discussion | See also

Description

The t-test (or student's t-test) gives an indication of the separateness of two sets of measurements, and is thus used to check whether two sets of measures are essentially different (and usually that an experimental effect has been demonstrated). The typical way of doing this is with the null hypothesis that means of the two sets of measures are equal.

The t-test assumes:

A normal distribution (parametric data)
Underlying variances are equal (if not, use Welch's test)

It is used when there is random assignment and only two sets of measurement to compare.

There are two main types of t-test:

Independent-measures t-test: when samples are not matched.
Matched-pair t-test: When samples appear in pairs (eg. before-and-after).

A single-sample t-test compares a sample against a known figure, for example where measures of a manufactured item are compared against the required standard.

Calculation

The value of t may be calculated using packages such as SPSS. The actual calculation for two groups is:

t = experimental effect / variability

= difference between group means /
standard error of difference between group means

Interpretation

The resultant t-value is then looked up in a t-table to determine the probability that a significant difference between the two sets of measures exists and hence what can be claimed about the efficacy of the experimental treatment.

Effect

The t-value can also be converted to a Pearson r-value to measure effect, which can be calculated as:

r = SQRT( t² / (t² + DF))

where DF is the degrees of freedom.

In a t-test, DF = N₁ + N₂ - 2.

Reporting

Reporting a t-test might look something like this:

On average, the reported relationship between holidays in the south (M=24.1, SE=1.5) were significantly preferred than holidays in the north (M=20.1, SE=1.2), t(22)=2.3, p<.05, r=.44.

In this, 'M' is the mean and 'SE' the standard error of each sample. In 't(X)=Y', X is the degrees of freedom and Y is the t-metric. 'p' is the probability of a type-1 error and 'r' is the effect.

Discussion

The t-test was described by 1908 by William Sealy Gosset for monitoring the brewing at Guinness in Dublin. Guinness considered the use of statistics a trade secret, so he published his test under the pen-name 'Student' -- hence the test is now often called the 'Student's t-test'.

The t-test is a basic test that is limited to two groups. For multiple groups, you would have to compare each pair of groups, for example with three groups there would be three tests (AB, AC, BC), whilst with seven groups there would need to be 21 tests.

The basic principle is to test the null hypothesis that the means of the two groups are equal.

A significant problem with this is that we typically accept significance with each t-test of 95% (p=0.05). For multiple tests these accumulate and hence reduce the validity of the results.