Edit

Edited by Flickety, Ben Rubenstein, Brandywine, Brandon and 4 others

Assess Statistical Significance

Statistical significance is the number, called a p-value, that tells you the probability of your result being observed, given that a certain statement (the null hypothesis) is true. If this p-value is sufficiently small, the experimenter can safely assume that the null hypothesis is false.

Edit Steps

  1. 1
    Determine the experiment that you wish to perform and what you'd like to learn from it. For this example, let us consider that you have purchased a piece of wood from a lumber store. You purchased a piece of wood that the seller told you was 8 feet long (let us denote this L=8). You think the seller is cheating you, and believe that the length of the wood is actually less than 8 feet (L<8). This is called your alternate hypothesis HA.

  2. 2
    State your null hypothesis. In order to show that L<8, we must show that it is unreasonable to believe L>=8 given the data we have observed. Therefore, we state our null hypothesis is that the length of the board is greater than or equal to 8 feet, or H0: L>=8.

  3. 3
    Determine how unusual your data must be before it can be considered significant. Many statisticians consider being 95% sure that the null hypothesis is false as a minimum for statistical significance (giving a p-value of 0.05). This is the confidence level. A higher confidence level (and, thus, a lower p-value) means the results are more significant. Notice that a 95% confidence level means that you are incorrect 1 time out of every 20 times you conduct the experiment.

  4. 4
    Collect data. Most of us would break out the tape measure, find that the length of the board is less than 8 feet, and then go ask the lumber dealer for a new piece of wood. However, science requires a much larger burden of proof than just a single measurement. For example, what if you are particularly poor with a measuring tape? Instead, we must make many measurements and use those results to determine our p-value.

  5. 5
    Calculate the mean (average) of your data. We will denote this μ.

     
    1. Add all your observations together
    2. Dividing by the number of measurements you have made (n).
  6. 6
    Calculate the sample standard deviation. We will denote this s.

     
    1. Subtracting the mean μ from all your observations.
    2. Square the resulting values.
    3. Add them together.
    4. Divide by n-1.
    5. Take the square root of this value.
  7. 7
    Convert your mean to a standard Normal value (Z score). We will denote this Z.

     
    1. Subtract the value from H0 (8) from your observed mean μ.
    2. Divide the result by the sample standard deviation s.
  8. 8
    Compare this value Z with the Z value for your confidence level. This value comes from a table of the Normal distribution. Determining this critical value is beyond the scope of this article, but if your calculated Z value is more negative than -1.645, then you can assume that your board has a length less than 8 feet with greater than 95% confidence. This is called rejecting the null hypothesis. This means that your observed μ is statistically significant (in that it is different from the stated length). If your observed Z is not less than -1.645, then you cannot reject H0. In this case, note that you have not proven that H0 is true. You simply don't have enough information to say that it is false.

  9. 9
    Consider a follow up study. Doing another study, with more measurements or a more accurate measuring device, will help increase your confidence about your conclusion.


Edit Video

Edit Tips

  • Statistics is a large and complicated field, take a high school or college level (or beyond) course on statistical inference to help understand statistical significance.

Edit Warnings

  • This analysis is specific to the example given. It will change depending on your hypothesis.
  • We have made a number of assumptions that are not discussed. A statistics class will help you understand these.

Edit Related wikiHows

Article Info

Categories: Science

Recent edits by: HKristineWhite, Teresa, Chris Hadley

Share this Article:

Thanks to all authors for creating a page that has been read 58,948 times.

Was this article accurate?

Yes No


x

Thank Our Volunteer Authors.

Follow us on Google+