划过的痕迹
martin-hxq · 2006-12-08
<< [转]一个冰淇淋的游戏
|
首 页
|
监考奇遇录 >>
2006-12-08
正态性检验
- [
研究/统计/方法
]
版权声明
:转载时请以超链接形式标明文章原始出处和作者信息及
本声明
http://martin-hxq.blogbus.com/logs/6467021.html
今天在spss交流群上遇到有人问为什么Explore里面的正态性检验结果和非参里面K-S的正态性检验结果不同。
Explore里面的正态性检验
非参里面的one sample K-S test
从上面两个图可以看到,Explore里面的检验显著而非参的没有。为什么会出现这样的情况呢?
结论是:Explore里面的其实是改良的K-S检验(就是进行了the Lilliefors correction,暂时没查到具体的计算方式),而非参里面是原汁原味的K-S检验。前者对分布比较敏感(就像LSD),后者相对保守、不敏感(就像Bonferroni)
。
那么对于这个例子是否意味着这个数据的分布是正态呢?因为后者保守嘛,也是经典算法
不放心,于是看看Skewness和Kurtonsis。
(.181, -.947)
和0很接近嘛,似乎是正态,似乎经典算法是对的。
正当我有点信心告诉别人很可能是正态的时候。对方发了茎叶图过来
1.00 3 . 0
1.00 4 . 0
12.00 5 . 000000000000
7.00 6 . 0000000
12.00 7 . 000000000000
6.00 8 . 000000
3.00 9 . 000
10.00 10 . 0000000000
垮了~~~ 看起来明显非正态
于是跟对方说,从茎叶图来看,Lilliefors test是正确的。确实应该拒绝零假设。
但是为了避免出现更多错误,我说,“看看数据的基本情况,看看图的样子和两种检验哪个的结果符合一些,然后确定一个标准 ”
当我开始写这个日志的时候,忽然从一类、二类错误的角度把这个问题看清一些。其实敏感就意味着一类错误可能性要大,因为容易错误拒绝;保守意味着二类错误可能性大,因为容易错误接受。象以上这个问题就是很明显的一、二类错误的抉择。在这种情况下,必须要多依靠别的参考数据,比如说峰-偏度(有书说这两个估计可能过分乐观,而且前者样本至少在1000以上后者200以上才比较可靠――据张厚粲的《现代心理教育统计学》)、p-p图、q-q图。至于K-S正态检验的话,只要样本够大,一些并不重要的偏移也可能造成拒绝正态的结果(就像t-test自由度问题)。另外explore里面的Shapiro-Wilks W test检验适合30-2000之间的样本
附一段评述,来自:
http://www.broadtarget.com/bs/archiver/?tid-5756.html
by.NewOccidental
Tests for Normality
The simplest method of assessing normality is to look at the frequency distribution histogram. The most important things to look at are the symetry and peakiness of the curve. In addition be aware of curves that indicate two or more peaks this would show a bimodal distribution and are not very friendly in statistics.
Visual appraisals must only be used as an indication of the distribution and subsequently better methods must be used. Values of skew and kurtosis as found in Excels Function Wizard (SKEW and KURT respectively) are another good indicator, but can be over optimistic regarding the datas match with normality. Before the advent of good computers and statistical programs, users could be forgiven for trying to avoid any surplus calculations. Now that both are available and much easier to use, tests for normality (and homogeneity of variance) should always be carried out as a best practice in statistics. SPSS and Minitab contain the Kolmogorov-Smirnov test, which is the principal goodness of fit test for normal and uniform data sets. Alternatively, if you are a whizz on the calculator or in Excel and have a day or two spare or have access to UNISTAT, you may wish to use the Shapiro-Wilk test which is more reliable when n<50.
Both of the above tests use the same hypotheses:
HO: there is no difference between the distribution of the data set and a normal one
HA: there is a difference between the distribution of the data set and normal
The P-value will be provided by SPSS or Minitab, if below 0.05 reject the HO.
Kurtosis is the peakedness of a distribution. A common rule-of-thumb test for normality is to run descriptive statistics to get skewness and kurtosis, then divide these by the standard errors. Kurtosis also should be within the +2 to -2 range when the data are normally distributed (a few authors use +3 to -3). Negative kurtosis indicates too many cases in the tails of the distribution. Positive kurtosis indicates too few cases in the tails. Note that the origin in computing kurtosis is 3 and a few statistical packages center on 3, but the foregoing discussion assumes that 3 has been subtracted to center on 0, as is done in SPSS and LISREL. The version with the normal distribution centered at 0 is Fisher kurtosis, while the version centered at 3 is Pearson kurtosis. SPSS uses Fisher kurtosis. Various transformations are used to correct kurtosis: cube roots and sine transforms may correct negative kurtosis. In SPSS 13, one of the places kurtosis is reported is under Analyze, Descriptive Statistics, Descriptives; click Options; select kurtosis.
Dichotomies. By definition, a dichotomy is not normally distributed. Many researchers will use dichotomies for procedures requiring a normal distribution as long as the split is less than 90:10.
Shapiro-Wilks W test is a formal test of normality offered in the SPSS EXAMINE module or the SAS UNIVARIATE procedure. This is the standard test for normality. W may be thought of as the correlation between given data and their corresponding normal scores, with W = 1 when the given data are perfectly normal in distribution. When W is significantly smaller than 1, the assumption of normality is not met. That is, a significant W statistic causes the researcher to reject the assumption that the distribution is normal. Shapiro-Wilks W is recommended for small and medium samples up to n = 2000. For larger samples, the Kolmogorov-Smirnov test is recommended by SAS and others. In SPSS 13, Shapiro-Wilks test is found under Analyze, Descriptive Statistics, Explore; select Both or Plots in the Display group; click Plots and select at least one plot.
Kolmogorov-Smirnov D test or K-S Lilliefors test, is an alternative test of normality for large samples, available in SPSS EXAMINE and SAS UNIVARIATE. This is sometimes called the Lilliefors test as a correction to K-S developed by Lilliefors is now normally applied. SPSS (as of Version 9), for instance, automatically applies the Lilliefors correction to the K-S test for normality in the EXAMINE module (but not in the NONPAR module). This test, with the Lilliefors correction, is preferred to the chi-square goodness-of-fit test when data are interval or near-interval. When applied without the Lilliefors correction, K-S is very conservative: that is, there is an elevated likelihood of a finding of non-normality. Note the K-S test can test goodness-of-fit against any theoretical distribution, not just the normal distribution. Be aware that when sample size is large, even unimportant deviations from normality may be technically significant by this and other tests. For this reason it is recommended to use other bases of judgment, such as frequency distributions and stem-and-leaf plots. In SPSS 13, Shapiro-Wilks test is found under Analyze, Descriptive Statistics, Explore; select Both or Plots in the Display group; click Plots and select at least one plot. The Kolmogorov-Smirnov test in SPSS 13, is found under Analyze, Descriptive Statistics, Explore; select Both or Plots in the Display group; click Plots and select at least one plot.
oxplot tests of the normality assumption: The SPSS boxplot output option (see the EXAMINE command in SPSS Base) produces charts in which the the Y axis is the interval dependent and categories of the independent are arrayed on the X axis. Inside the graph, for each X category, will be a rectangle indicating the spread of the dependents values for that category. If these rectangles are roughly at the same Y elevation for all categories, this indicates little difference among groups. Within each rectangle is a horizontal dark line, indicating the mean. If most of the rectangle is on one side or the other of the mean line, this indicates the dependent is skewed (not normal) for that group (category). Further out than the rectangle are the "whiskers," which mark the smallest and largest observations which are not outliers (defined as observations greater than 1.5 inter-quartile ranges [IQRs = boxlengths] from the 1st and 3rd quartiles). Note you can display boxplots for two factors (two independents) together by selecting Clustered Boxplots from the Boxplot item on the SPSS Graphs menu.
Graphical methods.
A histogram of a variable shows rough normality, and a histogram of residuals, if normally distributed, is often taken as evidence of normality of all the variables.
A graph of empirical by theoretical cumulative distribution functions (cdfs) simply shows the empirical distibution as, say, a dotted line, and the hypothetical distribution, say the normal curve, as a solid line.
A P-P plot is found in SPSS 13 under Graphs, P-P plots. One may test if the distribution of a given variable is normal (or beta, chi-square, exponential, gamma, half-normal, Laplace, Logistic, Lognormal, Pareto, Students t, Weibull, or uniform). he P=P plot plots a variables cumulative proportions against the cumulative proportions of the test distribution.The straighter the line formed by the P-P plot, the more the variables distribution conforms to the selected test distribution (ex., normal). Options within this SPSS procedure allow data transforms first (natural log, standardization of values, difference, and seasonally difference).
A quantile-by-quantile or Q-Q plot forms a 45-degree line when the observed values are in conformity with the hypothetical distribution. Q-Q plots plot the quantiles of a variables distribution against the quantiles of the test distribution.From the SPSS menu, select Graphs, Q-Q. The SPSS dialog box supports testing the following distributions: beta, chi-square, exponential, gamma, half-normal, Laplace, Logistic, Lognormal, normal, pareto, Students t, Weibull, and uniform.
随机文章:
第二次讨论会--MMPI & 偏态数据
2006-10-27
11.3-11.7 开封 高级统计方法和现代测验理论原理及应用研讨班
2007-08-21
均数上面的一横
2007-04-05
[书评]下流社会-符合我刻板印象的社会学书
2008-02-06
精神分裂症的人特别聪明吗?
2008-02-05
收藏到:
Del.icio.us
Tag:
心理学
数学
统计
正态性检验
正态
引用地址:
martin_hxq
发表于
00:00:00
|
编辑
|
继续话题
|
转发
|
推荐
发表评论
原始快照:web.archive.org