Английская Википедия:Empirical distribution function

Содержание

Текст

Текст

Текст

Текст

Текст

Текст

Текст

Текст

Текст

In statistics, an empirical distribution function (commonly also called an empirical cumulative distribution function, eCDF) is the distribution function associated with the empirical measure of a sample. This cumulative distribution function is a step function that jumps up by Шаблон:Math at each of the Шаблон:Math data points. Its value at any specified value of the measured variable is the fraction of observations of the measured variable that are less than or equal to the specified value.

The empirical distribution function is an estimate of the cumulative distribution function that generated the points in the sample. It converges with probability 1 to that underlying distribution, according to the Glivenko–Cantelli theorem. A number of results exist to quantify the rate of convergence of the empirical distribution function to the underlying cumulative distribution function.

Definition

Let Шаблон:Math be independent, identically distributed real random variables with the common cumulative distribution function Шаблон:Math. Then the empirical distribution function is defined as

<math>\widehat F_n(t) = \frac{\mbox{number of elements in the sample} \leq t}{n} = \frac{1}{n} \sum_{i=1}^n \mathbf{1}_{X_i \le t},</math>

where <math style="vertical-align:-.3em">\mathbf{1}_{A}</math> is the indicator of event Шаблон:Math. For a fixed Шаблон:Math, the indicator <math style="vertical-align:-.3em">\mathbf{1}_{X_i \le t}</math> is a Bernoulli random variable with parameter Шаблон:Math; hence <math style="vertical-align:-.3em">n \widehat F_n(t)</math> is a binomial random variable with mean Шаблон:Math and variance Шаблон:Math. This implies that <math style="vertical-align:-.3em">\widehat F_n(t)</math> is an unbiased estimator for Шаблон:Math.

However, in some textbooks, the definition is given as

<math>\widehat F_n(t) = \frac{1}{n+1} \sum_{i=1}^n \mathbf{1}_{X_i \le t}</math>

Mean

The mean of the empirical distribution is an unbiased estimator of the mean of the population distribution.

<math>E_n(X) = \frac{1}{n}\left (\sum_{i=1}^n{x_i}\right ) </math>

which is more commonly denoted <math>\bar{x}</math>

Variance

The variance of the empirical distribution times <math>\tfrac{n}{n-1}</math> is an unbiased estimator of the variance of the population distribution, for any distribution of X that has a finite variance.

<math>\begin{align}

\operatorname{Var}(X) &= \operatorname{E}\left[(X - \operatorname{E}[X])^2\right] \\[4pt] &= \operatorname{E}\left[(X - \bar{x})^2\right] \\[4pt] &= \frac{1}{n}\left (\sum_{i=1}^n{(x_i-\bar{x})^2}\right ) \end{align}</math>

Mean squared error

The mean squared error for the empirical distribution is as follows.

<math>

\begin{align} \operatorname{MSE}&=\frac{1}{n}\sum_{i=1}^n(Y_i-\hat{Y_i})^2\\[4pt] &=\operatorname{Var}_{\hat\theta}(\hat{\theta})+ \operatorname{Bias}(\hat{\theta},\theta)^2 \end{align}</math>

Where <math>\hat{\theta}</math> is an estimator and <math>\theta</math> an unknown parameter

Quantiles

For any real number <math>a</math> the notation <math>\lceil{a}\rceil</math> (read “ceiling of a”) denotes the least integer greater than or equal to <math>a</math>. For any real number a, the notation <math>\lfloor{a}\rfloor</math> (read “floor of a”) denotes the greatest integer less than or equal to <math>a</math>.

If <math>nq</math> is not an integer, then the <math>q</math>-th quantile is unique and is equal to <math>x_{(\lceil{nq}\rceil)}</math>

If <math>nq</math> is an integer, then the <math>q</math>-th quantile is not unique and is any real number <math>x</math> such that

<math>x_{({nq})}<x<x_{({nq+1})}</math>

Empirical median

If <math>n</math> is odd, then the empirical median is the number

<math>\tilde{x} = x_{(\lceil{n/2}\rceil)}</math>

If <math>n</math> is even, then the empirical median is the number

<math>\tilde{x} =\frac{ x_{n/2} + x_{n/2+1}}{2}</math>

Asymptotic properties

Since the ratio Шаблон:Math approaches 1 as Шаблон:Math goes to infinity, the asymptotic properties of the two definitions that are given above are the same.

By the strong law of large numbers, the estimator <math style="vertical-align:-.3em">\scriptstyle\widehat{F}_n(t)</math> converges to Шаблон:Math as Шаблон:Math almost surely, for every value of Шаблон:Math:

<math>

   \widehat F_n(t)\ \xrightarrow{\text{a.s.}}\ F(t);
 </math>

thus the estimator <math style="vertical-align:-.3em">\scriptstyle\widehat{F}_n(t)</math> is consistent. This expression asserts the pointwise convergence of the empirical distribution function to the true cumulative distribution function. There is a stronger result, called the Glivenko–Cantelli theorem, which states that the convergence in fact happens uniformly over Шаблон:Math:

<math>

   \|\widehat F_n-F\|_\infty \equiv
   \sup_{t\in\mathbb{R}} \big|\widehat F_n(t)-F(t)\big|\ \xrightarrow{\text{a.s.}}\ 0.
 </math>

The sup-norm in this expression is called the Kolmogorov–Smirnov statistic for testing the goodness-of-fit between the empirical distribution <math style="vertical-align:-.3em">\scriptstyle\widehat{F}_n(t)</math> and the assumed true cumulative distribution function Шаблон:Math. Other norm functions may be reasonably used here instead of the sup-norm. For example, the L²-norm gives rise to the Cramér–von Mises statistic.

The asymptotic distribution can be further characterized in several different ways. First, the central limit theorem states that pointwise, <math style="vertical-align:-.3em">\scriptstyle\widehat{F}_n(t)</math> has asymptotically normal distribution with the standard <math style="vertical-align:-.3em">\sqrt{n}</math> rate of convergence:

<math>

   \sqrt{n}\big(\widehat F_n(t) - F(t)\big)\ \ \xrightarrow{d}\ \ \mathcal{N}\Big( 0, F(t)\big(1-F(t)\big) \Big).
 </math>

This result is extended by the Donsker’s theorem, which asserts that the empirical process <math style="vertical-align:-.3em">\scriptstyle\sqrt{n}(\widehat{F}_n - F)</math>, viewed as a function indexed by <math style="vertical-align:-.3em">\scriptstyle t\in\mathbb{R}</math>, converges in distribution in the Skorokhod space <math style="vertical-align:-.3em">\scriptstyle D[-\infty, +\infty]</math> to the mean-zero Gaussian process <math style="vertical-align:-.3em">\scriptstyle G_F = B \circ F</math>, where Шаблон:Math is the standard Brownian bridge. The covariance structure of this Gaussian process is

<math>

   \operatorname{E}[\,G_F(t_1)G_F(t_2)\,] = F(t_1\wedge t_2) - F(t_1)F(t_2).
 </math>

The uniform rate of convergence in Donsker’s theorem can be quantified by the result known as the Hungarian embedding:

<math>

   \limsup_{n\to\infty} \frac{\sqrt{n}}{\ln^2 n} \big\| \sqrt{n}(\widehat F_n-F) - G_{F,n}\big\|_\infty < \infty, \quad \text{a.s.}
 </math>

Alternatively, the rate of convergence of <math style="vertical-align:-.3em">\scriptstyle\sqrt{n}(\widehat{F}_n-F)</math> can also be quantified in terms of the asymptotic behavior of the sup-norm of this expression. Number of results exist in this venue, for example the Dvoretzky–Kiefer–Wolfowitz inequality provides bound on the tail probabilities of <math style="vertical-align:-.3em">\scriptstyle\sqrt{n}\|\widehat{F}_n-F\|_\infty</math>:

<math>

   \Pr\!\Big( \sqrt{n}\|\widehat{F}_n-F\|_\infty > z \Big) \leq 2e^{-2z^2}.
 </math>

In fact, Kolmogorov has shown that if the cumulative distribution function Шаблон:Math is continuous, then the expression <math style="vertical-align:-.3em">\scriptstyle\sqrt{n}\|\widehat{F}_n-F\|_\infty</math> converges in distribution to <math style="vertical-align:-.3em">\scriptstyle\|B\|_\infty</math>, which has the Kolmogorov distribution that does not depend on the form of Шаблон:Math.

Another result, which follows from the law of the iterated logarithm, is that

<math>

   \limsup_{n\to\infty} \frac{\sqrt{n}\|\widehat{F}_n-F\|_\infty}{\sqrt{2\ln\ln n}} \leq \frac12, \quad \text{a.s.}
 </math>

and

<math>

   \liminf_{n\to\infty} \sqrt{2n\ln\ln n} \|\widehat{F}_n-F\|_\infty = \frac{\pi}{2}, \quad \text{a.s.}
 </math>

Confidence intervals

Файл:Empirical CDF, CDF and Confidence Interval plots for various sample sizes of Normal Distribution.png

Empirical CDF, CDF and confidence interval plots for various sample sizes of normal distribution

Файл:Cauchy emp .png

Empirical CDF, CDF and confidence interval plots for various sample sizes of Cauchy distribution

Файл:Triangle emp.png

Empirical CDF, CDF and confidence interval plots for various sample sizes of triangle distribution

As per Dvoretzky–Kiefer–Wolfowitz inequality the interval that contains the true CDF, <math>F(x)</math>, with probability <math>1-\alpha</math> is specified as

<math>F_n(x) - \varepsilon \le F(x) \le F_n(x) + \varepsilon \; \text{ where } \varepsilon = \sqrt{\frac{\ln{\frac{2}{\alpha}}}{2n}}.</math>

As per the above bounds, we can plot the Empirical CDF, CDF and confidence intervals for different distributions by using any one of the statistical implementations.

Statistical implementation

A non-exhaustive list of software implementations of Empirical Distribution function includes:

• In R software, we compute an empirical cumulative distribution function, with several methods for plotting, printing and computing with such an “ecdf” object.
• In MATLAB we can use Empirical cumulative distribution function (cdf) plot
• jmp from SAS, the CDF plot creates a plot of the empirical cumulative distribution function.
• Minitab, create an Empirical CDF
• Mathwave, we can fit probability distribution to our data
• Dataplot, we can plot Empirical CDF plot
• Scipy, we can use scipy.stats.ecdf
• Statsmodels, we can use statsmodels.distributions.empirical_distribution.ECDF
• Matplotlib, using the matplotlib.pyplot.ecdf function (new in version 3.8.0)
• Seaborn, using the seaborn.ecdfplot function
• Plotly, using the plotly.express.ecdf function
• Excel, we can plot Empirical CDF plot
• ArviZ, using the az.plot_ecdf function

See also

• Càdlàg functions
• Count data
• Distribution fitting
• Dvoretzky–Kiefer–Wolfowitz inequality
• Empirical probability
• Empirical process
• Estimating quantiles from a sample
• Frequency (statistics)
• Empirical likelihood
• Kaplan–Meier estimator for censored processes
• Survival function
• Q–Q plot

Further reading

• Шаблон:Cite book

External links

• Шаблон:Commons category-inline