Difference between revisions of "Maximum (random variable)"

Revision as of 18:56, 26 November 2017 (view source) Alec (Talk | contribs) (Saving work)		Latest revision as of 18:57, 26 November 2017 (view source) Alec (Talk | contribs) m (Wrong categories)
Line 42:		Line 42:
	==Notes==		==Notes==
	<references group="Note"/>		<references group="Note"/>
−	{{Definition|Statistics|Probability Theory|Elementary Probability Theory}}	+	{{Definition|Statistics|Probability|Elementary Probability}}

---

Latest revision as of 18:57, 26 November 2017

$$\newcommand{\E}[1]{\mathbb{E}{\left[{#1}\right]} } \newcommand{\Max}[0]{\text{Max} } \newcommand{\ddx}[2]{\dfrac{\mathrm{d} }{\mathrm{d}x}\left[{#1}\right]\Big\vert_{#2} }$$

Definition notes

Let $X_1,\ \ldots,\ X_n$ be i.i.d random variables that are sampled from a distribution, $X$ and additionally let $M:\eq\Max(X_1,\ldots,X_n)$ for short.

• $$\P{\Max(X_1,\ldots,X_n)\le x}\eq\P{X_1\le x}\Pcond{X_2\le x}{X_1\le x}\cdots\Pcond{X_n\le x}{X_1\le x\cap\cdots\cap X_{n-1}\le x}$$

  $$\eq\prod^n_{i\eq 1}\P{X_i\le x}$$ - provided the $X_i$ are independent random variables

  $$\eq\prod^n_{i\eq 1}\P{X\le x}$$

  $$\eq \big(\P{X\le x}\big)^n$$

We shall call this $F'(x):\eq \big(\P{X\le x}\big)^n$ (and use $F(x):\eq \P{X\le x}$, as is usual for cumulative distribution functions) Caveat:Do not confuse the 's for derivatives, then:

• the probability density function, $$f'(x):\eq \frac{\mathrm{d} }{\mathrm{d} x}\big[F'(x)\big]\Big\vert_{x}$$ ^{[Note 1]} is:
  • $$f'(x)\eq\ddx{\big(\P{X\le x}\big)^n}{x}$$

    $$\eq\ddx{\big(F(x)\big)^n}{x}$$

    $$\eq n\big(F(x)\big)^{n-1}f(x)$$ by the chain rule, herein written $nF(x)^{n-1}f(x)$ for simplicity.

  • so $$f'(x)\eq nF(x)^{n-1}f(x)$$

Expectation of the maximum

• $$\E{M}:\eq\int^\infty_{-\infty} xf'(x)\mathrm{d}x$$

  $$\eq n\int_{-\infty}^\infty xf(x)F(x)^{n-1}\mathrm{d}x$$ - I wonder if we could use integration by parts or a good integration by substitution to clear this up

Special cases

• For $X\sim$$\text{Rect}$$($$[a,b]$$)$:
  • $$\E{\Max(X_1,\ldots,X_n)}\eq \frac{nb+a}{n+1}$$
    • This is actually simplified from the perhaps more useful $$a+\frac{n}{n+1}(b-a)$$ , recognising $(b-a)$ as the width of the uniform distribution we see that it slightly "under estimates" $a+(b-a)\eq b$, from this we can actually obtain a very useful unbiased estimator.

$a\eq 0$ case

Suppose that $a\eq 0$, then to find $b$ we could observe that the

$$\E{X}\eq\frac{b}{2}$$ , so 2x the average of our sample would have expectation $\eq b$ - this is indeed true.

However note in this case that the maximum has expectation

$$\E{M}\eq \frac{n}{n+1}b$$

• Thus: $$\frac{n+1}{n}\E{M}\eq b$$ and so $\E{\frac{n+1}{n}M}\eq b$

So defining: $M'\eq \frac{n+1}{n}M$ we do obtain an unbiased estimator for $b$ from our biased one

It can be shown that for $n\ge 8$ that $M'$ has lower variance (and thus is better) than the 2x average estimator, they agree for $n\eq 1$. For $2\le n\le 7$ 2x the average has the lower variance and is thus objectively better (or the same, from the $n\eq 1$ case) as an estimator for b when $n\le 7$ Warning:Only the following is known Alec (talk) 18:56, 26 November 2017 (UTC)

• This is only true when comparing the variance of $M$ (not $M'$) to that of $\frac{1}{n}\sum^n_{i\eq 1}X_i$, as we double the average, the variance would go up 4 times making the difference even worse.
• To obtain $M'$ we multiply $M$ by a small (specifically $\frac{n+1}{n}$) constant slightly bigger than 1 as $n$ stops being tiny, this will increase the variance by a factor of this value squared, which will still be "slightly bigger than 1" so $M'$ is a better estimator, it may however move the specific bound of "better for $n\ge 8$ further down probably - this needs to be calculated

This is independent of $b$

Specifically:

• $$\text{Var}(M)\eq \left(\frac{n}{n+2}-\frac{n^2}{(n+1)^2}\right)b^2$$ - note this is for $M$ not $M'$ and
• $$\text{Var}\left(\frac{1}{n}\sum^n_{i\eq 1}X_i\right)\eq\frac{b^2}{12n}$$

Notes

1. Jump up ↑ The $x$ inside the square brackets is bound to the $x$ at the base of the $\vert$