Maximum (random variable)
Contents
[hide]Definition notes
Let X_1,\ \ldots,\ X_n be i.i.d random variables that are sampled from a distribution, X and additionally let M:\eq\Max(X_1,\ldots,X_n) for short.
- \P{\Max(X_1,\ldots,X_n)\le x}\eq\P{X_1\le x}\Pcond{X_2\le x}{X_1\le x}\cdots\Pcond{X_n\le x}{X_1\le x\cap\cdots\cap X_{n-1}\le x}
- \eq\prod^n_{i\eq 1}\P{X_i\le x} - provided the X_i are independent random variables
- \eq\prod^n_{i\eq 1}\P{X\le x}
- \eq \big(\P{X\le x}\big)^n
We shall call this F'(x):\eq \big(\P{X\le x}\big)^n (and use F(x):\eq \P{X\le x} , as is usual for cumulative distribution functions) Caveat:Do not confuse the 's for derivatives, then:
- the probability density function, f'(x):\eq \frac{\mathrm{d} }{\mathrm{d} x}\big[F'(x)\big]\Big\vert_{x} [Note 1] is:
- f'(x)\eq\ddx{\big(\P{X\le x}\big)^n}{x}
- \eq\ddx{\big(F(x)\big)^n}{x}
- \eq n\big(F(x)\big)^{n-1}f(x) by the chain rule, herein written nF(x)^{n-1}f(x) for simplicity.
- so f'(x)\eq nF(x)^{n-1}f(x)
- f'(x)\eq\ddx{\big(\P{X\le x}\big)^n}{x}
Expectation of the maximum
- \E{M}:\eq\int^\infty_{-\infty} xf'(x)\mathrm{d}x
- \eq n\int_{-\infty}^\infty xf(x)F(x)^{n-1}\mathrm{d}x - I wonder if we could use integration by parts or a good integration by substitution to clear this up
Special cases
- For X\sim\text{Rect} ([a,b]):
- \E{\Max(X_1,\ldots,X_n)}\eq \frac{nb+a}{n+1}
- This is actually simplified from the perhaps more useful a+\frac{n}{n+1}(b-a), recognising (b-a) as the width of the uniform distribution we see that it slightly "under estimates" a+(b-a)\eq b, from this we can actually obtain a very useful unbiased estimator.
- \E{\Max(X_1,\ldots,X_n)}\eq \frac{nb+a}{n+1}
a\eq 0 case
Suppose that a\eq 0, then to find b we could observe that the \E{X}\eq\frac{b}{2} , so 2x the average of our sample would have expectation \eq b - this is indeed true.
However note in this case that the maximum has expectation \E{M}\eq \frac{n}{n+1}b
- Thus: \frac{n+1}{n}\E{M}\eq b and so \E{\frac{n+1}{n}M}\eq b
So defining: M'\eq \frac{n+1}{n}M we do obtain an unbiased estimator for b from our biased one
It can be shown that for n\ge 8 that M' has lower variance (and thus is better) than the 2x average estimator, they agree for n\eq 1. For 2\le n\le 7 2x the average has the lower variance and is thus objectively better (or the same, from the n\eq 1 case) as an estimator for b when n\le 7 Warning:Only the following is known Alec (talk) 18:56, 26 November 2017 (UTC)
- This is only true when comparing the variance of M (not M') to that of \frac{1}{n}\sum^n_{i\eq 1}X_i, as we double the average, the variance would go up 4 times making the difference even worse.
- To obtain M' we multiply M by a small (specifically \frac{n+1}{n} ) constant slightly bigger than 1 as n stops being tiny, this will increase the variance by a factor of this value squared, which will still be "slightly bigger than 1" so M' is a better estimator, it may however move the specific bound of "better for n\ge 8 further down probably - this needs to be calculated
This is independent of b
Specifically:
- \text{Var}(M)\eq \left(\frac{n}{n+2}-\frac{n^2}{(n+1)^2}\right)b^2 - note this is for M not M' and
- \text{Var}\left(\frac{1}{n}\sum^n_{i\eq 1}X_i\right)\eq\frac{b^2}{12n}