Maximum (random variable)

From Maths
Jump to: navigation, search
(Unknown grade)
This page is a stub
This page is a stub, so it contains little or minimal information and is on a to-do list for being expanded.The message provided is:
This page is currently little more than notes, beware. Alec (talk) 18:56, 26 November 2017 (UTC)
\newcommand{\P}[2][]{\mathbb{P}#1{\left[{#2}\right]} } \newcommand{\Pcond}[3][]{\mathbb{P}#1{\left[{#2}\!\ \middle\vert\!\ {#3}\right]} } \newcommand{\Plcond}[3][]{\Pcond[#1]{#2}{#3} } \newcommand{\Prcond}[3][]{\Pcond[#1]{#2}{#3} }
\newcommand{\E}[1]{\mathbb{E}{\left[{#1}\right]} } \newcommand{\Max}[0]{\text{Max} } \newcommand{\ddx}[2]{\dfrac{\mathrm{d} }{\mathrm{d}x}\left[{#1}\right]\Big\vert_{#2} }

Definition notes

Let X_1,\ \ldots,\ X_n be i.i.d random variables that are sampled from a distribution, X and additionally let M:\eq\Max(X_1,\ldots,X_n) for short.

  • \P{\Max(X_1,\ldots,X_n)\le x}\eq\P{X_1\le x}\Pcond{X_2\le x}{X_1\le x}\cdots\Pcond{X_n\le x}{X_1\le x\cap\cdots\cap X_{n-1}\le x}
    \eq\prod^n_{i\eq 1}\P{X_i\le x} - provided the X_i are independent random variables
    \eq\prod^n_{i\eq 1}\P{X\le x}
    \eq \big(\P{X\le x}\big)^n

We shall call this F'(x):\eq \big(\P{X\le x}\big)^n (and use F(x):\eq \P{X\le x} , as is usual for cumulative distribution functions) Caveat:Do not confuse the 's for derivatives, then:

  • the probability density function, f'(x):\eq \frac{\mathrm{d} }{\mathrm{d} x}\big[F'(x)\big]\Big\vert_{x} [Note 1] is:
    • f'(x)\eq\ddx{\big(\P{X\le x}\big)^n}{x}
      \eq\ddx{\big(F(x)\big)^n}{x}
      \eq n\big(F(x)\big)^{n-1}f(x) by the chain rule, herein written nF(x)^{n-1}f(x) for simplicity.
    • so f'(x)\eq nF(x)^{n-1}f(x)

Expectation of the maximum

  • \E{M}:\eq\int^\infty_{-\infty} xf'(x)\mathrm{d}x
    \eq n\int_{-\infty}^\infty xf(x)F(x)^{n-1}\mathrm{d}x - I wonder if we could use integration by parts or a good integration by substitution to clear this up

Special cases

  • For X\sim\text{Rect} ([a,b]):
    • \E{\Max(X_1,\ldots,X_n)}\eq \frac{nb+a}{n+1}
      • This is actually simplified from the perhaps more useful a+\frac{n}{n+1}(b-a), recognising (b-a) as the width of the uniform distribution we see that it slightly "under estimates" a+(b-a)\eq b, from this we can actually obtain a very useful unbiased estimator.

a\eq 0 case

Suppose that a\eq 0, then to find b we could observe that the \E{X}\eq\frac{b}{2} , so 2x the average of our sample would have expectation \eq b - this is indeed true.

However note in this case that the maximum has expectation \E{M}\eq \frac{n}{n+1}b

  • Thus: \frac{n+1}{n}\E{M}\eq b and so \E{\frac{n+1}{n}M}\eq b

So defining: M'\eq \frac{n+1}{n}M we do obtain an unbiased estimator for b from our biased one

It can be shown that for n\ge 8 that M' has lower variance (and thus is better) than the 2x average estimator, they agree for n\eq 1. For 2\le n\le 7 2x the average has the lower variance and is thus objectively better (or the same, from the n\eq 1 case) as an estimator for b when n\le 7 Warning:Only the following is known Alec (talk) 18:56, 26 November 2017 (UTC)

  • This is only true when comparing the variance of M (not M') to that of \frac{1}{n}\sum^n_{i\eq 1}X_i, as we double the average, the variance would go up 4 times making the difference even worse.
  • To obtain M' we multiply M by a small (specifically \frac{n+1}{n} ) constant slightly bigger than 1 as n stops being tiny, this will increase the variance by a factor of this value squared, which will still be "slightly bigger than 1" so M' is a better estimator, it may however move the specific bound of "better for n\ge 8 further down probably - this needs to be calculated

This is independent of b


Specifically:

  • \text{Var}(M)\eq \left(\frac{n}{n+2}-\frac{n^2}{(n+1)^2}\right)b^2 - note this is for M not M' and
  • \text{Var}\left(\frac{1}{n}\sum^n_{i\eq 1}X_i\right)\eq\frac{b^2}{12n}

Notes

  1. Jump up The x inside the square brackets is bound to the x at the base of the \vert