Chapter 10: Generating FunctionsContent check — Probability (Grinstead and Snell), Chapter 10: Generating Functions

Let $X$ be a continuous random variable with range $[\mathrm{0,1}]$ and density function $f_X(x)=1$ for $0\le x\le 1$ (uniform density). Then $${\mu }_n={\int }_0^1{x}^{n}dx=\frac{1}{n+1},$$ and

$$\begin{array}{cc}\hfill g(t)& =\sum _{k=0}^{\infty }\frac{t^k}{(k+1)!}\hfill \\ \hfill & =\frac{e^t-1}{t}.\hfill \end{array}$$

Here the series converges for all $t$⁠. Alternatively, we have

$$\begin{array}{cc}\hfill g(t)& ={\int }_{-\infty }^{+\infty }{e}^{tx}{f}_X(x)dx\hfill \\ \hfill & ={\int }_0^1{e}^{tx}dx=\frac{e^t-1}{t}.\hfill \end{array}$$

Then (by L’Hôpital’s rule)

$$\begin{array}{cc}\hfill {\mu }_0& =g(0)=\underset{t\to 0}{\lim }\frac{e^t-1}{t}=1,\hfill \\ \hfill {\mu }_1& =g^{\prime }(0)=\underset{t\to 0}{\lim }\frac{t{e}^t-e^t+1}{t^2}=\frac{1}{2},\hfill \\ \hfill {\mu }_2& =g^{\prime \prime }(0)=\underset{t\to 0}{\lim }\frac{t^3{e}^t-2t^2{e}^t+2t{e}^t-2t}{t^4}=\frac{1}{3}.\hfill \end{array}$$

In particular, we verify that $\mu =g^{\prime }(0)=1/2$ and $${\sigma }^2=g^{\prime \prime }(0)-{(g^{\prime }(0))}^2=\frac{1}{3}-\frac{1}{4}=\frac{1}{12}$$ as before (see Example 6.25).

(continuation of Example 6.20) If $X$ is uniformly distributed on $[0,1]$⁠, then, using Theorem 6.15, we have $$V(X)={\int }_0^1(x-\frac{1}{2}{)}^{2}dx=\frac{1}{12}.$$

Let $X$ be uniformly distributed on the interval $[0,1]$⁠. Then $$E(X)={\int }_0^{1}xdx=1/2.$$ It follows that if we choose a large number $N$ of random numbers from $[\mathrm{0,1}]$ and take the average, then we can expect that this average should be close to the expected value of 1/2.

If $X$ is a real-valued random variable with $E(X)=\mu$⁠, then (cf. Theorem 6.6) $$V(X)=E(X^2)-{\mu }^2.$$

If $X$ is any random variable with $E(X)=\mu$⁠, then $$V(X)=E(X^2)-{\mu }^2.$$ Proof. We have

$$\begin{array}{cc}\hfill V(X)& =E({(X-\mu )}^2)=E(X^2-2\mu X+{\mu }^2)\hfill \\ \hfill & =E(X^2)-2\mu E(X)+{\mu }^2=E(X^2)-{\mu }^2.\hfill \end{array}$$

Let $X$ have range $[0,\infty )$ and density function $f_X(x)=\lambda e^{-\lambda x}$ (exponential density with parameter $\lambda$⁠). In this case

$$\begin{array}{cc}\hfill {\mu }_n& ={\int }_0^{\infty }{x}^n\lambda e^{-\lambda x}dx=\lambda {(-1)}^n\frac{d^n}{d{\lambda }^n}{\int }_0^{\infty }{e}^{-\lambda x}dx\hfill \\ \hfill & =\lambda {(-1)}^n\frac{d^n}{d{\lambda }^n}[\frac{1}{\lambda }]=\frac{n!}{{\lambda }^n},\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill g(t)& =\sum _{k=0}^{\infty }\frac{{\mu }_k{t}^k}{k!}\hfill \\ \hfill & =\sum _{k=0}^{\infty }{[\frac{t}{\lambda }]}^k=\frac{\lambda }{\lambda -t}.\hfill \end{array}$$

Here the series converges only for $|t|<\lambda$⁠. Alternatively, we have

$$\begin{array}{cc}\hfill g(t)& ={\int }_0^{\infty }{e}^{tx}\lambda e^{-\lambda x}dx\hfill \\ \hfill & ={\frac{\lambda e^{(t-\lambda )x}}{t-\lambda }|}_0^{\infty }=\frac{\lambda }{\lambda -t}.\hfill \end{array}$$

Now we can verify directly that $${\mu }_n=g^{(n)}(0)={\frac{\lambda n!}{{(\lambda -t)}^{n+1}}|}_{t=0}=\frac{n!}{{\lambda }^n}.$$

Let $X$ be a continuous random variable with values in $[\mathrm{0,2}]$ and density $f_X$⁠. Find the moment generating function $g(t)$ for $X$ if

(a) $f_X(x)=1/2$⁠.

(b) $f_X(x)=(1/2)x$⁠.

(c) $f_X(x)=1-(1/2)x$⁠.

(d) $f_X(x)=|1-x|$⁠.

(e) $f_X(x)=(3/8)x^2$⁠.

Hint: Use the integral definition, as in Examples 10.15 and 10.16.

Let $X$ be a continuous random variable with range $[\mathrm{0,1}]$ and density function $f_X(x)=1$ for $0\le x\le 1$ (uniform density). Then $${\mu }_n={\int }_0^1{x}^{n}dx=\frac{1}{n+1},$$ and

$$\begin{array}{cc}\hfill g(t)& =\sum _{k=0}^{\infty }\frac{t^k}{(k+1)!}\hfill \\ \hfill & =\frac{e^t-1}{t}.\hfill \end{array}$$

Here the series converges for all $t$⁠. Alternatively, we have

$$\begin{array}{cc}\hfill g(t)& ={\int }_{-\infty }^{+\infty }{e}^{tx}{f}_X(x)dx\hfill \\ \hfill & ={\int }_0^1{e}^{tx}dx=\frac{e^t-1}{t}.\hfill \end{array}$$

Then (by L’Hôpital’s rule)

$$\begin{array}{cc}\hfill {\mu }_0& =g(0)=\underset{t\to 0}{\lim }\frac{e^t-1}{t}=1,\hfill \\ \hfill {\mu }_1& =g^{\prime }(0)=\underset{t\to 0}{\lim }\frac{t{e}^t-e^t+1}{t^2}=\frac{1}{2},\hfill \\ \hfill {\mu }_2& =g^{\prime \prime }(0)=\underset{t\to 0}{\lim }\frac{t^3{e}^t-2t^2{e}^t+2t{e}^t-2t}{t^4}=\frac{1}{3}.\hfill \end{array}$$

In particular, we verify that $\mu =g^{\prime }(0)=1/2$ and $${\sigma }^2=g^{\prime \prime }(0)-{(g^{\prime }(0))}^2=\frac{1}{3}-\frac{1}{4}=\frac{1}{12}$$ as before (see Example 6.25).

(continuation of Example 6.20) If $X$ is uniformly distributed on $[0,1]$⁠, then, using Theorem 6.15, we have $$V(X)={\int }_0^1(x-\frac{1}{2}{)}^{2}dx=\frac{1}{12}.$$

Let $X$ be uniformly distributed on the interval $[0,1]$⁠. Then $$E(X)={\int }_0^{1}xdx=1/2.$$ It follows that if we choose a large number $N$ of random numbers from $[\mathrm{0,1}]$ and take the average, then we can expect that this average should be close to the expected value of 1/2.

If $X$ is a real-valued random variable with $E(X)=\mu$⁠, then (cf. Theorem 6.6) $$V(X)=E(X^2)-{\mu }^2.$$

If $X$ is any random variable with $E(X)=\mu$⁠, then $$V(X)=E(X^2)-{\mu }^2.$$ Proof. We have

$$\begin{array}{cc}\hfill V(X)& =E({(X-\mu )}^2)=E(X^2-2\mu X+{\mu }^2)\hfill \\ \hfill & =E(X^2)-2\mu E(X)+{\mu }^2=E(X^2)-{\mu }^2.\hfill \end{array}$$

Let $X$ have range $[0,\infty )$ and density function $f_X(x)=\lambda e^{-\lambda x}$ (exponential density with parameter $\lambda$⁠). In this case

$$\begin{array}{cc}\hfill {\mu }_n& ={\int }_0^{\infty }{x}^n\lambda e^{-\lambda x}dx=\lambda {(-1)}^n\frac{d^n}{d{\lambda }^n}{\int }_0^{\infty }{e}^{-\lambda x}dx\hfill \\ \hfill & =\lambda {(-1)}^n\frac{d^n}{d{\lambda }^n}[\frac{1}{\lambda }]=\frac{n!}{{\lambda }^n},\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill g(t)& =\sum _{k=0}^{\infty }\frac{{\mu }_k{t}^k}{k!}\hfill \\ \hfill & =\sum _{k=0}^{\infty }{[\frac{t}{\lambda }]}^k=\frac{\lambda }{\lambda -t}.\hfill \end{array}$$

Here the series converges only for $|t|<\lambda$⁠. Alternatively, we have

$$\begin{array}{cc}\hfill g(t)& ={\int }_0^{\infty }{e}^{tx}\lambda e^{-\lambda x}dx\hfill \\ \hfill & ={\frac{\lambda e^{(t-\lambda )x}}{t-\lambda }|}_0^{\infty }=\frac{\lambda }{\lambda -t}.\hfill \end{array}$$

Now we can verify directly that $${\mu }_n=g^{(n)}(0)={\frac{\lambda n!}{{(\lambda -t)}^{n+1}}|}_{t=0}=\frac{n!}{{\lambda }^n}.$$