Maximize an Integral involving a probability distribution

Using the inequality $e^z \geq 1+z$ , for any function $p(x)$ satisfying the given properties, we have $\ln\frac{p(x)}{4 e^{-4x}} \leq \frac{p(x)}{4e^{-4x}}-1$ Multiplying both sides by $4e^{-4x}$ , we have $4 e^{-4x} \ln\frac{p(x)}{4 e^{-4x}} \leq p(x)-4e^{-4x}$ Integrating both sides in the interval $[0,\infty)$ , we have $\int_{0}^{\infty} 4 e^{-4x} \ln\frac{p(x)}{4 e^{-4x}} dx \leq \int_{0}^{\infty}\big( p(x)-4e^{-4x}\big) dx=1-1=0$ i.e., $4\int_{0}^{\infty} e^{-4x} \ln(p(x)) dx \leq \int_{0}^{\infty} 4 e^{-4x} \ln (4 e^{-4x})dx= \ln(4)-1$ Thus, we have the following upper-bound on $I_p$ $I_p \leq \frac{1}{4} (\ln(4)-1) \hspace{15pt} (*)$ By tracing the above series of inequalities backwards, we see that the upper-bound is achievable iff we have equality in the inequality we started with, i.e., $\frac{p(x)}{4 e^{-4x}}=1, \forall x \in [0,\infty)$ . i.e., the Exponential Distribution $p(x)=4e^{-4x}, x\geq 0$ achieves the upper-bound (*) on $I_p$ .

I did a discrete approach. Let us define the function $p_L(x)$ as $p_n$ , if $nL \leq x < (n+1)L$ .

Then, we must maximize $\begin{aligned} I_{p_{L}} &= \sum_{n=0}^{+\infty} \int_{nL}^{(n+1)L} e^{-4x} \ln(p_n) \\ &= \dfrac{1}{4}(1-e^{4L}) \sum_{n=0}^{+\infty} e^{-4nL}\ln(p_n) \end{aligned},$

with the constraint, $\sum_{n=0}^{+\infty} p_n L = 1.$

We can solve this problem using Lagrange multipliers. Writing the Lagrangian,

$\mathcal{L} = \dfrac{1}{4}(1-e^{4L}) \sum_{n=0}^{+\infty} e^{-4nL}\ln(p_n) + \lambda \left( \sum_{n=0}^{+\infty} p_n L - 1 \right),$

we know that $\dfrac{\partial \mathcal{L}}{\partial p_n} = 0$ , i.e,

$\begin{aligned} & \dfrac{1}{4} (1-e^{-4L}) e^{-4nL}\dfrac{1}{p_n} + \lambda L = 0\\ \Rightarrow & p_n = -\dfrac{1-e^{-4L}}{4\lambda L} e^{-4nL}. \end{aligned}$

Now we use the constraint to determine $\lambda$ ,

$\begin{aligned} & \sum_{n=0}^{+\infty} -\dfrac{1-e^{-4L}e^{-4nL}}{4\lambda L}L = 1 \\ \Rightarrow & \lambda = - \dfrac{1-e^{-4L}}{4}\sum_{n=0}^{+\infty} e^{-4nL} = -\dfrac{1}{4}. \\ \end{aligned}$

Therefore, $p_n = \dfrac{1-e^{-4L}}{L} e^{-4nL}$ .

Now we use the function $p_L(x)$ as we defined and make $L \to 0$ . This way, we find that $p(x) = 4 e^{-4x},$ which is continuous. Note that $p_L(x)$ is not continuous, so when we take the $\lim_{L \to 0} p_L(x) = p(x)$ , we may not necessarily find a continuous function $p(x)$ .

Finally, it is easy to calculate $I_p$ and we find $\dfrac{1}{4}(\ln(4) - 1)$ .

If $p(x)$ maximises $I_p$ then any change to $p(x)$ will either decrease or not change $I_p$ .

So, we define a function: $p_2(x)=\begin{cases}p(x)-dy, & \text{if } x=\sigma \\ p(x)+dy, & \text{if } x=d \\ p(x), & \text{otherwise}\end{cases}$

Note that $\int_{0}^{\infty} p(x)\,dx=\int_{0}^{\infty} p_2(x)\,dx$ .

Now we look at the relationship between $I_p$ and $I_{p_2}$ : $I_{p_2}=I_p+dx[e^{-4\sigma}(\ln(p(\sigma)-dy)-\ln(\sigma))+e^{-4d}(\ln(p(d)+dy)-\ln(d))] = I_p+dx[-e^{-4\sigma}\frac{dy}{p(\sigma)}+e^{-4d}\frac{dy}{p(d)}]$

Given our condition of $I_p$ being maximal, we know that $I_{p_2}-I_p\leq0$ . This implies that: $dy dx[-\frac{e^{-4\sigma}}{p(\sigma)}+\frac{e^{-4d}}{p(d)}]\leq0$

Since the sign of $dy dx$ is not specified, we have that: $-\frac{e^{-4\sigma}}{p(\sigma)}+\frac{e^{-4d}}{p(d)}=0$ $e^{4\sigma}p(\sigma)=e^{4d}p(d)$

Since the two sides are independent, we have that: $e^{4\sigma}p(\sigma)=\lambda=e^{4d}p(d)$ $p(\sigma)=\lambda e^{-4\sigma}$

Now we solve for $\lambda$ : $\int_{0}^{\infty} p(x)\,dx = \int_{0}^{\infty} \lambda e^{-4x}\,dx = \lambda/4 = 1$ $\lambda=4$

Substituting back into $I_p$ we get: $I_p = \int_{0}^{\infty} e^{-4x}\ln(4 e^{-4x})\,dx = \int_{0}^{\infty} e^{-4x}(\ln(4) -4x)\,dx = \boxed{\frac{1}{4}(\ln{4}-1)}$