Down the hill

Let us consider only the positive- $x$ portion of the graph, for simplicity. The coordinates and velocities are:

$x = x \\ y = -x^n \\ \dot{x} = \dot{x} \\ \dot{y} = -n \, x^{n-1} \, \dot{x} \\ v^2 = \dot{x}^2 + \dot{y}^2 = \dot{x}^2 (1 + n^2 \, x^{2n - 2} )$

The kinetic energy is:

$E = \frac{1}{2} m v^2 = \frac{1}{2} m \dot{x}^2 (1 + n^2 \, x^{2n - 2} )$

The potential energy is (relative to $y = 0$ ):

$U = m g y = -m g \, x^n$

System Lagrangian:

$L = E - U = \frac{1}{2} m \dot{x}^2 (1 + n^2 \, x^{2n - 2} ) + m g \, x^n$

Equation of motion:

$\frac{d}{dt} \frac{\partial{L}}{\partial{\dot{x}}} =\frac{\partial{L}}{\partial{x}}$

Evaluation step one:

$\frac{\partial{L}}{\partial{\dot{x}}} = m \dot{x} (1 + n^2 \, x^{2n - 2} )$

Evaluation step two:

$\frac{d}{dt} \frac{\partial{L}}{\partial{\dot{x}}} = m \ddot{x} (1 + n^2 \, x^{2n - 2} ) + m \dot{x} [n^2 (2n - 2) \, x^{2n - 3} \dot{x}] \\ = m \ddot{x} (1 + n^2 \, x^{2n - 2} ) + m \dot{x}^2 \, n^2 (2n - 2) \, x^{2n - 3}$

Evaluation step three:

$\frac{\partial{L}}{\partial{x}} = \frac{1}{2} m \dot{x}^2 \, n^2 (2n-2) \, x^{2n - 3} + m g n \, x^{n-1}$

Equating:

$m \ddot{x} (1 + n^2 \, x^{2n - 2} ) + m \dot{x}^2 \, n^2 (2n - 2) \, x^{2n - 3} = \frac{1}{2} m \dot{x}^2 \, n^2 (2n-2) \, x^{2n - 3} + m g n \, x^{n-1} \\ \ddot{x} (1 + n^2 \, x^{2n - 2} ) = -\frac{1}{2} \dot{x}^2 \, n^2 (2n-2) \, x^{2n - 3} + g n \, x^{n-1}$

Horizontal acceleration:

$\ddot{x} = \frac{-\frac{1}{2} \dot{x}^2 \, n^2 (2n-2) \, x^{2n - 3} + g n \, x^{n-1} }{1 + n^2 \, x^{2n - 2}} \\ = \frac{-\frac{1}{2} \dot{x}^2 \, n^2 (2n-2) \, x^{2n - 3} }{1 + n^2 \, x^{2n - 2}} + \frac{ g n \, x^{n-1} }{1 + n^2 \, x^{2n - 2}}$

Applying conservation of energy:

$m g \, x^n = \frac{1}{2} m \dot{x}^2 (1 + n^2 \, x^{2n - 2} ) \\ \dot{x}^2 = \frac{2 g x^n}{1 + n^2 \, x^{2n-2}}$

Substituting back in:

$\ddot{x} = \frac{g \, n^2 (2 - 2n) \, x^{3n - 3} }{[1 + n^2 \, x^{2n - 2}]^2} + \frac{ g n \, x^{n-1} }{1 + n^2 \, x^{2n - 2}}$

The key point now is that since the mouse is sliding on top of the hill, the hill can only supply a horizontal force in the positive (rightward) direction. So in order for the mouse to remain on the hill indefinitely, the horizontal (x) acceleration must never be negative. This also means that the horizontal acceleration must never be zero, since it presumably starts out positive.

Setting the horizontal acceleration equal to zero yields the following:

$(n^2 - 2n) \, x^{2n - 2} = 1$

The equation has solutions only for $n > 2$ . This indicates that the mouse only remains on the hill forever for $n \leq 2$

Alak Bhattacharya used the "radius of curvature", a non-trivial result from differential geometry. Here is a solution that does not use that concept.

Let us change the direction of the $y$ axis to point downwards, and let us look at values of positive $x$ only. The function and its time derivatives are

$y=x^n$ (1)

$\dot y=y'\dot x$ (2)

$\ddot y=y''\dot x^2+y'\ddot x$ (3)

Assume that the mass remains on this curve. Our goal is to test if the horizontal component of the acceleration is positive or negative. For negative values our assumption is wrong and the mouse separates from the slope.

According to Newton's law $mg\cos \alpha= ma$ , where $mg \cos \alpha$ is the tangential component of the force of gravity and $\alpha$ is the angle between the vertical and the tangent to the slope. Therefore the vertical component of the acceleration is $\ddot y=g\cos^2 \alpha$ . Since $\tan \alpha =dx/dy=1/y'$ and $\cos^2 \alpha=\frac{1}{1+\tan^2 \alpha}$ we get

$\ddot y=\frac{g}{1+1/y'^2}$ (4)

Energy conservation yields $\frac {1}{2} m(\dot x^2+\dot y^2)=mgy$ or, using Eq. (2)

$\dot x^2=2gy-\dot y^2= \frac {2gy}{1+y'^2}$ (5)

We substitute Eq (4) and Eq. (5) to Eq. (3) to get

$\frac{g}{1+1/y'^2}= y''\frac {2gy}{1+y'^2}+y'\ddot x$ , or

$\frac{y'\ddot x}{g}=\frac{1}{1+1/y'^2}- y''\frac {2y}{1+y'^2}$

We re-arrange this to

$\frac{y' (1+y'^2)}{g}\ddot x=y'^2- 2yy''$ (6)

For $n=1$ , $y'^2- 2yy''=1$ , and the orbit is stable. For $n=2$ , $y'^2- 2yy''=0$ , and the orbit is (marginally) stable. For $n=3$ , $y'^2- 2yy''=-2x^4$ , and the orbit is unstable. The same is true for any $n>2$ .

Let us consider only the fourth quadrant, and a point (x, -x^n) on the curve. Energy conservation principle gives the velocity of the mouse, v as v^2=2g(x^n). Slope of the tangent to the curve at that point is -(nx^(n-1)). If the tangent makes an angle α with vertical, then sin(α)=1/[1+(nx^(n-1))^2]. Radius of curvature of the curve at that point is r=[1+(nx^(n-1))^2]^(3/2)]/n(n-1)(x^(n-2)). For the mouse not to loose contact with the curve, gsin(α)>(v^2)/r. This implies 1+(nx^(n-1))^2>2n(n-1)x^(2n-2), or n(n-2)x^(2n-2)<1 for all x. This holds true if and only if n is less than or equal to 2.