A gentle flux problem

My general approach will be to form a closed surface, and per the Divergence theorem, equate the volume integral of the divergence to the sum of the outward fluxes over the constituent faces.

First, identify the constituent faces (all of which are open surfaces):

1) The given surface
2) A rectangle in the $x y$ plane $(z = 0)$ . This is the "bottom" surface
3) Two "side" surfaces parallel to the $y z$ plane located at $x = 0$ and $x = 1$
4) A "side" rectangle parallel to the $x z$ plane located at $y = 3$

The divergence of the vector field is equal to $x$ , and the volume integral of the divergence is:

$\int_0^3 \int_0^1 \int_0^{y^2} x \, dz \, dx \, dy = \frac{9}{2}$

By inspection, only the given surface from (1) and the surface from (4) have non-zero fluxes. Find the outward flux through the surface from (4).

$\phi_4 = \int \int_S \vec{F} \cdot \vec{n} \, dS = \int_0^1 27 x \, dx = \frac{27}{2}$

We know that the volume integral of the divergence equals $\phi_4$ plus the outward flux over the given surface $(\phi)$ .

$\frac{9}{2} = \frac{27}{2} + \phi \\ \phi = \boxed{-9}$

Steven has written an elegant solution based on Ostrogradsky's theorem (aka the Divergence Theorem or Gauss' theorem). Alternatively, we can use the definition of the flux in terms of a parameterisation.

The given surface $S$ is the graph of the function $z=f(x,y)=y^2$ on $D=[0,1]\times[0,3]$ . In the case of a graph, the standard normal vector is $\vec{N}=\left(-\frac{\partial z}{\partial x},-\frac{\partial z}{\partial y},1\right)$ so $\vec{N}=(0,-2y,1)$ in our problem. Now $\int\int_S \vec{F}\cdot d\vec{S}=\int\int_D \vec{F}\cdot \vec{N} \ dA=-\int_0^1\int_0^3 2xy^2\ dy\ dx=\boxed{-9}$