Difficult system of equations

You start by multiplying both sides of the first equation by a and both sides of the second equation by b to get
${ a }^{ 2 }+\frac { { a }^{ 2 }+8ab }{ { a }^{ 2 }+{ b }^{ 2 } } \quad =\quad 2a$
and
$b^{ 2 }+\frac { 8ab-{ b }^{ 2 } }{ { a }^{ 2 }+{ b }^{ 2 } } \quad =\quad 0$
Solving by subtracting the first equation from the second and simplifying yields ${ (a-1) }^{ 2 }\quad =\quad { b }^{ 2 }$
Expanding and moving stuff around should give you $(a-b)(a+b)+1\quad =\quad 2a$ Then, you multiply both sides of the first equation by b, and both sides of the second equation by a to get
$ab+\frac { ab+8{ b }^{ 2 } }{ { a }^{ 2 }+{ b }^{ 2 } } \quad =\quad 2b$
and
$ab+\frac { { 8a }^{ 2 }-ab }{ { a }^{ 2 }+{ b }^{ 2 } } \quad =\quad 0$
Solving by adding the equations together and simplifying yields $b\quad =\quad \frac { 4 }{ 1-a }$
When you plug this into $(a-b)(a+b)+1\quad =\quad 2a$ , you get an equation which simplifies into
${ a }^{ 4 }-4{ a }^{ 3 }+6{ a }^{ 2 }-4a-15\quad =\quad 0$
Factoring gives you $(a-3)(a+1)({ a }^{ 2 }-2a+5)\quad =\quad 0$
Plugging in a for the real solutions gives you (3,-2) and (-1,2)
$3\times (-2)\times (-1)\times 2\quad =\quad 12$