area of polygon with 3-sides

Using Sine Rule, we have: $\quad \dfrac {\sin {A}}{6} = \dfrac {\sin {B}}{3}$ .

Let $\sin {B} = \beta$ , then $\sin {A} = 2 \beta$ , $\cos {A} = \sqrt{1-4\beta^2}$ and $\cos {B} = \sqrt{1-\beta^2}$

It is given that: $\quad \cos {A-B} = 0.8\quad \Rightarrow \cos {A} \cos {B} + \sin {A} \sin {B} = 0.8$

$\Rightarrow \sqrt{1-4\beta^2} \sqrt{1-\beta^2} + \beta (2 \beta) = 0.8$

$\quad (1-4\beta^2) (1-\beta^2) = (0.8 - 2 \beta^2)^2$

$\quad 1-5\beta^2 + 4\beta^4 = 0.64 - 3.2 \beta^2 + 4\beta^4$

$\Rightarrow 1.8 \beta^2 = 0.36 \quad \Rightarrow \beta^2 = \frac {0.36}{1.8} = \frac {1}{5} \quad \Rightarrow \beta = \frac {1}{\sqrt{5}}$

$\Rightarrow \cos {A} = \frac {2}{\sqrt{5}}\quad \cos {B} = \frac {1}{\sqrt{5}} \quad \sin {A} = \sqrt{1 - \frac {4}{5}} = \frac {1}{\sqrt{5}} \quad \sin {B} = \sqrt{1 - \frac {1}{5}} = \frac {2}{\sqrt{5}}$

Now $C = 180^\circ - A - B$ . Let us check what $A+B$ is.

$\cos {A+B} = \cos {A} \cos {B} - \sin {A} \sin {B} = \frac {2}{\sqrt{5}}\times \frac {1}{\sqrt{5}} - \frac {1}{\sqrt{5}}\times \frac {2}{\sqrt{5}} = 0$

$\Rightarrow A+B = 90^\circ$ , $C = 90^\circ$ and $\triangle ABC$ is a right-angle triangle and its area $= \frac {1}{2}\times 3\times 6 = \boxed {9}$ .

You could do it with Law of Tangents

$\frac{a+b}{a-b} = \frac {tan \frac{\alpha + \beta}{2}}{tan \frac{\alpha - \beta}{2}}$

We have

$tan\frac{\alpha - \beta}{2} = \sqrt{\frac {1-cos(\alpha - \beta)}{1+cos(\alpha - \beta)}} = \sqrt {\frac{1-4/5}{1+4/5}}=\frac{1}{3}$

Now $\frac{6+3}{6-3} = \frac{cot \gamma /2}{1/3}$

So $cot (\gamma /2)=1\$ , therefore $\gamma/2 = \pi/4$ and $\gamma =\pi/2$ ie that triangle is right-angled

As we know area of the right-angled triangle $area = a \times b /2 =18/2 =9$