Squaring it

Careful counting shows that in a 3- sided polygon, only 1 triangle is present.

In 4 sided polygon,no. of trianglea with 3 endpoints = 4.No. of triangles with 4 endpoints = 4. So, total no. of triangles = 8.

In a 5 sided polygon, no. of triangles with 3 endpoints = 10. No. of triangles with 4 endpoints = 20.No. of triangles with 5 endpoints = 5.So, total no. of triangles = 35.

Counting this way, we get that, in a 6 sided polygon there are 110.

In a 7 sided polygon, there are 287 triangles.

In a 8 sided polygon, there are 632 triangles.

In a 9 sided polygon, there are 1302 triangles.

In a 10 sided polygon, there are 2400 triangles.

In a 11 sided polygon, there are 4257 triangles.

In a 12 sided polygon, there are 6956 triangles.

In a 13 sided polygon, there are 11297 triangles.

In a 14 sided polygon, there are 17234 triangles.

In a 15 sided polygon, there are 25935 triangles.

In a 16 sided polygon, there are 37424 triangles.

In a 17 sided polygon , there are 53516 triangles.

In a 18 sided polygon, there are 73404 triangles.

In a 19 sided polygon, there are 101745 triangles.

In a 20 sided polygon, there are 136200 triangles.

So, the sum of the squares of the above results = ${1^2}+{8^2}+{35^2} ..... {136200^2}$

= $18550440000 + 10352045025 + 5388147216 + 2863962256 + 1400555776 +672624225 +297010756 +127622209 +48385936 +18122049 +5760000+1695204 + 399424 + 82369 + 12100 + 1225 + 64 + 1$

= $39726865835$ .