Tell the sum.

Let $x = n + a$ with $1 \leq n \leq N$ an integer and $0 \leq a < 1$ . Then $\{x\} = a$ and $\{x^2\} = \{n^2 + 2an + a^2\} = \{2an + a^2\}.$ In order for $\{x^2\} = a^2$ , it is therefore necessary that $2an$ is an integer. This is possible provided that $a = \frac{k}{2n}\ \ \ \ 1 \leq k < 2n\ \text{an integer}.$ There are $\sum_{n=1}^N (2n-1) = N^2$ such combinations of $n$ and $k$ . To add them, consider that the average of the fractional parts $\{x\}$ is $\tfrac12$ , so that we calculate $\sum_{n=1}^N (2n-1)(n + \tfrac12) = \sum_{n=1}^N 2n^2 - \tfrac12 = \tfrac13N(N+1)(2N+1) - \tfrac12N.$ Substitute $N = 100$ to find $100\cdot 101\cdot \frac{201}3 - \frac{100}2 = 676\,700 - 50 = \boxed{676,650}.$

To be in G.P. they must satisfy the equation $\{x^2\}$ = $\{x\}^2$ with $x$ not being integer. Let $x = a$ be one of the solutions between $n$ and $n+1$ where $n$ is any natural number. Then $n<a<n+1$ which implies $n^2<a^2<(n+1)^2$ or $n^2<a^2<n^2+2n+1$ or $n^2 + 1\leq \lfloor a^2\rfloor\leq n^2 + 2n$

In general, $\lfloor a^2\rfloor$ = $n^2+r$ , $1\leq r\leq2n$ , $r \in N$

Now, $\{a^2\}$ = $\{a\}^2$
$\Rightarrow$ $a^2$ $-$ $\lfloor a^2\rfloor$ = $(a-\lfloor a\rfloor)^2$ $\Rightarrow$ $a^2$ $-$ $\lfloor a^2\rfloor$ = $a^2+ \lfloor a\rfloor^2\ - 2a\lfloor a\rfloor$ $\Rightarrow$ $2a\lfloor a\rfloor = \lfloor a\rfloor^2\ + \lfloor a^2\rfloor$

We have $\lfloor a^2\rfloor$ = $n^2+r$ and $\lfloor a\rfloor = n$

So, $2an = n^2+n^2+r$

$\Rightarrow$ $a = \frac{2n^2+r}{2n}$ or $a=n+\frac{r}{2n}$
$a$ is not an integer $\Rightarrow$ $r\ne 2n$

Hence, between $n$ and $n+1$ , $x=n+\frac{r}{2n}$ ; $1\leq r\leq 2n-1$ , $r \in N , n \in N$ are the solutions. Now sum of these solution,

$S_n = \sum_{r=1}^{2n-1} (n+\frac{r}{2n})$ $\Rightarrow$ $S_n = n(2n-1) + \frac{(2n-1)(2n)}{2.2n}$ = $(2n-1)(n+\frac{1}{2})$ = $\frac{4n^2-1}{2}$ Our final sum $S = \sum_{n=1}^{100} S_n$ $\Rightarrow$ $S = \sum_{n=1}^{100} (\frac{4n^2-1}{2}) = 2\sum_{n=1}^{100} n^2 - \sum_{n=1}^{100} \frac{1}{2}$

$\Rightarrow$ $S = \frac{1}{3}\cdot 100\cdot 101\cdot 201 - 50$ = $676700 - 50 = 676650$