Lost in space

* Note: This problem and solution have been edited due to concerns raised by Maria Kozlowska. Thanks go to her for catching my error and proposing a suitable remedy to save the question from getting truly "Lost in Space" Also, apologies to Bant Paswan for not following up on similar concerns he raised so long ago. *

We know from the 3-D version of the British Flag Theorem that if $ABCD$ is a square in space and $P$ is any point then $(AP)^{2} + (CP)^{2} = (BP)^{2} + (DP)^{2}.$

Now of any five vertices of an octahedron, four must form a square, so we just need to go through the given distances to find two distinct pairs of distances whose squares sum to the same value. We find that $3^{2} + 11^{2} = 7^{2} + 9^{2} = 130$ is the only possibility.

So suppose, without loss of generality, that $AP = 3, CP = 11, BP = 7$ and $DP = 9.$ Then with the other two vertices of the octahedron being $E$ and $F$ we must also have $AECF$ be a square. So assigning $EP = 8$ , (the last of the given distances), by the BFT we have that

$(AP)^{2} + (CP)^{2} = (EP)^{2} + (FP)^{2} \Longrightarrow 3^{2} + 11^{2} = 8^{2} + (FP)^{2} \Longrightarrow (FP)^{2} = \boxed{66}.$

Let's place origin of Cartesian coordinate system in the middle of the octahedron a the axes will go through three vertices. Cartesian coordinates of vertices will be $[\pm d,0,0]; [0,\pm d,0]$ and $[0,0,\pm d]$ , where $d$ is some positive real number. Let the position of the point $P$ be $[x,y,z]$ . Then the square of distances from all six vertices will be

$r_1^2 = (x+d)^2 + y^2 + z^2 = x^2 + 2 x d + d^2 + y^2 + z^2 \, ,$

$r_2^2 = (x-d)^2 + y^2 + z^2 = x^2 - 2 x d + d^2 + y^2 + z^2 \, ,$

$r_3^2 = x^2 + (y+d)^2 + z^2 = x^2 + y^2 + 2 y d + d^2+ z^2 \, ,$

$r_4^2 = x^2 + (y-d)^2 + z^2 = x^2 + y^2 - 2 y d + d^2 + z^2 \, ,$

$r_5^2 = x^2 + y^2 + (z+d)^2 = x^2 + y^2 + z^2 + 2 z d + d^2\, ,$

$r_6^2 = x^2 + y^2 + (z-d)^2 = x^2 + y^2 + z^2 - 2 z d + d^2 \, .$

When we add squares of distances from opposite vertices, we obtain same number

$r_1^2 + r_2^2 = 2 x^2 + 2 y^2 + 2 z^2 + 2 d^2 \, ,$

$r_3^2 + r_4^2 = 2 x^2 + 2 y^2 + 2 z^2 + 2 d^2 \, ,$

$r_5^2 + r_6^2 = 2 x^2 + 2 y^2 + 2 z^2 + 2 d^2 \, .$

Now we only need to find out, which distances are paired. Using simple table

we obtain

$3^2 + 11^2 = 7^2 + 9^2 = 6^2 + a = 130 \, ,$

from where we finally have answer

$a = 130 - 6^2 = \underline{94} \, .$