Let a unit vector $$\hat{u} = x\hat{i} + y\hat{j} + z\hat{k}$$ make angles $$\frac{\pi}{2}, \frac{\pi}{3}$$ and $$\frac{2\pi}{3}$$ with the vectors $$\frac{1}{\sqrt{2}}\hat{i} + \frac{1}{\sqrt{2}}\hat{k}, \frac{1}{\sqrt{2}}\hat{j} + \frac{1}{\sqrt{2}}\hat{k}$$ and $$\frac{1}{\sqrt{2}}\hat{i} + \frac{1}{\sqrt{2}}\hat{j}$$ respectively. If $$\vec{v} = \frac{1}{\sqrt{2}}\hat{i} + \frac{1}{\sqrt{2}}\hat{j} + \frac{1}{\sqrt{2}}\hat{k}$$, then $$|\hat{u} - \vec{v}|^2$$ is equal to

Let $$\hat{u} = x\hat{i} + y\hat{j} + z\hat{k}$$ be a unit vector making the given angles with the three vectors. The three given vectors are already unit vectors (each has magnitude 1), so using the dot product to express each angle gives the following equations.

For the angle $$\frac{\pi}{2}$$ with $$\frac{1}{\sqrt{2}}\hat{i} + \frac{1}{\sqrt{2}}\hat{k}$$, we have $$ \frac{x + z}{\sqrt{2}} = \cos\frac{\pi}{2} = 0 \implies x + z = 0 \quad \cdots(1) $$. For the angle $$\frac{\pi}{3}$$ with $$\frac{1}{\sqrt{2}}\hat{j} + \frac{1}{\sqrt{2}}\hat{k}$$, $$ \frac{y + z}{\sqrt{2}} = \cos\frac{\pi}{3} = \frac{1}{2} \implies y + z = \frac{1}{\sqrt{2}} \quad \cdots(2)$$. For the angle $$\frac{2\pi}{3}$$ with $$\frac{1}{\sqrt{2}}\hat{i} + \frac{1}{\sqrt{2}}\hat{j}$$, $$ \frac{x + y}{\sqrt{2}} = \cos\frac{2\pi}{3} = -\frac{1}{2} \implies x + y = -\frac{1}{\sqrt{2}} \quad \cdots(3)$$.

From (1), $$z = -x$$. Substituting into (2) gives $$y - x = \frac{1}{\sqrt{2}} \quad \cdots(4)$$, and (3) states $$x + y = -\frac{1}{\sqrt{2}} \quad \cdots(5)$$. Adding (4) and (5) yields $$2y = 0 \implies y = 0$$. Then (5) implies $$x = -\frac{1}{\sqrt{2}}$$ and (1) gives $$z = \frac{1}{\sqrt{2}}$$, so $$\hat{u} = -\frac{1}{\sqrt{2}}\hat{i} + 0\hat{j} + \frac{1}{\sqrt{2}}\hat{k}$$.

To compute $$|\hat{u} - \vec{v}|^2$$ where $$\vec{v} = \frac{1}{\sqrt{2}}\hat{i} + \frac{1}{\sqrt{2}}\hat{j} + \frac{1}{\sqrt{2}}\hat{k}$$, note that

$$\hat{u} - \vec{v} = \left(-\frac{1}{\sqrt{2}} - \frac{1}{\sqrt{2}}\right)\hat{i} + \left(0 - \frac{1}{\sqrt{2}}\right)\hat{j} + \left(\frac{1}{\sqrt{2}} - \frac{1}{\sqrt{2}}\right)\hat{k}$$

$$= -\sqrt{2}\,\hat{i} - \frac{1}{\sqrt{2}}\hat{j} + 0\hat{k}$$

$$ |\hat{u} - \vec{v}|^2 = 2 + \frac{1}{2} + 0 = \frac{5}{2} $$. Thus the correct answer is Option (2): $$\frac{5}{2}$$.