Let $$\vec{a} = \hat{i} + \hat{j} + \sqrt{2}\hat{k}$$, $$\vec{b} = b_1\hat{i} + b_2\hat{j} + \sqrt{2}\hat{k}$$ and $$\vec{c} = 5\hat{i} + \hat{j} + \sqrt{2}\hat{k}$$ be three vectors such that the projection vector of $$\vec{b}$$ on $$\vec{a}$$ is $$|\vec{a}|$$. If $$\vec{a} + \vec{b}$$ is perpendicular to $$\vec{c}$$, then $$|\vec{b}|$$ is equal to:

We have the three vectors

$$\vec a=\hat i+\hat j+\sqrt2\,\hat k,\qquad \vec b=b_1\hat i+b_2\hat j+\sqrt2\,\hat k,\qquad \vec c=5\hat i+\hat j+\sqrt2\,\hat k.$$

The statement “the projection of $$\vec b$$ on $$\vec a$$ is $$|\vec a|$$” refers to the scalar projection. The scalar projection formula is

$$\text{comp}_{\vec a}\vec b=\frac{\vec a\cdot\vec b}{|\vec a|}.$$

Setting this equal to $$|\vec a|$$ we obtain

$$\frac{\vec a\cdot\vec b}{|\vec a|}=|\vec a|.$$

Multiplying by $$|\vec a|$$ gives

$$\vec a\cdot\vec b=|\vec a|^{\,2}.$$

First we compute $$|\vec a|^{\,2}$$:

$$|\vec a|^{\,2}=1^{2}+1^{2}+(\sqrt2)^{2}=1+1+2=4,$$

so $$|\vec a|=2.$$

Now we evaluate $$\vec a\cdot\vec b$$:

$$\vec a\cdot\vec b=(1)(b_1)+(1)(b_2)+(\sqrt2)(\sqrt2)=b_1+b_2+2.$$

The equality $$\vec a\cdot\vec b=4$$ therefore becomes

$$b_1+b_2+2=4\qquad\Longrightarrow\qquad b_1+b_2=2.$$

The second condition says that $$\vec a+\vec b$$ is perpendicular to $$\vec c$$. For perpendicular vectors the dot product is zero, so

$$(\vec a+\vec b)\cdot\vec c=0.$$

First compute $$\vec a+\vec b$$:

$$\vec a+\vec b=(1+b_1)\hat i+(1+b_2)\hat j+(2\sqrt2)\hat k.$$

Now take its dot product with $$\vec c$$:

$$$ (\vec a+\vec b)\cdot\vec c= (1+b_1)(5)+(1+b_2)(1)+(2\sqrt2)(\sqrt2). $$$

Since $$(2\sqrt2)(\sqrt2)=2\cdot2=4,$$ we have

$$5(1+b_1)+1(1+b_2)+4=0.$$

Expanding and collecting terms gives

$$5+5b_1+1+b_2+4=0\quad\Longrightarrow\quad5b_1+b_2+10=0.$$

Thus

$$5b_1+b_2=-10.$$

We now solve the simultaneous linear equations

$$\begin{cases} b_1+b_2=2,\\ 5b_1+b_2=-10. \end{cases}$$

Subtracting the first from the second eliminates $$b_2$$:

$$4b_1=-12\quad\Longrightarrow\quad b_1=-3.$$

Substituting $$b_1=-3$$ into $$b_1+b_2=2$$ gives

$$-3+b_2=2\quad\Longrightarrow\quad b_2=5.$$

Hence

$$\vec b=-3\hat i+5\hat j+\sqrt2\,\hat k.$$

Finally, we find $$|\vec b|$$:

$$|\vec b|^{\,2}=(-3)^{2}+5^{2}+(\sqrt2)^{2}=9+25+2=36,$$

so

$$|\vec b|=6.$$

Hence, the correct answer is Option C.