Let $$\vec a=\hat{i}+2\hat{j}+3\hat{k},  b=3\hat{i}+\hat{j}-\hat{k} $$ and  be three vectors such that  $$c$$ is coplanar with $$ \vec a$$  and  $$\vec b$$. If  $$\vec c $$ is perpendicular to  $$\vec b$$  and  $$\vec a\cdot \vec c=5,$$  then  $$|\vec c|$$  is equal to: 

$$\vec{a} = \hat{i}+2\hat{j}+3\hat{k}$$, $$\vec{b} = 3\hat{i}+\hat{j}-\hat{k}$$. $$\vec{c}$$ is coplanar with $$\vec{a}$$ and $$\vec{b}$$, perpendicular to $$\vec{b}$$, and $$\vec{a}\cdot\vec{c} = 5$$.

Since $$\vec{c}$$ is coplanar with $$\vec{a}$$ and $$\vec{b}$$: $$\vec{c} = \lambda\vec{a} + \mu\vec{b}$$

$$\vec{c} \perp \vec{b}$$: $$\vec{c}\cdot\vec{b} = 0$$

$$\lambda(\vec{a}\cdot\vec{b}) + \mu(\vec{b}\cdot\vec{b}) = 0$$

$$\vec{a}\cdot\vec{b} = 3+2-3 = 2$$, $$|\vec{b}|^2 = 9+1+1 = 11$$

$$2\lambda + 11\mu = 0 \Rightarrow \lambda = -\frac{11\mu}{2}$$

$$\vec{a}\cdot\vec{c} = 5$$

$$\lambda|\vec{a}|^2 + \mu(\vec{a}\cdot\vec{b}) = 5$$

$$|\vec{a}|^2 = 1+4+9 = 14$$

$$14\lambda + 2\mu = 5$$

$$14(-11\mu/2) + 2\mu = 5$$

$$-77\mu + 2\mu = 5$$

$$-75\mu = 5 \Rightarrow \mu = -1/15$$

$$\lambda = -11(-1/15)/2 = 11/30$$

$$\vec{c} = \frac{11}{30}(1,2,3) - \frac{1}{15}(3,1,-1)$$

$$= (\frac{11}{30}-\frac{2}{10}, \frac{22}{30}-\frac{1}{15}, \frac{33}{30}+\frac{1}{15})$$

$$= (\frac{11-6}{30}, \frac{22-2}{30}, \frac{33+2}{30}) = (\frac{5}{30}, \frac{20}{30}, \frac{35}{30}) = (\frac{1}{6}, \frac{2}{3}, \frac{7}{6})$$

$$|\vec{c}|^2 = \frac{1}{36} + \frac{4}{9} + \frac{49}{36} = \frac{1+16+49}{36} = \frac{66}{36} = \frac{11}{6}$$

$$|\vec{c}| = \sqrt{\frac{11}{6}}$$

The correct answer is Option 1: $$\sqrt{\frac{11}{6}}$$.