The resultant $$R$$ of two forces acting on a particle is at right angles to one of them and its magnitude is one third of the other force. The ratio of larger force to smaller one is

Let the two forces be the vectors $$\vec{A}$$ and $$\vec{B}$$ with magnitudes $$|\vec{A}| = a$$ and $$|\vec{B}| = b$$, where $$a \gt b$$ (so $$\vec{A}$$ is the larger force).

Their resultant is $$\vec{R} = \vec{A} + \vec{B}$$. We are told:

(i) $$\vec{R}$$ is at right angles to one of the forces.
(ii) The magnitude of $$\vec{R}$$ is one-third of the other force.

Assume $$\vec{R}$$ is perpendicular to the smaller force $$\vec{B}$$ (this will keep the algebra consistent with $$a \gt b$$).
Hence, $$\vec{R}\,\cdot\,\vec{B} = 0$$.

Compute the dot product:
$$\bigl(\vec{A} + \vec{B}\bigr)\!\cdot\!\vec{B} = \vec{A}\!\cdot\!\vec{B} + \vec{B}\!\cdot\!\vec{B} = 0$$

Let $$\theta$$ be the angle between $$\vec{A}$$ and $$\vec{B}$$. Then

$$ab\cos\theta + b^{2} = 0 \; \Longrightarrow \; \cos\theta = -\frac{b}{a}.$$

Next, the magnitude of the resultant is given by

$$R^{2} = a^{2} + b^{2} + 2ab\cos\theta.$$

Substitute $$\cos\theta = -\frac{b}{a}:$$
$$R^{2} = a^{2} + b^{2} - 2ab\left(\frac{b}{a}\right) = a^{2} + b^{2} - 2b^{2} = a^{2} - b^{2}.$$

Condition (ii) states that $$R = \dfrac{1}{3}\,a$$ (one-third of the other, larger, force). Therefore

$$\left(\frac{a}{3}\right)^{2} = a^{2} - b^{2} \; \Longrightarrow \; \frac{a^{2}}{9} = a^{2} - b^{2}.$$

Rearrange:

$$a^{2} - \frac{a^{2}}{9} = b^{2} \; \Longrightarrow \; \frac{8a^{2}}{9} = b^{2}.$$

Taking square roots (all quantities are positive):

$$\frac{a}{b} = \sqrt{\frac{9}{8}} = \frac{3}{2\sqrt{2}}.$$

Thus, the ratio of the larger force to the smaller force is

$$a : b = 3 : 2\sqrt{2}.$$

Option D which is: $$3 : 2\sqrt{2}$$