A circular current loop of radius $$R$$ is placed inside square loop of side length $$L$$ ($$L \gg R$$) such that they are co-planar and their centers coincide. The permeability of free space is $$\mu_0$$. The mutual inductance between circular loop and square loop is :

Mutual inductance $$M$$ is defined as the flux through one circuit when the other carries unit current.
We let a current $$I$$ flow in the square loop (side $$L$$). The circular loop (radius $$R,\,R\ll L$$) lies in the same plane with the same centre, so it intercepts the flux produced at the square’s centre.

1. Magnetic field at the centre of a square loop (single turn)
Consider one side of the square. The centre of the square is at a perpendicular distance $$r=\dfrac{L}{2}$$ from this straight segment of length $$L$$.
For a finite straight wire carrying current $$I$$, the magnetic field at a point a perpendicular distance $$r$$ away is

$$B_{\text{segment}}=\frac{\mu_0 I}{4\pi r}\left(\sin\theta_1+\sin\theta_2\right)$$

where $$\theta_1$$ and $$\theta_2$$ are the angles subtended by the two ends of the wire at the point.

Here each half-length is $$\dfrac{L}{2}$$, so
$$\theta_1=\theta_2=\arctan\!\left(\frac{L/2}{\,L/2}\right)=45^{\circ}$$,
and $$\sin45^{\circ}=\dfrac{1}{\sqrt2}.$$ Therefore for one side

$$B_{\text{side}}=\frac{\mu_0 I}{4\pi \bigl(L/2\bigr)}\left(2\cdot\frac1{\sqrt2}\right) =\frac{\mu_0 I}{2\pi L}\,\sqrt2.$$ The direction is perpendicular to the plane of the loop (by the right-hand rule) and is the same for all four sides.

2. Field at the centre due to the complete square
Adding the contribution of the four sides,

$$B_{\text{centre}}=4B_{\text{side}} =4\left(\frac{\mu_0 I}{2\pi L}\,\sqrt2\right) =\frac{2\sqrt2\,\mu_0 I}{\pi L}.$$

3. Flux linked with the small circular loop
Because $$R\ll L$$, the field is practically uniform over the area of the small circle, so

$$\Phi = B_{\text{centre}}\;(\text{area of circle}) =\frac{2\sqrt2\,\mu_0 I}{\pi L}\;\bigl(\pi R^2\bigr) =\frac{2\sqrt2\,\mu_0 I R^2}{L}.$$

4. Mutual inductance
By definition, $$M=\dfrac{\Phi}{I}$$, hence

$$M=\frac{2\sqrt2\,\mu_0 R^2}{L}.$$

5. Choosing the correct option
This value corresponds to Option D.

Option D which is: $$\dfrac{2\sqrt{2}\,\mu_0 R^2}{L}$$