Consider two circles, each having radius of 5cm (centimeters), touching each other at a point P. A direct tangent QR is drawn touching one circle at a point Q and the other circle at a point R. Inside the region PQR inscribed by the two circles and the tangent, a square ABCD is inscribed with its base AB on the tangent and the other side touching the two circles at points D and C, respectively.
Find the area of the square ABCD.

The image will be drawn as:

We will assume the side of the square to be 2a. XY will be half of CD i.e. 2a/2 = a. This means that OX = radius - XY ==> 5 - a.
Similarly, CX = radius - AC ==> 5 - 2a and OC = radius.
Now, we can use pythagoras theorem in triangle OCX:
$$\left(5-a\right)^2+\left(5-2a\right)^2=5^2$$ ==> $$25\ +\ a^2-10a\ +\ 25\ +\ 4a^2\ -20a=25$$
==> $$5a^2-30a+25\ =\ 0$$ ==> $$a^2\ -\ 6a\ +\ 5\ =\ 0\ $$.
This gives a = 5 or 1. 
a can not be 5 as that will make CX i.e. 5 - 2a to be of negative length.
This means that side of the square is 2*1 = 2 cm and Area of the square is 2*2 = 4 sq. cm.