Determinant of a Product

First, compute $$A^T$$, the transpose of A:

$$A^T = \begin{pmatrix} \sqrt{2} & -1 \\ 1 & \sqrt{2} \end{pmatrix}$$.

Now, compute $$A^T A$$:

$$A^T A = \begin{pmatrix} \sqrt{2} & -1 \\ 1 & \sqrt{2} \end{pmatrix} \begin{pmatrix} \sqrt{2} & 1 \\ -1 & \sqrt{2} \end{pmatrix} = \begin{pmatrix} 3 & 0 \\ 0 & 3 \end{pmatrix} = 3I$$.

Thus, $$A^T A = 3I$$.

 $$C = ABA^T$$.

 $$C^2 = (ABA^T)(ABA^T) = AB(A^T A)BA^T = AB(3I)BA^T = 3AB^2A^T$$, since scalar multiplication commutes.

 compute $$B^2$$:

$$B^2 = B \cdot B = \begin{pmatrix} 1 & 0 \\ 1 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 1 & 1 \end{pmatrix} = \begin{pmatrix} (1)(1) + (0)(1) & (1)(0) + (0)(1) \\ (1)(1) + (1)(1) & (1)(0) + (1)(1) \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix}$$.

So, $$C^2 = 3A B^2 A^T = 3A \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix} A^T$$.

Now, $$X = A^T C^2 A = A^T \left(3A B^2 A^T\right) A = 3 A^T A B^2 A^T A$$.

Substitute $$A^T A = 3I$$:

$$X = 3 (3I) B^2 (3I) = 3 \cdot 3 \cdot 3 \cdot I B^2 I = 27 B^2$$, since the identity matrix commutes.

$$X = 27 \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix} = \begin{pmatrix} 27 & 0 \\ 54 & 27 \end{pmatrix}$$.

compute $$\det X$$:

For a matrix $$\begin{pmatrix} a & b \\ c & d \end{pmatrix}$$, $$\det = ad - bc$$.

So, $$\det X = (27)(27) - (0)(54) = 729 - 0 = 729$$.

Use the Commutative Property: We are given that $$A$$ and $$B$$ commute ($$AB = BA$$). This allows us to use standard algebraic identities. Consider the product:
$$(A + B)(A - B) = A^2 - AB + BA - B^2$$
Since $$AB = BA$$, the middle terms cancel out:
$$(A + B)(A - B) = A^2 - B^2$$
 Substitute Given Identities: We are given $$A^2 = I$$ and $$B^2 = I$$. Substituting these into the equation gives:
$$(A + B)(A - B) = I - I = 0$$
Take the Determinant: Apply the determinant property $$\det(XY) = \det(X)\det(Y)$$ to both sides:
$$\det((A + B)(A - B)) = \det(0)$$
$$\det(A + B) \cdot \det(A - B) = 0$$
Analyze the Condition: For the product of two real numbers to be zero, at least one must be zero.
o If $$\det(A - B) \neq 0$$, then $$\det(A + B)$$ must be $$0$$.
o Since $$A$$ and $$B$$ are symmetric matrices ($$A^T=A, B^T=B$$) that commute and square to $$I$$, they share a common basis of eigenvectors with eigenvalues $$\pm 1$$. Given $$\det(A) \neq \det(B)$$, their eigenvalue distributions must differ, ensuring that $$\det(A-B) \neq 0$$ is a valid state, forcing:
$$\det(A + B) = 0$$

 For any non-singular matrix $$M$$, we know that $$M^{-1} = \frac{1}{|M|}\operatorname{adj}(M)$$, which can be rewritten as:
$$\operatorname{adj}(M) = |M|M^{-1}$$
Substitute $$M = A^{-1}$$ and $$M = B^{-1}$$ into the property:
$$\operatorname{adj}(A^{-1}) = |A^{-1}|(A^{-1})^{-1} = \frac{1}{|A|}A$$
$$\operatorname{adj}(B^{-1}) = |B^{-1}|(B^{-1})^{-1} = \frac{1}{|B|}B$$

$$X = A \left( \frac{1}{|A|}A + \frac{1}{|B|}B \right)^{-1} B$$
 ($$(XYZ)^{-1} = Z^{-1}Y^{-1}X^{-1}$$):
$$X^{-1} = \left[ A \left( \frac{1}{|A|}A + \frac{1}{|B|}B \right)^{-1} B \right]^{-1}$$
$$X^{-1} = B^{-1} \left[ \left( \frac{1}{|A|}A + \frac{1}{|B|}B \right)^{-1} \right]^{-1} A^{-1}$$
$$X^{-1} = B^{-1} \left( \frac{1}{|A|}A + \frac{1}{|B|}B \right) A^{-1}$$
$$X^{-1} = \frac{1}{|A|} B^{-1} A A^{-1} + \frac{1}{|B|} B^{-1} B A^{-1}$$
Since $$A A^{-1} = I$$ and $$B^{-1} B = I$$:
$$X^{-1} = \frac{1}{|A|} B^{-1} + \frac{1}{|B|} A^{-1}$$
Replace $$B^{-1}$$ with $$\frac{\operatorname{adj}(B)}{|B|}$$ and $$A^{-1}$$ with $$\frac{\operatorname{adj}(A)}{|A|}$$:
$$X^{-1} = \frac{1}{|A|} \left( \frac{\operatorname{adj}(B)}{|B|} \right) + \frac{1}{|B|} \left( \frac{\operatorname{adj}(A)}{|A|} \right)$$
$$X^{-1} = \frac{\operatorname{adj}(B)}{|A||B|} + \frac{\operatorname{adj}(A)}{|A||B|}$$
$$X^{-1} = \frac{1}{|AB|} (\operatorname{adj}(B) + \operatorname{adj}(A))$$