Let A be a matrix such that $$A \cdot \begin{bmatrix} 1 & 2 \\ 0 & 3 \end{bmatrix}$$ is a scalar matrix and $$|3A| = 108$$. Then $$A^2$$ equals:

We have the condition that $$A\cdot\begin{bmatrix}1 & 2\\0 & 3\end{bmatrix}$$ is a scalar matrix. A scalar matrix can be written as $$kI$$, where $$I=\begin{bmatrix}1 & 0\\0 & 1\end{bmatrix}$$ and $$k$$ is a real number. So the given information is

$$A\begin{bmatrix}1 & 2\\0 & 3\end{bmatrix}=kI.$$

Because the matrix on the right is invertible, we can post-multiply both sides by its inverse to isolate $$A$$. The rule is: if $$XY=Z$$ and $$Y$$ is invertible, then $$X=ZY^{-1}$$. Thus,

$$A=kI\left(\begin{bmatrix}1 & 2\\0 & 3\end{bmatrix}\right)^{-1}.$$

Now we need the inverse of the upper-triangular matrix $$\begin{bmatrix}1 & 2\\0 & 3\end{bmatrix}$$. For a matrix $$\begin{bmatrix}a & b\\0 & d\end{bmatrix}$$ the inverse is $$\begin{bmatrix}\dfrac1a & -\dfrac{b}{ad}\\0 & \dfrac1d\end{bmatrix}$$. Substituting $$a=1,\;b=2,\;d=3$$ we get

$$\left(\begin{bmatrix}1 & 2\\0 & 3\end{bmatrix}\right)^{-1}= \begin{bmatrix}1 & -\dfrac{2}{3}\\0 & \dfrac13\end{bmatrix}.$$

Multiplying by the scalar $$k$$ gives

$$A=k\begin{bmatrix}1 & -\dfrac{2}{3}\\0 & \dfrac13\end{bmatrix}= \begin{bmatrix}k & -\dfrac{2k}{3}\\0 & \dfrac{k}{3}\end{bmatrix}.$$

Next, the problem tells us $$|3A|=108$$. First compute $$3A$$: multiplying every entry of $$A$$ by 3 results in

$$3A=\begin{bmatrix}3k & -2k\\0 & k\end{bmatrix}.$$

The determinant of an upper-triangular matrix is the product of its diagonal elements. Therefore

$$|3A|=(3k)(k)=3k^2.$$

We are told that $$|3A|=108$$, so

$$3k^2=108\;\;\Longrightarrow\;\;k^2=36\;\;\Longrightarrow\;\;k=\pm6.$$

Because $$k^2$$ will appear in $$A^2$$, the sign of $$k$$ will not matter. We keep $$k^2=36$$ for further calculation.

Now compute $$A^2=A\cdot A$$. Using

$$A=\begin{bmatrix}k & -\dfrac{2k}{3}\\0 & \dfrac{k}{3}\end{bmatrix},$$

we multiply the matrices entry by entry:

$$A^2=\begin{bmatrix}k & -\dfrac{2k}{3}\\0 & \dfrac{k}{3}\end{bmatrix} \begin{bmatrix}k & -\dfrac{2k}{3}\\0 & \dfrac{k}{3}\end{bmatrix}.$$

Row 1, Column 1: $$k\cdot k+\left(-\dfrac{2k}{3}\right)\cdot 0=k^2.$$

Row 1, Column 2: $$k\left(-\dfrac{2k}{3}\right)+\left(-\dfrac{2k}{3}\right)\left(\dfrac{k}{3}\right)= -\dfrac{2k^2}{3}-\dfrac{2k^2}{9}= -\dfrac{8k^2}{9}.$$

Row 2, Column 1: $$0\cdot k+\dfrac{k}{3}\cdot0=0.$$

Row 2, Column 2: $$0\left(-\dfrac{2k}{3}\right)+\dfrac{k}{3}\cdot\dfrac{k}{3}= \dfrac{k^2}{9}.$$

Putting these four results together,

$$A^2=\begin{bmatrix}k^2 & -\dfrac{8k^2}{9}\\0 & \dfrac{k^2}{9}\end{bmatrix}.$$

Finally substitute $$k^2=36$$:

$$A^2=\begin{bmatrix}36 & -\dfrac{8\cdot36}{9}\\0 & \dfrac{36}{9}\end{bmatrix} =\begin{bmatrix}36 & -32\\0 & 4\end{bmatrix}.$$

This matrix matches Option D. Hence, the correct answer is Option D.