Let $$Q$$ be the foot of the perpendicular from the point $$P(7, -2, 13)$$ on the plane containing the lines $$\frac{x+1}{6} = \frac{y-1}{7} = \frac{z-3}{8}$$ and $$\frac{x-1}{3} = \frac{y-2}{5} = \frac{z-3}{7}$$. Then $$(PQ)^2$$ is equal to _________

We have two lines whose symmetric equations are

$$\dfrac{x+1}{6}=\dfrac{y-1}{7}=\dfrac{z-3}{8} \quad\text{and}\quad \dfrac{x-1}{3}=\dfrac{y-2}{5}=\dfrac{z-3}{7}.$$

The first line can be written in parametric form by introducing a parameter $$s$$:

$$x=6s-1,\;y=7s+1,\;z=8s+3.$$

Hence it passes through the point $$(-1,1,3)$$ and has direction vector

$$\vec{d_1}= \langle 6,7,8\rangle.$$

Similarly the second line, with parameter $$t$$, is

$$x=3t+1,\;y=5t+2,\;z=7t+3,$$

so it passes through the point $$(1,2,3)$$ and possesses direction vector

$$\vec{d_2}= \langle 3,5,7\rangle.$$

Because both lines lie in the required plane, the normal vector of that plane can be obtained from the cross product of their direction vectors:

$$\vec{n}= \vec{d_1}\times\vec{d_2} =\begin{vmatrix} \mathbf{i}&\mathbf{j}&\mathbf{k}\\[4pt] 6&7&8\\ 3&5&7 \end{vmatrix} =\mathbf{i}(7\cdot7-8\cdot5)-\mathbf{j}(6\cdot7-8\cdot3)+\mathbf{k}(6\cdot5-7\cdot3).$$

Evaluating each component we get

$$\vec{n}= \langle 9,-18,9\rangle = 9\langle 1,-2,1\rangle.$$

Hence a convenient normal vector is $$\langle 1,-2,1\rangle.$$

To write the equation of the plane we still need one point on it. Let us show that the two lines intersect and find the intersection point, although (because the normal vector is ready) any point of either line would also suffice.

Equating the parametric coordinates, we set

$$6s-1 = 3t+1,\quad 7s+1 = 5t+2,\quad 8s+3 = 7t+3.$$

From the third equality $$8s+3 = 7t+3$$ gives $$7t=8s$$ and therefore $$t=\dfrac{8}{7}s.$$ Substituting this in the first equality,

$$6s-1 = 3\left(\dfrac{8}{7}s\right)+1,$$ $$6s-1 = \dfrac{24}{7}s +1,$$ $$42s -7 = 24s +7,$$ $$18s = 14 \;\Rightarrow\; s=\dfrac{7}{9}.$$

Hence $$t=\dfrac{8}{7}\cdot\dfrac{7}{9}=\dfrac{8}{9}.$$ Inserting $$s=\dfrac{7}{9}$$ in the first line gives the common point

$$x = 6\left(\dfrac{7}{9}\right)-1 = \dfrac{42}{9}-\dfrac{9}{9}= \dfrac{33}{9}= \dfrac{11}{3},$$ $$y = 7\left(\dfrac{7}{9}\right)+1 = \dfrac{49}{9}+\dfrac{9}{9}= \dfrac{58}{9},$$ $$z = 8\left(\dfrac{7}{9}\right)+3 = \dfrac{56}{9}+\dfrac{27}{9}= \dfrac{83}{9}.$$

Denote this intersection point by $$I\left(\dfrac{11}{3},\dfrac{58}{9},\dfrac{83}{9}\right).$$

Using point-normal form, the plane through $$I$$ with normal $$\langle 1,-2,1\rangle$$ is

$$1\,(x-\tfrac{11}{3}) -2\,(y-\tfrac{58}{9}) +1\,(z-\tfrac{83}{9}) = 0.$$

Multiplying term by term,

$$x-\tfrac{11}{3} -2y + \tfrac{116}{9} + z - \tfrac{83}{9} = 0.$$

The constant terms combine to zero because

$$-\tfrac{11}{3} + \tfrac{116}{9} - \tfrac{83}{9} = -\tfrac{33}{9} + \tfrac{116}{9} - \tfrac{83}{9}=0.$$

Thus the plane’s simplified Cartesian equation is beautifully concise:

$$x - 2y + z = 0.$$

Now we bring in the given point $$P(7,-2,13).$$ The squared length of the perpendicular from a point $$P(x_1,y_1,z_1)$$ to a plane $$ax+by+cz+d=0$$ is obtained by first projecting $$P$$ onto the normal and then applying the distance formula.

The formula for the foot $$Q$$ of the perpendicular is

$$Q = P - \dfrac{a x_1 + b y_1 + c z_1 + d}{a^2+b^2+c^2}\,\langle a,b,c\rangle.$$

For our plane $$x-2y+z=0$$ we have $$a=1,\;b=-2,\;c=1,\;d=0.$$ Calculating the numerator,

$$a x_1 + b y_1 + c z_1 + d = 1\cdot7 + (-2)\cdot(-2) + 1\cdot13 + 0 = 7 + 4 + 13 = 24.$$

The denominator is

$$a^2+b^2+c^2 = 1^2 + (-2)^2 + 1^2 = 1+4+1 = 6.$$

Hence the scaling factor is

$$k = \dfrac{24}{6}=4.$$

Therefore the foot of the perpendicular is

$$Q = (7,-2,13) - 4\langle 1,-2,1\rangle = (\,7-4,\; -2-4(-2),\; 13-4\,)$$

$$= (3,6,9).$$

Finally, the vector $$\overrightarrow{PQ}$$ is

$$\langle 3-7,\;6-(-2),\;9-13\rangle = \langle -4,\;8,\;-4\rangle.$$

The square of its magnitude, i.e. $$(PQ)^2,$$ is

$$(-4)^2 + 8^2 + (-4)^2 = 16 + 64 + 16 = 96.$$

So, the answer is $$96$$.