Let $$\frac{\pi}{2} \lt x \lt \pi$$ be such that $$\cot x = \frac{-5}{\sqrt{11}}$$. Then $$\left(\sin \frac{11x}{2}\right)(\sin 6x - \cos 6x) + \left(\cos \frac{11x}{2}\right)(\sin 6x + \cos 6x)$$ is equal to

We are given $$\frac{\pi}{2}\lt x\lt \pi$$ and $$\cot x=\frac{-5}{\sqrt{11}}.$$
Since $$x$$ lies in the second quadrant, $$\sin x\gt 0$$ and $$\cos x\lt 0.$$ Write $$\cot x=\dfrac{\cos x}{\sin x}=\dfrac{-5}{\sqrt{11}}.$$ Choose a common factor $$k$$ such that

$$\cos x=-5k,\qquad\sin x=\sqrt{11}\,k.$$ Using $$\sin^2x+\cos^2x=1$$ gives $$(-5k)^2+(\sqrt{11}\,k)^2=1\;\Longrightarrow\;25k^2+11k^2=36k^2=1\;\Longrightarrow\;k=\frac16.$$

Thus $$\sin x=\frac{\sqrt{11}}6,\qquad\cos x=-\frac56.$$

------------------------------------------------------------

1. Half-angle values

Because $$\dfrac{\pi}{4}\lt\dfrac{x}{2}\lt\dfrac{\pi}{2}$$ (first quadrant), both $$\sin\dfrac{x}{2}$$ and $$\cos\dfrac{x}{2}$$ are positive.

Using the half-angle identities:

$$\cos x=1-2\sin^2\frac{x}{2} \;\Longrightarrow\; -\frac56=1-2\sin^2\frac{x}{2}$$ $$\;\Longrightarrow\; 2\sin^2\frac{x}{2}=\frac{11}{6} \;\Longrightarrow\; \sin^2\frac{x}{2}=\frac{11}{12} \;\Longrightarrow\; \sin\frac{x}{2}=\frac{\sqrt{11}}{2\sqrt{3}}.$$

Then $$\cos\frac{x}{2}=\sqrt{1-\sin^2\frac{x}{2}} =\sqrt{1-\frac{11}{12}} =\sqrt{\frac1{12}} =\frac1{2\sqrt{3}}.$$

------------------------------------------------------------

2. Simplifying the required expression

Let $$A=\frac{11x}{2},\qquad E=\left(\sin\frac{11x}{2}\right)(\sin6x-\cos6x)+\left(\cos\frac{11x}{2}\right)(\sin6x+\cos6x).$$ Rewrite with $$A$$:

$$E=\sin A(\sin6x-\cos6x)+\cos A(\sin6x+\cos6x).$$

Separate the $$\sin6x$$ and $$\cos6x$$ terms:

$$E=\sin6x(\sin A+\cos A)+\cos6x(\cos A-\sin A).$$

Use the identities $$\sin A+\cos A=\sqrt2\,\sin\!\left(A+\frac{\pi}{4}\right),\qquad \cos A-\sin A=\sqrt2\,\cos\!\left(A+\frac{\pi}{4}\right).$$

Hence

$$E =\sqrt2\Bigl[\sin6x\sin\!\left(A+\frac{\pi}{4}\right)+\cos6x\cos\!\left(A+\frac{\pi}{4}\right)\Bigr] =\sqrt2\,\cos\!\left[6x-\left(A+\frac{\pi}{4}\right)\right],$$

because $$\cos(\theta-\phi)=\cos\theta\cos\phi+\sin\theta\sin\phi.$$ Substitute $$A=\dfrac{11x}{2}$$:

$$6x-\left(\frac{11x}{2}+\frac{\pi}{4}\right) =\frac{12x-11x}{2}-\frac{\pi}{4} =\frac{x}{2}-\frac{\pi}{4}.$$

Therefore

$$E=\sqrt2\,\cos\!\left(\frac{x}{2}-\frac{\pi}{4}\right).$$

------------------------------------------------------------

3. Evaluating $$\displaystyle\cos\!\left(\dfrac{x}{2}-\dfrac{\pi}{4}\right)$$

Using $$\cos(\alpha-\beta)=\cos\alpha\cos\beta+\sin\alpha\sin\beta$$ with $$\alpha=\frac{x}{2},\;\beta=\frac{\pi}{4},$$

$$\cos\!\left(\frac{x}{2}-\frac{\pi}{4}\right) =\cos\frac{x}{2}\cos\frac{\pi}{4}+\sin\frac{x}{2}\sin\frac{\pi}{4} =\cos\frac{x}{2}\cdot\frac{\sqrt2}{2}+\sin\frac{x}{2}\cdot\frac{\sqrt2}{2}$$ $$=\frac{\sqrt2}{2}\left(\cos\frac{x}{2}+\sin\frac{x}{2}\right) =\frac{\sqrt2}{2}\left(\frac1{2\sqrt3}+\frac{\sqrt{11}}{2\sqrt3}\right) =\frac{\sqrt2}{2}\,\frac{1+\sqrt{11}}{2\sqrt3} =\frac{\sqrt2(1+\sqrt{11})}{4\sqrt3}.$$

------------------------------------------------------------

4. Final result

$$E=\sqrt2\;\times\;\frac{\sqrt2(1+\sqrt{11})}{4\sqrt3} =\frac{2(1+\sqrt{11})}{4\sqrt3} =\frac{1+\sqrt{11}}{2\sqrt3}.$$

Hence the required value is $$\boxed{\dfrac{\sqrt{11}+1}{2\sqrt{3}}}.$$

Option B is correct.