Đáp án+Giải thích các bước giải:

Ta có:

$\sin^2 \alpha +\cos^2 \alpha =1$

`\Rightarrow` $\cos^2 \alpha =1-(\frac{3}{5})^2$

`\Leftrightarrow` $\cos^2 \alpha=\frac{16}{25}$

`\Rightarrow` $\left[\begin{matrix}\cos\alpha=\frac{4}{5}(loại)\\\cos\alpha=-\frac{4}{5}(nhận)\end{matrix}\right.$

(Do $\cos\alpha\in(\frac{\pi}{2};\frac{3\pi}{2})$

`\Rightarrow` $\cos\alpha=-\frac{4}{5}$

Theo đề bài thì:

$\cos(\alpha -\frac{21\pi}{4})$

$=\cos(\alpha-\frac{5\pi}{4})$

$=\cos\alpha.\cos\frac{5\pi}{4}+\sin\alpha.\sin\frac{5\pi}{4}$

$=-\frac{\sqrt{2}}{2}.\cos\alpha-\frac{\sqrt{2}}{2}.cos\alpha$

$=-\frac{\sqrt{2}}{2}(\cos\alpha+\sin\alpha)$

$=-\frac{\sqrt{2}}{2}.(-\frac{4}{5}+\frac{3}{5})$

$=(-\frac{\sqrt{2}}{2}).(-\frac{1}{5})$

$=\frac{\sqrt{2}}{10}$

Vậy $\cos(\alpha -\frac{21\pi}{4})=\frac{\sqrt{2}}{10}$