Given:
$\sec\theta=\frac{17}{8}=\frac{17\text{k}}{8\text{k}}$
Let us draw a $\triangle\text{ABC},$ in which $\angle\text{B}=90^\circ$ and $\angle\text{A}=\theta$

By pythagoras theoram we have
$\text{AC}^2 = \text{AB}^2 + \text{BC}^2$
Or $\text{BC}^2 = \text{AC}^2 + \text{AB}^2$
$\therefore\text{BC}^2=\big(17\text{k}\big)^2-\big(8\text{k}\big)^2$
$=289\text{k}^2-64\text{k}^2=225\text{k}^2$
$=\text{BC}=15\text{k}$
$\therefore\ \sin\theta=\frac{\text{AC}}{\text{BC}}=\frac{15{\text{k}}}{17\text{k}}=\frac{15}{17}$
$\cos\theta=\frac{\text{AB}}{\text{AC}}=\frac{8\text{k}}{17\text{k}}=\frac{8}{17}$
$\tan\theta=\frac{15\text{k}}{8\text{k}}=\frac{15}{8}$
$\text{L.H.S.}=\frac{3-4\sin^2\theta}{4\cos^2\theta-3}$
$=\frac{3-4\times\big(\frac{15}{17}\big)^2}{4\times\big(\frac{8}{17}\big)^2-3}$
$=\frac{3-\frac{4\times225}{289}}{4\times\frac{64}{289}-3}$
$=\frac{\frac{3\times289-4\times225}{289}}{\frac{4\times64-3\times289}{289}}$
$=\frac{867-900}{256-867}=\frac{-33}{-611}=\frac{33}{611}$
$\text{R.H.S.}=\frac{3-\tan^2\theta}{1-3\tan^2\theta}=\frac{3-\big(\frac{15}{8}\big)^2}{1-3\times\big(\frac{15}{8}\big)^2}$
$=\frac{3-\frac{225}{64}}{1-3\times\frac{225}{64}}=\frac{\frac{3\times64-225}{64}}{\frac{64-3\times225}{64}}$
$=\frac{192-225}{64-675}=\frac{-33}{-611}=\frac{33}{611}$
Hence, L.H.S = R.H.S.