An anapole state that breaks inversion and time-reversal symmetries while preserving translation symmetry of an underlying lattice has aroused great interest as a new quantum state, but only a few candidate materials have been reported. Recently, in a spin-orbit coupled Mott insulator ${\mathrm{Sr}}_2\left({\mathrm{Ir}}_{1-x}{\mathrm{Rh}}_x\right)O_4$, the emergence of a possible hidden-order phase with broken inversion symmetry has been suggested at $T_{\Omega }$ above the Néel temperature by optical second-harmonic-generation measurements. Moreover, polarized neutron diffraction measurements revealed broken time-reversal symmetry below $T_{\Omega }$, which was supported by subsequent muon spin relaxation experiments. However, the nature of this mysterious phase remains largely elusive. Here, we investigate the hidden-order phase through the combined measurements of the in-plane magnetic anisotropy with exceptionally high-precision magnetic torque and the nematic susceptibility with elastoresistance. A distinct twofold in-plane magnetic anisotropy along the [110] Ir-O-Ir bond direction sets in below about $T_{\Omega }$, providing thermodynamic evidence for a nematic phase transition with broken $C_4$ rotational symmetry. However, in contrast to the even-parity nematic transition reported in other correlated electron systems, the nematic susceptibility exhibits no divergent behavior towards $T_{\Omega }$. These results provide bulk evidence for an odd-parity order parameter with broken rotational symmetry in the hidden-order state. We discuss the hidden order in terms of an anapole state, in which the polar toroidal moment is induced by two current loops in each ${\mathrm{IrO}}_6$ octahedron of opposite chirality. Contrary to the simplest loop-current pattern previously suggested, the present results are consistent with a pattern in which the intra-unit cell loop current flows along only one of the diagonal directions in the ${\mathrm{IrO}}_4$ square.