In this paper, we propose a temperature-insensitive polarimetric fiber pressure sensor based on a polarization-diversity loop structure (PDLC) composed of a polarization beam splitter, half- and quarter-wave plates, high birefringent fiber (HBF), and a fiber Bragg grating (SFBG). The proposed sensor can measure pressure applied to a sensor head by observing the wavelength shift of the output interference spectrum of the PDLC. The sensor head is composed of 10-cm-long HBF and an SFBG directly spliced with HBF. HBF is used as a birefringence element in the PDLC to generate an interference spectrum through polarization interference. When external pressure is applied to HBF, the HBF birefringence is changed, and the absolute wavelength location of the output interference spectrum of the proposed sensor also varies along with the birefringence change. Therefore, the applied pressure can be obtained by measuring this wavelength shift and converting it using the precalculated pressure sensitivity. However, HBF has a cross-sensitivity to pressure and temperature. To tackle this issue, an SFBG was added to HBF and used to measure ambient temperature changes in HBF. Because the reflection spectrum of the SFBG is hardly affected by applied pressure, the SFBG can measure a temperature change independently of pressure even if pressure and temperature are simultaneously applied to the sensor head. The pressure sensitivity of the sensor was measured as approximately -12.713 nm/MPa, and a R2 value was measured as ~0.9986 in a measurement range of 0-0.5 MPa. By concatenating the HBF segment and the SFBG, the proposed sensor could solve the cross-sensitivity of HBF to pressure and temperature. In particular, the birefringence modulation of HBF due to external environmental perturbations could be minimized by employing short birefringent fiber, of which length is the shortest in comparison with the sensor head lengths of previously reported fiber polarimetric pressure sensors.