Rocky desertification is an urgent ecological problem in karst areas of Southwest China. The control of rocky desertification leads to the transformation of soil carbon and has an impact on land carbon cycle. However, the effects mechanism of soil and water conservation measures on soil organic carbon in rocky desertification areas is not clear. Three soil and water conservation measures—Nested Ladder (NL), Fish Scale Pit (FSP) and Terrace (TR) in typical rocky desertification area were selected, and compared with natural slope (CK) to study the distribution law of soil organic carbon content. The results showed that the organic carbon content of FSP was 1.4 and 6.2 times higher than that of NL and TR respectively. The organic carbon storage in the 0—20 cm soil layer of the FSP was higher than that in NL and TR by 30.78%, 444.44% respectively. The soil active carbon content of FSP was also significantly higher than that of the other two water and soil conservation measures, so its carbon sequestration effect was the best. The storage of organic carbon was significantly positively correlated with the easily oxidized carbon, soluble carbon and microbial biomass carbon (p < 0.01), the easily oxidized carbon was significantly positively correlated with soluble carbon and microbial biomass carbon (p < 0.01), and the soluble carbon was significantly correlated with microbial biomass carbon (p < 0.05). The increase of carbon nitrogen ratio in FSP and NL would promote soil organic carbon sequestration. The study shows that FSP can be used as a priority control measure in the ecological restoration and construction of rocky desertification areas in the future.