To study the mechanism of vegetation affecting gravity erosion may provide the theoretical basis for the design of soil and water conservation on the Loess Plateau, China. This study explore the effects of vegetation on the gravity erosion using the models of loess gully sidewall with the slope degree of 70° and height of 1.5 m under a series of rainfall simulations, in which vegetational and bare lands were used, respectively. The experimental results are shown as follows:(1) The influence of the vegetation on the total amount of gravity erosion might be ignored. The average amount of gravity erosion for each event of rainfall on the vegetational sidewall was only decreased by 12% compared with that on the bare sidewall. (2) The vegetation had a positive effect on the gravity erosion after rainfalls. The ratio of the gravity erosion volume after rainfalls to the total on the planted slope was almost 12%, while the ratio on the bare slope was only 1%. (3) The vegetation on the gully sidewall had different influences on various types of gravity erosion. Compared with those on the bare land, the average amounts of the avalanche and mudslide on the vegetated slope was decreased by 72% and 69%, respectively. In the meantime, the average amount of landslide in the vegetated slope was increased by 220% compared with that on the bare slope. (4) The effects of vegetation on gravity erosion were caused by several factors. The plant roots might reinforce the soil and reduce the amount of gravity erosion, but the self-gravity of the plant and increase of the permeability on the planted slope might aggravate the gravity erosion. In addition, the soil permeability was increased by the plant roots, which could intensify the landslides. As a result, the vegetation construction was preferable for the sidewall vulnerable to mudslide. The results might be referred to analyze the mechanism of mass failure and design the control of gravity erosion.