Air does not just flow horizontally -- it also flows up and down. The motion is not separated into only horizontal and only vertical flow, we simply break it up that way for simplicity. There are three major forces that influence vertical motion -- the vertical PGF, gravity, and friction. There are also three other factors influencing vertical motion -- convergence, buoyancy, and terrain.
This pressure gradient is about four times larger in magnitude than the horizontal PGF. However, it is also balanced by a strong force -- gravity.