Very interesting question. I believe the answer lies in Newton's third law of motion. The bars are made thick to prevent a prisoner breaking out.
Case 1: Assume the bars are horizontal. In this case, the prisoner has a better chance to bend the bars as the ground gives the desired 'normal reaction' to the prisoner against the force applied by the prisoner to bend the bars. This makes it easy to achieve the result.
Case 2: Assume the bars are vertical. The force required by the arms (which are way weaker than the thighs) is the same, but the arms require better leverage than thighs to bend the bars, which makes it way more difficult to break free.
It requires less steel. Consider a 3x4 grid the proportion of a prison door. If I put bars vertically, I'd need only 2x5=10 units of rod. But maintaining the same gap if I put bars horizontally, it'd take 3x4=12 units. This is true for all tall rectangles. Vertical configurations require less steel. Less steel equals less cost.
Also, if the bars were horizontal, the prisoners would climb up the bars and hop on them until they buckled and came off their sockets.
It's a lot easier to use one's weight to apply a force than to apply it with one's arms, as is the case for vertical bars.
One may also consider static structural issues, like when the bars are vertical they will support the wall structurally in which the hole has been made to allow air and light inside.
And there might also be a psychological or affordance-based reason, namely that the horizontal bars would allow the prisoner to look from side to side over the horizon and have a sense of 'free range' from that.
The vertical bars will obstruct this visual sense of freedom and 'clip' the visual space into smaller bits.
I am guessing that could be an 'effect' subconsciously wanted by the imprisoner.
Martin Kofod Ludvigsen, designer of social spaces
Firstly, so that prisoners can lean against bars and play Noughts and Crosses on the horizontal stripes they wear.
Secondly, it's easier to hold the bars and scream "Let me out!".
And, on a serious note, I think the bars are vertical so people don't hang themselves. It's hard to tie a noose to vertical bars.