What about men holding polearms or flags? I don't about harry potter but if a door is at the top of a steep staircase it is intended to be a defensive position. Usually these doors aren't very tall but in the case of fight over the threshold being able to lower a pike would be advantageous, and the extra clearance provides room for missiles as well. Most extra tall doors wouldn't lead indoors but to an inner courtyard, and yes they would normally be for horsemen.