It has to do with several of the things you mentioned. To be brief, the decision to have Election Day on a Tuesday was made when a vast majority of the people living in the United States were farmers. It could have taken at least a day for a farmer to travel to the relevant populated center or the county seat to go and vote. They therefore needed a two-day period to travel, to vote, and then to return. Most people attended church on Sundays, making the weekend rather impractical. Furthermore, Wednesday was commonly reserved for market days in small towns. Therefore the two-day period of Monday and Tuesday was seen as the most practical time for farmers to travel to their polling place without interrupting their business and their private lives.