I don't think it would have been feasible. While I suppose they could have moved down from Canada, moving over those kinds of forces and establishing a foothold would have been hilariously difficult. They closest they got was burning the capitol in 1812, and even then they were quickly forced out (admittedly by an act of god so hilariously dramatic that it could be argued their capturing DC at that point in time aided the US by forcing an evacuation).