Yay, a use for my architectural history minor at last!

The simple answer is that architectural trends (and aesthetic trends in general) tend to follow trends in economic and political power. Britain was the most powerful country in the world, economically and politically, during the 19th century, and this meant that trends in British architecture proliferated to the rest of the English-speaking world. Of course, the full story is a bit more complex.

Victorian architecture reflects, first, the emergence new architectural forms made possible by the transition away from an artisanal/cottage method of construction to largely industrial methods of construction. Second, Victorian architecture reflects a partial drift away from the "classical" Greek and Roman architectural idiom which dominated Europe and (later) the United States from Palladio really through the early 19th century. This went through ebbs and flows mainly dealing with how designers made use of curvature and ornamentation, but the fundamentals of the classical orders remained largely unquestioned. Even in England, where Gothic architecture had never truly died out, at least in religious architecture, classicism was far more fashionable.

To address the first point, the Industrial Revolution started in England, which dramatically transformed the country within the span of about a century from a largely rural one to a heavily urbanized one. Prior to the 18th century, urban population growth had been relatively slow for quite a long time. With industrialization, cities began to grow exponentially, and became physically much larger to accommodate increased populations, and economic production which was more heavily urbanized. Consequently, industrial methods were adapted to produce buildings as well. By applying industrial techniques, buildings could be constructed more quickly, easily, and precisely. Stones no longer needed to be quarried and hauled for each project. Bricks in standardized sizes could be purchased from huge industrial brickworks. Wood was processed at industrial sawmills, rather than being felled and hewn to order. The construction of a house ceased to be an organic process carried out by the homeowner and perhaps some friends and local artisans, and began to be a more industrial one, capable of being reproduced with essentially interchangeable parts.

In large-scale architecture, this shift is even more pronounced. Once iron and glass could be manufactured precisely, rather than wrought by a blacksmith, architects and engineers began to experiment with its structural potential almost immediately, first with bridges and greenhouses, and then with buildings. In 1864, Oriel Chambers, the first building with a metal structure and glass curtain wall (like a modern skyscraper) was completed in Liverpool. The building was mocked at the time, but its techniques were quickly adapted.

Across the Atlantic, where buildings are made of wood instead of masonry, an immigration boom was underway, and the largest industrial cities in America struggled to keep up with the growing demand for housing. Sometime in the 1830s, probably in Chicago, carpenters realized that long studs of consistently milled dimensional lumber could be used to erect multi-story buildings quickly through a method called balloon framing. Wood framing predates the balloon frame, but balloon framing makes it easy to build larger wooden buildings faster, provided you have easy access to lumber. The US, with its massive and then-untapped old-growth forests, did.

This enabled the shift in my second point. The construction techniques of Gothic architecture, which used intricate masonry work to create soaring vertical spaces, had been somewhat lost to time by the 19th century. Classical architecture relied on an (originally) structurally-determined order which was much better understood, and idealized as the "rational" architecture of the Enlightenment, a symbolic meaning that was embraced by both the young US and revolutionary France, neither of which were well regarded by many Brits. Gothic architecture, which had been unfashionable, began to undergo a rehabilitation in many parts of Europe , due in part to Romanticism's backlash against industrial society's alienation of the individual. Gothic architecture represented a prior flourishing of architecture dominated by the artisan, and was, conveniently, also the "national" architectural heritage of Britain (Germany and France also claimed it), the dominant power of the era. Ironically, it was the adaptability of industrial construction that enabled that allowed a Gothic Revival to take place. As craftsmen rediscovered the techniques of Gothic construction, architects began to expand its stylistic cues beyond what could be rendered in stone. Iron-frame construction was used to enable the construction of some of the grandest Neo-Gothic buildings of the era, like Manchester Town Hall and St. Pancras Station.

In the United States, the aesthetic trends which had begun in Europe spilled over. Gothic styling became popular in the US as well, often rendered in wood. In residential architecture, wealthy Americans borrowed elements of European architecture, again, usually rendered in wood. As the 19th century progressed, more elements fused into the architectural milieu, with the Arts and Crafts Movement and the Eastlake Movement both beginning in Britain before being adopted in America. Even the Italianate tendency within 19th century American architecture reflect Italian taste as interpreted by British architects like Thomas Cubitt, who designed many of London's stylish neighborhoods, and even Prince Albert, who designed Osborne House himself in an Italian style. The Paris-oriented Beaux-Arts style was influential in both Britain and America, but is largely treated separately.

Basically, Britain under Queen Victoria, as the dominant country of the second half of the 19th century, played a crucial role in creating and disseminating the key architectural innovations which define what we now call Victorian architecture: industrial production of materials, a resulting increase in standardization of techniques and designs, and the re-entry of non-Classical design influences into Western architecture, including a greater emphasis on verticality through the innovative use of these materials and techniques.