I can't really speak for Broadway, but the prominence of the West End in London goes back through several centuries of Royal Patronage.

London's renaissance theatre (Shakespeare etc.) was hugely popular, but came into conflict with the increasingly Puritan aftershocks of the Reformation. This came to a head in the Interregnum, when most theatrical performances were banned. Following the Restoration, Charles II sought to bring back drama to London.

He established the Theatre Royal on Drury Lane, which became the first proper theatre in the West End (previously performances were held in converted tennis courts and similar). As well as the sponsorship of the king, the two companies based at the Theatre Royal enjoyed a monopoly over 'legitimate drama' which lasted for around two centuries. This focused the centre of London drama in the west end.

Until 1843 the monopoly held, and so other theatres could only host 'low-brow' performances such as musicals. Due to the limits on performing 'legitimate plays' the other theatres which opened in the West End had to innovate in their content. In the 18th century, this tended to mean so called 'Restoration comedies' and in the 19th things like pantomimes, music hall and farces. It also meant that they remained populist in their output, and consequently, popular. They also benefited from industrial innovations such as stage lighting and cooling systems. The West End was largely created by enterprising theatre owners in this period.

The theatres themselves were physically sited in the west end as this was largely where the patrons lived and wanted them. Through the 18th and 19th centuries, the west end was where most aristocrats, merchants etc who sponsored the arts lived, whilst also allowing access to the lower classes who bought the tickets.

As time went on the West End had the theatres, the audiences and the money and so continued to be 'the place' for English plays. When the monopoly on drama was abolished in the 1840s, it was natural that the other west end theatres would take on this mantel.

TL;DR: Royal patronage, geography/demographics and innovative business practices.

Further reading: The Cambridge Guide to Theatre Earl, John and Sell, Michael Guide to British Theatres 1750–1950