The standard answer: resources and markets.

Production (whether in an agricultural economy, a marxist economy, or an industrialized, capitalist economy) require 4 things:

a) capital inputs (think machinery, or seeds); b) land (you need to put your machinery on something, or you need to grow plants); c) labor (you need somebody to run the machine or plant the seeds); d) entrepreneurship (somebody needs to find the capital, land, and labor).

Prior to the industrial revolution, economic growth was driven by changes in land-labor ratio (your analysis is correct, especially in pre-modern times.) An economy expands at a sustainable rate if that ratio stays relatively constant (land is generally constant). If it is falling, chances are there will be a recession in the future (population increase). If it is rising, growth is uncertain (could be during a famine, or a time of conquest). The basic idea is that the population cannot grow faster than production. See the Malthusian Economic Growth Model if you want more details.

However, the Industrial Revolution changed things. Capital accumulation became more important. Industrializing economies are capital intensive and capital owners desired returns to scale. To achieve this, they needed large markets (conquering another country can their markets for your goods) and cheap inputs (the powers needed coal, and other capital inputs.) Land was much less important (a factory could require less land than a large farm).