As you have alluded to, you would need an extraordinary amount of money to finance a castle during this period. For example: the average income of King Henry II was anywhere between £10-20,000 annually(not adjusted for inflation). To put this in context, Rochester Castle is estimated to have cost approximately £3,000 and nearly a decade to construct. Therefore, in the century before your selected time period, only a small number of people in England could even attempt such a project.

Most "castle builders" are lost to us, with the exception of Maurice the Engineer, who was known to be the preferred castle builder of Henry II. Maurice was responsible for both the construction at Dover and Newcastle.

So, what were castles made from? If we stick with Rochester as an example we know that it was primarily constructed from Kentish ragstone, likely quarried from nearby Maidstone. Other castles from this period had stone shipped from France, including Caen stone, which was used to carve fireplaces and windows. Stones for castles were most readily shipped via river.

But let us say that you are not the king, nor are you a high ranking member of the aristocracy. How would you go about affording the expense of a castle? Sir Edward Dallingridge provides us some insight through his vanity project castle at Bodiam. By 1385, Sir Edward found himself well-married, with strong political ties to the local area. When he chose to build a castle in 1385, his estate amounted to approximately £200 per year. To afford the construction Sir Edward employed a variety of means, including the sale of his wife's properties, loans, and from a healthy bit of war profiteering during the Hundred Years War. Sir Edward also petitioned, and was granted, a license for his castle from the monarch, which granted it official status as a "castle" (very important for appearances sake) and allowed Sir Edward to stake out the moral high ground as a defender of the local area in the eyes of the king.

I hope this answers your question, OP. I have steered away from the obvious strategic and political significance of castles during this period, and attempted to focus on your practical questions.

Source: Marc Morris, Castle: A History of the Buildings that Shaped Medieval Britain (London: Windmill, 2003)