This might be a better question for /r/Economics, but here's an ELI5:

A government that has a track record of taking on, then paying off debt is considered more safe to do business with. This is similar to personal debt - it's much easier to buy a car or rent a house if you've shown that you're responsible in managing debt (your credit history). In addition, buying up governmental debt (especially those of stable countries like the US) is considered a good investment in some ways - returns of 3-4% with almost no risk. This encourages people to lend to the government, which will then spend it on various projects, thus keeping the money in circulation instead of having it sit in a (metaphorical or literal) safe somewhere.

I'll preface this by saying that I'd love to hear from someone who knows more about this than I do, but I'll give an answer a go as far as is my understanding.The quote in question comes from Hamilton's letter to Robert Morris in 1781.

In it, Hamilton was proposing the creation of a national bank which would guarantee the wartime debts of the revolutionary government, thus establishing a secure line of credit for the United States as well as confidence in its paper money.

The quote comes at the end of the letter, when he's discussing how and why this debt ought to be paid off: "A national debt if it is not excessive will be to us a national blessing; it will be powerfull cement of our union. It will also create a necessity for keeping up taxation to a degree which without being oppressive, will be a spur to industry; remote as we are from Europe and shall be from danger, it were otherwise to be feard our popular maxims would incline us to too great parsimony and indulgence. We labour less now than any civilized nation of Europe, and a habit of labour in the people is as essential to the health and vigor of their minds and bodies as it is conducive to the welfare of the State. We ought not to Suffer our self-love to deceive us in a comparrison, upon these points."

In context its worth pointing out how this conforms with some of Hamilton's later views. In his "First Report on the Public Credit" after the war in 1790 Hamilton proposed that not only should all of the debt taken on during the war be recognized and taken on (there had been discussion of giving a "haircut" as we would now call it to speculators) but that the federal government should also pay assume and pay off the debts of the individual states. These state debts were far larger than the federal debt.

This had the effect of not only ensuring that the United States would continue to have the confidence of its creditors but that the states would be, as he had written in 1781, be cemented to the federal government.