I can comment most extensively on the financing of the American Revolution, but I might leave it to the likes of /u/irishfafnir to comment on the early republic stuff.

As you say, the newly United States had to fund a major war effort as it's first order of business, even before it could figure out any kind of constitution (which it didn't manage to to until 1781). Given the all-consuming American reluctance to grant a central government much real power, he Continental Congress did not have the power to tax either the states or the American people directly. It could, and did, apportion the tasks of funding each states' troops to that state, and requesting money from the states for more general tasks. Some states took these obligations seriously, while others would not or could not meet the demands set on them. As such, a soldier from Massachusetts, which firmly supported the rebellion and enjoyed an open harbor, was much more likely to be better equipped and more frequently paid than one from New York, a state with deep political divisions and whose major port city remained in British hands throughout the war.

As the war dragged on, the lack of steady income for the government became more and more problematic. Much of the gap was filled by both gifts and loans from the French crown, which entered the war in 1778. More slack was taken up by two wealthy brothers, Robert and Goveneur Morris, who printed paper money backed by their own fortunes on behalf of the Continental Congress. Even then, the American dollar suffered from runaway inflation. The problem persisted until the Washington presidency, when Alexander Hamilton backed the dollar with land rather than gold.