It's an extension of the powers of governors and kings to do so, which comes from practices in English common law that date back well before the founding of European colonies in the Americas (the "Royal prerogative of mercy"). It is a hedge against the potential for injustice inherent in a flawed judicial system, especially one that depends substantially on eyewitness testimony and the logical/rational powers of ordinary humans (whether they are judges or jurors). It's especially important when, as was common, a significant number of offenses were considered capital crimes. An executive could commute a sentence or grant clemency based on a strong suspicion of a miscarriage of justice, which were, even more than today, not at all uncommon. And to some extent it was a counterbalancing power against the power of the legislative to make laws and the judicial branch to try offenses.

Additionally, it's a way to allow the head of state to smooth over potential legal difficulties in service to the national interest. For example, a powerful figure may commit "treason" during a period of civil war or conflict, though later the government may desire to bring them back into the fold for pragmatic reasons. The ability to simply have the head of state make such things kosher from a legal perspective is helpful.

These factors were all in the minds of those writing the constitution, and these various points are used as arguments in support of such powers in the constitution in the Federalist Papers (specifically #74 by Hamilton).

From Federalist 74:

(in regards to compensating for injustice):

He is also to be authorized to grant "reprieves and pardons for offenses against the United States, EXCEPT IN CASES OF IMPEACHMENT.'' Humanity and good policy conspire to dictate, that the benign prerogative of pardoning should be as little as possible fettered or embarrassed. The criminal code of every country partakes so much of necessary severity, that without an easy access to exceptions in favor of unfortunate guilt, justice would wear a countenance too sanguinary and cruel. As the sense of responsibility is always strongest, in proportion as it is undivided, it may be inferred that a single man would be most ready to attend to the force of those motives which might plead for a mitigation of the rigor of the law, and least apt to yield to considerations which were calculated to shelter a fit object of its vengeance.

...

(in regards to the use of pardons to quell insurrection):

On the other hand, when the sedition had proceeded from causes which had inflamed the resentments of the major party, they might often be found obstinate and inexorable, when policy demanded a conduct of forbearance and clemency. But the principal argument for reposing the power of pardoning in this case to the Chief Magistrate is this: in seasons of insurrection or rebellion, there are often critical moments, when a welltimed offer of pardon to the insurgents or rebels may restore the tranquillity of the commonwealth; and which, if suffered to pass unimproved, it may never be possible afterwards to recall.