Yep, it would absolutely have been considered legal. Disguising one's ship or approaching under a false flag was a legitimate ruse de guerre, as long as you would display your own flag/identity before firing on another ship. Not doing so might result in being considered a pirate, though if you were a commissioned officer, an angry diplomatic note might be the only retaliation.

But keep in mind that there was no such thing as an international court to adjudicate these things; damages for piracy and related actions would generally have to be recovered from a court in the offending nation, or by recourse to a letter of reprisal. For example, if I were a French shipowner sailing out of Toulon, and were attacked by a British pirate, I would have to either sue in British court to attempt to recover damages, or, alternatively, ask the French government for a letter of reprisal allowing me to seize goods in kind from another British ship.