As you might expect, you get what you pay for with these books. For one, they're not the most durable. Second, thrift books are going to be very bare bones. For instance, there aren't any supplements in the Dover edition of The Prince. In all honesty, it's not worth the $3 unless you just like to physically hold a book. You could get the exact same thing online, for free, at a website like Project Gutenberg.

Now, if you spend a bit more money, there are very good editions of the classics. Take, for instance, Cambridge University Press' edition of The Prince, which you can get for $15. It comes with an introduction by one of the world's leading scholars on political philosophy, notes on the translation, and copious footnotes to provide context and further information to the reader. These features are important, especially for novices who might not know the historical context of the book or who might want suggestions for further reading.

On another note, I wouldn't call classics like The Prince and the Republic history books. They're certainly historical, but they're not history books. You might get better answers to your question over at /r/askphilosophy.

Instead of reading the classics, my suggestion would be for you to read a history book about the classics, something like Anthony Kenny's New History of Western Philosophy, which is a great introduction to the topic.

It depends what you mean by "good". That texts are in the public domain (in the US) means that they're from before 1924; in the case of something like The Prince, that means the translation is from before 1924. That means that the language may be quite dated, and as /u/gent2012 says, the text as a whole will lack annotations and introductions written by modern scholars. On the other hand, if you're looking at fiction written in the past (which is the majority of these printed public-domain books), there's really no quality difference. These are cheap for the publisher to produce because they need no editing and are free for them to use, but it's not like they're actively making the text worse.

However, if you want to read public domain literature, you really don't need to pay for it. Go to Google Books, set the search type of "free Google ebooks", and look for whatever title you're interested in. It'll have to be read on an electronic device of some kind, but you won't have to feel you've wasted your money if it turns out that you wanted a version with more contextualization instead.