"A usurer, a miller, a banker and a publican, are the four evangelists of Lucifer" - from an old book of proverbs. Why hate on the miller?

I'm not sure when and where your book is from, but at least in 14th century England, millers seem to have had a bad reputation for unfair business practices and shady dealings. You can see this in Chaucer's The Canterbury Tales, especially in the prologue to The Miller's Tale and in The Reeve's Tale. Chaucer's millers were shady dudes: bellicose, violent, drunken and/or lustful, and most importantly, larcenous. From The Reeve's Tale (in idiomatic modern English):

A long knife by his belt was always seen,

Also a sword, no other blade as keen;

His pouch contained a dagger. He was such

That none, for fear of death, would dare to touch

Him. He'd a Sheffield knife inside his hose.

Round was his face, and snub-like was his nose,

His head as bald as any ape's. He fully

Was a swaggerer, a market-bully,

And on him none a finger dared to lay

Or one would pay, he promised, right away.

He also was a thief of corn and meal,

A sly one, too, his habit was to steal.

A little bit of context on the subject. Basically, when you, Joe Farmer, brought your grain to the miller to be ground into flour, it was understood that the Miller would take a certain percentage of the flour as payment for the use of his mill. The trope, as embodied in Chaucer, was that the miller would utilize various tricks and deceptions to take more than his agreed share, thus enriching himself at his customer's expense. An analogous situation in modern life might be a check cashing store or a pawn shop that use technicalities to siphon off the top.