How did Machiavelli's reforms of Florence's military work?
First you have to consider exactly what the Florentines were using before Machiavelli’s reform; mercenaries. He didn’t so much reform the Florentine military, as replace the hired fighters with what we’d call a citizen militia. Recruiting from the people of Florence itself was politically unpalatable, and recruiting form a city under Florentine rule was unwise – those cities were not reliable. Instead he choose to recruit from the towns of Mugello and Casentino.
To take charge of the militia, he choose Miguel de Corella – a former lieutenant of Cesare Borgia – which was a controversial choice, but effective; under de Corella’s leadership, they trained between ten and sixteen times a year (a decision of Machiavelli’s – he had a hand in every part of their organisation right down to the uniform). Luca Landucci, a contemporary, who witnessed their first parade, wrote that they were organised into “battalions: they were given a constable who would lead them, and teach them how to use arms”.
Matters of the chain of command were less clear; the Ten of War ostensibly held control over military matters, but in practice Florentine politics was a mess, and the Signoria liked to stick its nose into everything.
As a side note, their organisation wasn’t so much Roman; it was more based on contemporary Spanish, Papal, French, and Venetian forces.
How effective were their Roman style legionaries?
This is difficult to say, as to the best of my knowledge they only saw battle twice; they weren't around for very long (from 1505/06-1512), and the Florentine Government tended to use diplomacy rather than military force against its main threats (usually France, Spain, the Papacy, and occasionally the Holy Roman Empire).
The first was the siege of Pisa in 1509, where the militia did what the mercenaries could not, and took the city; Machiavelli was raising levies in 1508, and the city fell very quickly. With this case you could argue a decent level of effectiveness; the routing of multiple mercenary forces by the Pisans is what provided the impetus to form the militia in the first place, and with what were essentially raw recruits, they succeeded where the coin-hires couldn’t. But one data point doesn’t make a trend.
The second time was less favourable; in mid-1512, the Spanish under the leadership of Ramon de Cordona had laid siege to Prato, and attacked it in August. In that first engagement, the militia successfully fended off the Spanish assault; although the Spanish were exhausted and hungry, and de Cordona offered to negotiate because of this, which does detract from that victory somewhat. The Florentines refused, and on 30 August the Spanish soldiers stormed the fortress, sacked the city, and slaughtered the defenders. Machiavelli himself reported that the defenders were routed and fled before the Spanish; in fear of what they'd do to Florence, the government surrendered to de Cordona. In September, the Medici, now back in power, had Machiavelli’s militia abolished.
My main source for this is; White, Michael. 2004. Machiavelli: A Man Misunderstood. United Kingdom: Little, Brown, but also with reference to what I have of Machiavelli's letters (very little, admittedly), and, of course, The Prince.