In some ways, your last statement is correct and answers the question in itself . Generals have more to lose and less to gain unless the coup is entirely ideological. But if we dig into the details, then we can learn a bit about how coups function, and that explains why it's often colonels.

First of all let's argue with the premise of the question. One semi-recent work on coups -- Naunihal Singh, “Seizing Power: The Strategic Logic of Military Coups” (2014) -- disagrees with the “colonels” theory, and says instead that senior officer coups are more likely to be successful. This is because they have greater influence and connections, which Singh believes are at heart of any coup attempt succeeding.

Support for Singh's view can be seen in how many coups failed, rather than just how they succeeded. This includes the Cherry Blossom Society coup attempts that you mentioned: Japanese colonels who planned a coup but backed off when they realized senior military leaders would not support. Then in the 1936 “February incident” the ringleaders – lieutenants and captains – started assassinating political figures and occupying Tokyo. As with many/most coup attempts, the first stage included military leaders (including generals) feeling out whether it would succeed, and as long as it had forward momentum it kept going. But when it stalled, it lost support because nobody wants to be on the losing side. Then when the emperor opposed it, support vanished and it was crushed. So in the failed coups, we see counter examples to the colonel theory.

That aside, it does seem -- particularly in African and South American cases, and cases after 1945 -- that the colonel theory has some weight to it. Why would that be?

Generals are inherently political creatures, and they are therefore close to the existing power structure the coup intends to overthrow. They got their position through connections, potentially through patronage, and they themselves enjoy powers of patronage and influence. They’re less likely to act against a system that works for them.

Conversely, those on the bottom of the military structure are young, inexperienced, and have few resources and connections. They have the numbers, but not the power to organize anything comprehensive or carry it out. (Consider a counter-example: the naval mutinies that brought down imperial Germany at the end of WWI. This was an act of specific resistance at specific locations by a mass of enlisted sailors, but it was not a military coup in any sense. Rather, it was a local military uprising that then combined with labor activism, and spread across a country whose government was already reeling form the war. That government then abandoned the country and deposed itself, leaving a new civilian government in charge.)

That leaves colonels in the sweet spot – they have access to resources but are not on the top of the system. Therefore they have room for ambition. They may have failed to gain higher office, have resentments against those above them, but they're sufficiently powerful to amass a base of support.

This brings us to the most important point: the logistics of a coup, and indeed any military operation. The colonels are the ones that are closer in touch with the operational and logistic realities of the military. Generals may control resources on a national or strategic level, but colonels translate strategy into operations, and direct the tactics of the level below them. So they can become dangerous when disgruntled, because they have the resources to make and carry out plans out of the view of a perhaps-distant headquarters bureaucracy.

Colonels have lateral connections to each other, having grown up through the system together. For coup planning purposes, they can coordinate among themselves without the upper brass knowing. The social dimension can't be overlooked. A lot of the dynamic of coups (this is a big point in Singh's work) is that they are essentially guessing games among those with power, many of whom wait to declare their allegiance until they see who will win.

In other words, they are often the big boss of their local military area. Think of the US Air Force – where colonels are called the “wing king” and they’re essentially the highest ranking officer that the rank and file owe loyalty to on a daily basis. They are a face that the lower ranks actually recognize and feel personal loyalty to, versus a distant general who they don’t know, and now they’re being told is a corrupt tool of a bad system. That's often credited as a factor in this theory, that they can mobilize personal loyalty that higher levels can't. A lot of coups and revolutions come down to a critical moment when a small military force has to decide to shoot or not. Personal loyalty increases that chance.

Colonels are therefore able to command both the resources, the personnel, and the organization on a level to accomplish practical operations needed to make a coup function. That's perhaps why - if the theory is true in the first place - their coup attempts succeed more often than those from higher ranking officers.

(As a final aside, which I couldn't work into the above answer but how can I not mention it at any opportunity: look up the case of Yukio Mishima, a gay poet warrior whose civilian militia attempted a coup against the Japanese military, then planned to use the military to stage a coup against the civilian government. Wild stuff.)