So, in my view your premise is flawed: about the extent to which KGB and Mossad (which are two very different kinds of intelligence services) sought to "hide their hand" in direct action operations like assassinations, and to what extent that's down to 'sloppiness'.

It's true that the Mossad (or more accurately, its political masters in the Israeli government) have at times tacitly allowed it to be understood that they were involved in assassination operations — because they intended those operations to act as a warning and as a deterrent. It's not that they were sloppy — though they certainly have been at times. The most notorious example is the Lillehammer affair, when Mossad operatives assassinated an innocent waiter in Norway, after mistaking him for a senior Black September member.

There's a good history of the Israeli intelligence services, written by two long-serving intelligence/national security journalists and with a particular focus on Israel's direct action operations: Spies Against Armageddon: Inside Israel's Secret Wars. It paints a rather more nuanced picture of Israeli intelligence operations than you normally receive.

The KGB are a much more complicated case. Yes, they used direct action widely, and were responsible for carrying out assassinations, or orchestrating them with the help of client intelligence agencies from elsewhere in the Warsaw Pact. But the Soviets never acknowledged those operations, and most of what we know about them is either the result of deduction, leaks from Western counterintelligence agencies, or the testimony of Soviet defectors to the West.

The CIA were heavily involved in those kinds of black operations, including assassination, during the mid-20th century, until a series of botched operations, political scandals and congressional investigations — most notably the Church Committee — left the Agency with much less political and operational latitude for those kinds of operations. Political assassinations were explicitly prohibited by Executive Orders 11905 (Gerald Ford, 1975) and 12036 (Jimmy Carter, 1978.)

The CIA has something of a checkered history, a case Tim Weiner makes pretty effectively in his history of the Agency, Legacy of Ashes. His essential argument is that its failures, at both the micro and macro levels, have outweighed its successes — and in some ways, I'm inclined to agree. (Judging by the blistering response the CIA gave when the book was published, it seems Weiner touched a nerve at Langley...)

The CIA have become substantially more militarised and the ban on assassinations has been significantly relaxed under the last two administrations, but that falls outside the 20-year rule for this sub, so I'll just suggest that you read Mark Mazzetti's The Way of the Knife and Jeremy Scahill's Dirty Wars for more on that front.

It's near impossible to do a quantitative analysis of which agency was more 'brutal' than the other, because intelligence history is a mosaic with lots of missing pieces, and because intelligence operations can have violent consequences far beyond the prick of the umbrella.

On a cultural or qualitative level, you could argue that the KGB were more brutal by the same rationale which allows us to call the USSR a more brutal state than the US: it was an agency of repression in an authoritarian state. It was certainly more violent against traitors in its own ranks than the Anglo-American agencies, for example. But violence is only the most noisy and visible, and among the least effective tools in espionage — and traditionally, it was one of the most rarely used. The vast majority of intelligence operations during the Cold War were a different kind of combat.