There are a lot of reasons, but a great deal of it comes down to allocation of resources and economic efficiency.
Soviet Agriculture wasn't completely terrible and the base theory actually makes some sense. A collective farm where you grab all the plots too small to use a tractor effectively and make one big tract of land where large investment in machinery actually makes sense isn't terribly different from current American Agribusiness. The small farmer in the US has been squeezed by capital requirements just as much as their Russian counterpart. The problem comes in implementation. "Model Farms" in both Soviet and non-communist nations tend to work. If everyone has buy in and the capital investments are spot on then the framework works. If those components, aren't there... well, let's just say that the return is terrible. Soviet Sources from the 1980's claim that Russian Agriculture was only yielded 20-25% per worker when compared to American Agriculture and that "Private Auxiliary Holdings" (that 3% of farmland that was privatized) provided over a quarter of gross agricultural output, including about 30 percent of meat and milk, 66 percent of potatoes, and 40 percent of fruits, vegetables, and eggs. This is covered by Michael Ellman's Soviet Agricultural Policy.
So, even with massive investment in terms of both capital and men Soviet Agriculture failed to provide the farmer with what he needed to achieve. We know they could do better, they did so on private plots, so the problem is with the economic or transportation infrastructure. Soviets at the time believed that it was with an inability to move the right machinery to the right places, but there are criticisms of the incentive structure to produce. That and the Soviets never had a good relationship with the peasantry. During the Russian Civil War there were peasant-Communist factions that fought against the Soviets, particularly in modern Ukraine. Even after that there were the adventures of Trofim Lysenko and his "Nah, bro, wheat totally evolves to grow in Tundra if you replant the same seed often enough" that endeared the Soviet leadership to exactly zero peasants.
Similar problems occurred in industry. The Soviet 5-year plans were pretty good in theory, but lacked accurate information as to the needs they were trying to fill. This led to some particularly noticeable misallocations of resources, which markets handle essentially automatically. So, in many cased good investment money was lost because it was trying to solve a problem that was already solved or simply weren't being devoted to a problem that needed solving because that need wasn't being communicated up the chain of command to those making the decisions. The distance between the users and the budgeters cost a great deal of efficiency.
That said, the Soviet Economy grew rapidly, expanding to $2 trillion by the 1970's from several hundred billion dollars in the 1920's. It's not that the Soviet Economy failed, it grew rapidly and consistently, even in per capita terms but what it didn't do was keep pace with US GDP growth. When the Soviets hit $2 trillion the Japanese hit $3 trillion and the United States hit $5 trillion. Basically, the tax base of the United States expanded so rapidly and was so much larger that by the 1980's a comparatively casual effort by the United States was a major budgetary decision on the part of the Soviet Union.
While there were a great many boondoggles in the West, and the lack of controls have a number of clear negative side effects after a certain point (somewhere around '65) the US government just had the budget to eat it and move on whereas the same error in the Soviet Union would require significant belt tightening and ending other projects to balance the budget. Not that the US ever bothered to balance its budget and as long as US Treasuries are the classroom example of "riskless investment" they don't have any need to. Soviets never had equal access to capital markets for obvious reasons.
To further complicate matters there is also the phenomenon described by Gregory Grossman as "The Second Economy". Informal and "Black" markets are a universal phenomenon, everyone has them. It's usually something like 10% of the stated economy over again in people being paid "under the table", legal goods being traded in such a way to avoid taxes or regulation, or illegal goods being traded. In the Soviet Union there was a uniquely powerful interrelationship between the official command economy and the second economy. Shortages in consumer goods, failure to meet production targets, and cost overruns were often blamed on this secondary market without deeper investigations as to the root causes, which meant that true shortages, inefficiency, and corruption often went unaddressed because it was easier to arrest a couple of black marketers and declare it solved. It was also ideologically attractive and politically advantageous to blame "capitalists" rather than go into detail about how party members have failed others. As a result an extensive network of Blat (a network of official and "unofficial" favors like a "good ol' boy networks" in America or Guanxi in China) straddling both Official and Black Markets developed. This turns out to be a fairly significant drag on the economy and often directly undermines the value provided by having a command economy in the first place by redirecting resources away from the supposed socially optimum amounts. Such networks exist everywhere, but in the Soviet Union they had a much more pronounced and pernicious effect than the United States in terms of policy decision making and resource allocation.
Finally, despite being the #2 economy in the world (well #3 towards the end as Japan went all crazy) they didn't engage in the same amount of trade as United States and other western economies. There wasn't an emphasis on design in Eastern Bloc countries, so the products being designed didn't fare well in international markets. The emphasis on being self reliant and ready for an evitable showdown with the US meant that they couldn't outsource work to cheaper places to drive down prices. Once an Eastern Bloc nation like Poland started trading and borrowing money it often took the form of a drain on the economy as locally produced good didn't bring in a lot of cash, but there was a lot of demand for foreign goods that just weren't being produced locally. A number of Warsaw Pact nation's ends were hastened by massive, crippling foreign debt and trade deficit. The Soviet Union proper managed to avoid this fate by discovering a massive amount of oil and diamonds. Rather than this being a massive boon like it was to Saudi Arabia or Venezuela it ended up just being a wash. Ultimately, the second largest economy in the world had little presence in international markets outside of key commodities.
In short, it wasn't so much that the Soviets were losing, they were clearly "winning". It was that they weren't beating the spread. And every year it got harder and harder to cover as western economies functionally exploded. So, while the Soviets had a number of key advantages, the combination of ideologically driven decision making and lack of information infrastructure cost them just enough to fail to keep pace with the rapid development of wealth and resources going on in many Western Nations.
The comment below by A_Soporific is very detailed about the Soviet economy in general, but one thing that can't be ignored is that Ronald Reagan convinced Saudi Arabia, and thus OPEC, to slash the price of oil. There is some debate about the extent to which the Reagan administration influenced the decision by OPEC to cut oil prices, but the result was disastrous for the Soviet economy.
http://www.frumforum.com/the-unravelling-of-reagans-mideast-policy/
Between 1985 and 1986, Saudi Arabia increased oil production from two million barrels a day to five million barrels. The oil price tumbled as oil supply surged: from US$30 a barrel to US$20 in just a few months.
The effect on the Soviet economy was devastating. Oil was the Soviet Union’s main – practically only – exportable product, the most important source of hard currency for the economically stagnant regime.
The soviet economy, and the arms race in particular, was financed with oil, and the loss of that revenue meant the inability to keep up with the arms race.
