A lot of things contributed to the decision to invade. First, there was the policy of regaining former Imperial Russian territories that had been lost to Soviet rule during the Revolution and Civil War. This contributed to their takeovers/invasions of the Baltic states (Estonia, Latvia, Lithuania) and eastern Poland. It wasn't the main motivation for invading Finland, judging by the end results - the treaties at the end of both the Winter War and the Continuation War left Finland independent. However, the fates of Estonia, Latvia and Lithuania (and the shifting of the Polish borders post-WW2) suggest that bringing Finland into the Soviet Union would have been acceptable to the Soviet leadership. The Soviets actively worked towards having Finland as well as the Baltic states recognised as being within the Soviet sphere of influence - first seeking such recognition from Britain and France, and then getting it from Germany courtesy of the Molotov-Ribbentrop Pact.
Second, the inter-war relationship between Finland and the Soviet Union had not been friendly. Apart from things starting off on the wrong foot with White victory in the Finnish Civil War and banning of the Communist Party in Finland, there was Finnish involvement in the Estonian War of Independence and in the revolt in Ingria, and in East Karelia. While the dreams of some Finns of a Greater Finland had to lie dormant until the Continuation War, the Soviet leadership didn't forget. Neither side was happy with the terms of the Treaty of Tartu (1920), each feeling that they hadn't achieved their goals due to their weakness.
As a result, there was significant mistrust between Finland and the Soviet Union as the pre-war negotiations began.
The Soviets were concerned about their security, and Finland featured in these concerns in multiple ways. First, Finland was seeking closer ties, including military cooperation with Sweden. To Soviet eyes, the best outcome of this would be a stronger and neutral Finland, and the worst would be a stronger and hostile Finland. Absent Finland being part of a Nordic bloc with Sweden (and perhaps others), Finland wasn't strong enough to be a threat, but a weak Finland was also problematic - a weak Finland would not be able to resist a German incursion, and the Finnish border could offer Germany a route to attack Leningrad. The Finnish government assured the Soviet government that they would stop any German attempt to take over Finland or to attack the Soviet Union through Finland; the Soviets didn't think that Finland was strong enough to do so. They were also frightened by the prospect of Finland allied with Germany. The Soviets wanted neither a strong Finland nor a weak Finland, unless they could base troops in a weak Finland. This is the background of the Soviet demands for a base in Hanko and control of some islands in the Gulf of Finland. For further security, and a further weakening of Finnish defences, the border should also be moved further from Leningrad.
To Soviet eyes, these were perfectly reasonable demands. Compensation for these concessions in the form of East Karelian territory, and Soviet strength, made Soviet victory in the pre-war negotiations appear inevitable. After all, the Baltic states had given in to Soviet demands to base troops in them, without compensation of the kind offered to Finland. But the Finns were unreasonably - from the Soviet viewpoint - stubborn.
The Soviets had three choices:
They could accept the failure of negotiations and not base troops in Finland and leave the borders where they were. This would leave them vulnerable to German takeover of Finland.
They could negotiate more reasonably, and take less and/or give more in return. It could be said that this would have been the most rational choice. Stalinist paranoia and faith in the power of bullying, and faith in quick victory in the event of war kept the Soviets from such rational choices. It's likely that it was felt that backing down from their bullying position might weaken future negotiations with other countries - better to appear strong and unyielding, and future bullying will be more likely to succeed.
War.
Pre-war plans for the invasion of Finland - such plans had existed for years - were based on reality. Recognition of the difficulty of the task and the possibility of strong Finnish resistance meant that the plans were much more conservative than the plans developed for the Winter War. This didn't please Stalin, who insisted on a plan for quick and decisive victory. A plan promising such was delivered, and Stalin approved. This put the Soviet leadership in the position of feeling that a resort to war to get what they wanted would be resolved by a quick and decisive victory - this was, after all, what the plans promised.
With all of this, there was a combination of past poor relations leading to mutual distrust, Soviet paranoia and fear of Germany, and a belief in quick victory if there was war. A negotiated outcome leading to Soviet bases in Hanko and the Gulf (and consequently stronger Soviet influence over Finland) was preferable to war for the Soviets, but war was preferable to backing down in the negotiations. Thus, war.
Further reading:
Vesa Nenye et al., Finland at War: The Winter War 1939-40, Osprey, 2015, has a good overview of the politics and negotiations leading up to the war (and not just the immediate pre-war negotiations).
I don't know of any recent works devoted to Finnish-Soviet relations leading up to the war. There are some older works that might be of interest:
Anderson, A. (1954), "Origins of the Winter War: A Study of Russo-Finnish Diplomacy", World Politics, 6(2), 169-189. doi:10.2307/2008988
Condon, Richard William, The Winter War: A study of Russo-Finnish diplomacy 1938-1940, MA thesis, University of Nebraska at Omaha, 1960
For the reasons that the Soviet government gave to its own people - its internal propaganda - see:
James Venceslav Anzulovic, Jr., The Russian Record of the Winter War, 1939-1940: An Analytical Study of Soviet Records of the War with Finland from 30 November 1939 to 12 March 1940, PhD thesis, University of Maryland, 1968