The biggest sticking point with Russia was probably Poland. The Duchy of Warsaw, carved from Prussian territory in 1807, represented a revival of a Polish state after a series of partitions by the Eastern Powers. The Poles were armed, and in 1809 they fought alongside France against Austria; they were initially set to receive all of Austrian Poland, and in the final terms of the treaty received far more of Galicia than the Russians did. The tsar feared that a restored Polish kingdom would ignite uprisings in his majority-polish Western territories and lead to French demands to renounce these territories, and Napoleon refused to assuage these fears with even a nominal guarantee against restoring the Polish kingdom. Napoleon ensured the Polish people were well armed; they made good blackmail against Russia. Further French encroachments -annexing German states to which the tsar had family ties, placing an erstwhile puppet on the Swedish throne, and hindering Russian conquests in Ottoman Europe after having promised a free hand at the Peace of Tilsit- pushed Russia to formally break from the French alliance.

This tied into the Continental System, by which any power in the reach of French armies was forbidden from trade with Britain. While typically framed as a measure intended to undermine Britain, this was in fact really a secondary aim. To raise money for the war in 1809, Napoleon secretly sold licenses to trade with Britain. By 1810, France had mostly give up on trying to defeat Britain through economic warfare, even importing British colonial goods through a system of licenses and tariffs formalized in the St. Cloud and Trianon decrees. The real goal was French domination of the European continent through economic colonialism; French subject states provided raw materials for French industries and markets for the industrial products (and imports from Britain).

As such, when Russia broke from the Continental System, it wasn't by officially opening its ports to British trade, but by closing off French trade with high tariffs. Russia had, however, allowed considerable amounts of British goods into its harbors on the grounds that they were carried by neutral shipping, in this case predominantly American. Even British ships with false papers docked in Russia, and their goods fanned out into the rest of Europe. This last point would seem inconsequential for the French war against Britain, the money having already changed hands for the British, and to a degree it was. What the Russian exports into the rest of Europe did was undermine France's militarily-enforced monopoly on trade in British goods. This undercut Napoleon's project of economic domination in Europe.

I emphasize this point because it's important to realize that it was not the physical security of Napoleon's empire that was under threat in 1812; Napoleon had several advisors who presented reasonable proposals towards that end which he rejected. Rather it was his ability to dictate foreign policy to the Great Powers that was being tested. All of Russia's acts of 'defiance' were the normal prerogatives of independent states, but Napoleon could not abide this.