Learning Russian history that comes up again and again, but being in Siberia doesn't seem like much of punishment, seems to me like being exiled to Utah, it's not New York but one can manage
Without even getting into why Siberia was a common location for exile, I don't think you've appreciated what Siberia was like, particularly during the periods when exile was a common punishment.
At a basic level Siberia is remote, particularly before some significant rail projects. In the tsarist era when deportation/exile to Siberia was first introduced those being punished were expected to get to Siberia on foot if they had no other means of transport. The locals didn't make the trip any easier. The House of the Dead: Siberian Exile under the Tsars by Daniel Beer goes into this - here is a podcast the author did
Even in the Stalin era deportees could expect long and difficult journeys to get to where they were going. While Vorkuta isn't in Siberia, Alan Barenberg's Gulag Town, Company Town reinforces just how far away the camp was from anything like civilisation, how long and hard it was to get there etc.
The other thing to keep in mind is the weather in Siberia can be incredibly harsh. The average temperature in Siberia is about 0.5 celcius - it gets a bit warmer in the southern parts but those were not necessarily where people would be exiled to. Winter is naturally considerably colder.
So while exile might not have been as bad as being sent to prison (or a prison camp in Siberia for that matter...) it was very isolated, there weren't a heap of resources and when it got cold it was very cold. Examples of this can be found in biographies of famous exiles such as Stalin (Stephen Kotkin's bio being fairly recent) or Lenin (Robert Service's bio for example).