I mean, when the written word was invented, hierarchical structures and complex economic systems were already relatively widespread in the places writing appeared. If we only looked at society how it looked when writing was developed, then we'd probably get a very skewed look at "human nature", right?
They don't 'limit themselves' to written history, it's just that evidence for events is often limited to written history.
There's are thousands of things that historians take into account when trying to piece events together. For example, it wasn't until the Cuerdale Hoard was found that we even knew of a Viking King in Jorvik named Cnut because he hadn't been named in any existing primary sources at the time. The Hoard gave us this realisation because of a single coin minted in his name. It's thought he may have been a co-ruler because these things were usually noted by Annalists.
Additionally, a lot of what we know about Anglo/Hiberno-Norse religion is based on interpretations of things like burial sites and monuments like the great Gosforth Cross or Hogback Stone.
With regards to oral history, historians typically don't give too much value to it because of how unreliable it can be. Stories often change over time and it can be a bit of a game of Chinese Whispers. That being said, Oral histories are being taken more and more seriously as time goes on, particularly within Native American communities.
Even written histories are never taken at face value though. If we were taking these at face value, then we would be led to believe that Judgement Day occurred at Lindisfarne in 793 as they suggested that there was lightning with dragons in the sky. This is to say that all sources, written or otherwise, are always highly scrutinised and of course, wherever possible, corroborated by other primary sources at the time.