I read a few things on the late Roman empire and have a few questions about the Eastern Roman Empire.
There is the "new" theory contrary to gibbons that the (west) Roman Empire did not decline, but did in fact "evolve".
I suppose the eastern Roman Empire underwent a similar/analogous transformation, especially after the loss of the rich provinces of Syria and Egypt. How did state and society change? Did it become medieval/ feudal?
There is the often called trinity of roman state, Christianity and Greek culture, but how "roman" was the imperial administration and society?
This is an extremely broad and general set of questions, so I'll deal with the subset with which I have the most experience.
It's a cop-out, but it depends on how you define "Roman." Since the 3rd century, the pan-Mediterranean Roman empire already had the experience of rule by low-born military emperors of provincial origin, men with few ties to the aristocracies of Rome and its subordinate municipalities and colonies (from which the archetypal "Roman" emperors of the past had come). The transformation of the empire from one that was still nominally republican and Rome-centered into one that was led by military men who rarely (if ever) saw Rome itself and who ruled from provincial centers had already been accomplished by the time of the "fall."
One of my instructors, Raymond Van Dam, invited us to think of the "decline and fall" of western empire as basically Roman expansion accelerated, with the takeover of political/military institutions by migrant, quasi-Latinized Germanic peoples mirroring the takeover of those same institutions under the early empire by conquered, Latinized provincials. This process of outward devolution and splintering of imperial authority wasn't necessarily traumatic or even particularly destructive; see James O'Donnell's Ruin of the Roman Empire for a more detailed account of this.
The eastern empire's more densely-settled, urban established population and its greater wealth allowed it to absorb similar peoples without undergoing the dissolution that its western counterpart faced. This process did not come without losses; see, for instance, the loss of the northern and western Balkans to Slavs and Avars. It became "medieval" in the sense that the scale, duration, and devastation of Byzantine wars in the east, especially against the caliphs, destroyed the great-city-centered culture of the classical-age Mediterranean everywhere except Constantinople. Unlike the Roman empire in preceding eras, the Byzantine empire was, with that major exception, a fundamentally rural one.
It wasn't "feudal" in the sense that the Byzantine state did not alienate its territories and the legal powers it had over them by parceling them out to aristocratic supporters. It did, over time, lose access to the taxable produce and manpower of its territories as dynatoi families simultaneously accrued land holdings and senior offices in government. This, however, was perceived as a problem by the Byzantine state; it wasn't a feature or a basis of its authority.
Other sources: Lectures by Paolo Squatriti, Rosenwein's A Short History of the Middle Ages, Vol. 1
Edit: paragraph spacing