(Disclaimer: most of the following is from an undergraduate essay I wrote in 2013 on the question "Does the 'Abbasid revolution constitute a Sasanian revival?" which tackles most of the same issues. I apologize for its flaws. I've noticed that I haven't really addressed Crone's God's Caliph or Robinson's 'Abd al-Malik which are also very relevant books for your question.)

I think it's important to distinguish between the Sufyanid Umayyads ('Uthman, Mu'awiya I, Yazid I, and Mu'awiya II) and the Marwanid Umayyads (Marwan I, 'Abd al-Malik, etc.). The reason for this is the large-scale administrative reform instituted by 'Abd al-Malik b. Marwan.

Irit Bligh-Abramski writes that “the rise of the Abbasid dynasty in 133/750 did not cause a sudden, total transformation in the order of society which had been created by the Umayyads.” The argument that she makes hinges on two characteristics of the transition from the Umayyad dynasty to the 'Abbasid dynasty. Firstly, it relies on the idea that some of the changes which are considered characteristic of the 'Abbasid dynasty’s rule were initiated by late Umayyad caliphs in general and Marwan II in particular. Second is the notion that the nature of the organisation of the bureaucracy encouraged a continuity that bridged the 'Abbasid revolution.

Bligh-Abramski argues that “the later Umayyads had begun gradual fundamental alterations of governmental and military practices which directly influenced the shaping of the Abbasid regime.” The notion that the move of the imperial capital from Damascus to Baghdad is symbolic for the Abbasid refocus towards the east, is false on a basic level. Damascus lost its status as the capital of the Umayyad caliphate under the last Umayyad caliph, Marwan II, to the city of Harran, in what is now southern Turkey, and earlier the Zubayrids had hoped to rule the caliphate from Medina. The fact that the capital was then moved further to the east under the second Abbasid caliph is not nearly as consequential when considering that the geographical location of the caliphate's capital had been relatively flexible throughout the Umayyad period. Another significant change that was already underway during the reigns of the late Umayyad caliphs, was the increased deployment of mawālī in military posts.

Both developments must be seen in the light of the struggle between the Yamani and Qaysi factions. The Umayyad caliphs had traditionally supported the Yamani faction, whose powerbase was centred on Damascus, and on whom the Umayyad caliphs relied for their military. The Qaysi faction, on the other hand, was more prominent in the area around Harran, as well as Iraq, and thus Marwan II’s move eastwards can convincingly be interpreted as a favouring of the Qaysi faction over the Yamani faction, possibly in an effort to weaken the Yamani faction more than to strengthen the Qaysi faction. Bligh-Abramski has the following to say about this: “the traditional junūd (administrative-military units) gradually lost their power, first to the Qaysi-Jazirian army [of Marwan II] and later to the easterners, Khurasanians and Turks.”

Although some have been mentioned already, it is worth highlighting the crucial nature of events during Marwan II’s reign for the transition from Umayyad to Abbasid caliphate. Bligh-Abramski writes that “Marwan II marks the transitional period from Umayyad to Abbasid rule in a most illustrative fashion.” In addition to the events discussed above, “he was the first Umayyad caliph to provide his son and heir apparent with administrative and governmental training by appointing him governor of the Jazira and commander of the army,” something which then formed a precedent for Abbasids deliberately appointing their heirs apparent as governors of either Khurasan or Syria. On the administrative level, “Marwan II’s talented secretary Abd al-Hamid b. Yahya al-Katib (127/744-132/749) demonstrated administrative abilities which to a great extent foreshadowed the qualities of the Viziers of the later Abbasid period. His post is even referred to as wizāra (vizierate) in the sources.”

It would appear, then, that instead of being a radical break at the time of the Abbasid revolution, the administration of the Islamic empire gradually developed during the reigns of the late Umayyad and early Abbasid caliphs. Bligh-Abramski writes that “continuity of basic Umayyad governmental principles was simplified thanks to the professional personnel who dutifully served both masters.” and that “the bureaucracy was a stronghold of continuity despite the political overthrow.” Amikam Elid echoes this when he states that “one conclusive piece of evidence for the continuity between the Umayyad and Abbasid dynasties is that many members of Umayyad family went on to serve the Abbasid family.”

There are a number of things to be said about this supposed continuity in the administration during the transition from the Umayyad dynasty to the Abbasid dynasty. First of all, as Bligh-Abramski writes, “many of the leading bureaucratic families had been in government service for generations, and they tended to train their children to follow in their footsteps, which constituted a striking element of the continuity of bureaucratic personnel from the Umayyad to the Abbasid regime.” These families, which had survived the initial Arab conquests in a similar manner, were often non-Muslims. Jonathan Berkey writes that “in Iraq, for example, the religious and legal institutions, which both Christian and Jewish communities had developed through their interaction with the Sasanian state and which presumed a degree of communal autonomy, survived the conquest.”

Secondly, the early 'Abbasids were not shy of integrating the aristocracy associated with the Umayyads into the Abbasid courts. Bligh-Abramski poses that by “including the Umayyad nobility in the caliphal courts, the first Abbasid caliphs demonstrated tolerance and political shrewdness.” This facilitated a smooth transition from the Umayyad to the Abbasid dynasty, while the Abbasids consolidated their hold on the caliphate. The notion that this may initially have been done for reasons of stability is supported by the fact that “al-Mahdi’s policy of choosing his officials without regard to either their genealogy or previous service changed significantly under Harun al-Rashid.” Nevertheless, “even in the later Abbasid period, the families of Umayyad clerks continued to transfer their skills from generation to generation.”

Indeed, the amount of evidence for evolution as opposed to revolution in the transition from the Umayyad caliphate to the Abbasid caliphate suggests that perhaps we are wrong in approaching early Islamic history through a categorisation in dynasties. Michael Morony argues that “the standard division of early Islamic history into Rashidun, Umawi and Abbasi periods is essentially a dynastic approach to history which is based on assumptions about the causative impact of the personality and policies of rulers on those features which are considered to be typical of each period.” According to him, this view of history is flawed because “everything from administrative appointments to economic prosperity or disaster tends to be explained in terms of the personality of the ruler and everything else is explained in terms of reactions to a ruler’s policies, approval or disapproval, support or opposition, i.e. history is reduced to the personality of the ruler.” He continues, “Instead of making the ruler responsible for social and economic conditions it is possible to argue that rulers were themselves the products of a changing social background and were affected by changing economic circumstances.”

The dynastic approach to early Islamic history is not a recent one, and has been the default way of approaching early Islamic history since 'Abbasid historiography. In this reading of history, the division of the Islamic caliphate into the Rashidun, Umayyad and 'Abbasid caliphates carries with it a moral judgment. The Rashidun caliphates, frequently referred to as the Rightly Guided Caliphs, represent the Islamic Caliphate at a time when it was supposedly still close to the ideal of the Prophet, and are defined as such against the later Umayyads, whose reputation is one of a secular and materialist empire. The 'Abbasid dynasty then, symbolises a return to the ideals of the Prophet and the Rightly Guided Caliphs. Needless to say, this interpretation is a largely the result of projection by Abbasid apologists writing in the aftermath of the Abbasid revolution and at the height of Abbasid influence. Morony writes that “[periodisation] rests on the double assumption that a degree of coherence exists during a particular time-span and that features which are typical of that time-span are different from those that preceded it and from those that followed it.” As we have seen, the strong continuity between the late Umayyad caliphate and the early Abbasid caliphate, as well as the relative discontinuity within the Umayyad and Abbasid dynasties, suggest that the dynastic approach to periodisation is at best inadequate, and at worst incorrect when describing this part of early Islamic history. [1/2]