Reigning for almost 300 years, they obviously did something right, but what were their wrongs/what happened that made their dynasty end?
Well there are a few ways to approach this question and different answers depending on how we quantify success (expansion/contraction of borders, economic growth, internal stability and foreign relations are all possible metrics which can be applied). In history sweeping changes like the collapse of empires are rarely due to a single or even a few major choices but are the result of trends and factors sometimes centuries apart. There is a nearly infinite amount of possible reasons why the Ptolemaic dynasty was doomed to fail and at the same time there is an infinite amount of reasons why they might not have, exactly why things happen is something historians struggle with and can not always divine with any certainty so for now we are going to satisfy ourselves with what, when and to the best of my ability how.
First of all I think that looking at the issue from the perspective that the rulers of the dynasty were "doing it right" until the formula was changed and bad decisions were made does not do it proper justice. The Ptolemaic dynasty was something entirely new for Egypt and the Hellenistic kingdoms in general were something that the Mediterranean had not yet seen although they built on older systems of governance. It is this continuous administrative, social and cultural development that makes pointing out just where the Ptolemids "went wrong" difficult. Tempting though it may be to imagine the Ptolemaic state as a direct extension of the king's will, royal power was far from absolute and many of the changes that the kingdom underwent should not always be viewed as products of a state agenda but as reactionary developments that progressed in response to environmental, political and internal stimuli such as famines, unrest and changing political climate domestically and throughout the Mediterranean. Whatever theoretical powers lay with the crown, the ruling party would still have to act in accordance with the practical necessities brought on factors by outside of their control. But of course, some of them were more effective than others, sometimes by a lot, and the impact of indibidual rulers and their policies is also important to consider.
With this in mind, I generally take the stance that at no point did the early Ptolemies set anything approaching a comprehensive plan for the administration of their fledgling empire but through a series of reforms built on and adapted older methods of governance to suit the immediate needs of the state.
Arguably it is true that the reign of the first three Lagides appears to have been the most, broadly speaking, successful time for the dynasty. The wars of the Diadochi and reign of Ptolemy I saw virtually all significant growth for the Ptolemaic Kingdom with the acquisition of Coele-Syria, Cyrene and Cyprus and by the end of the 4th Century BC significant expansion has effectively stalled. However, the Ptolemaic Kingdom was still able to exert considerable influence in Asia Minor afterwards thanks to the highly profitable trade from Alexandria which was supported by an admirably efficient and fairly secure internal administration which saw surplus revenues and resources allocated to the crown and administrative elites. The conditions for this success can be attributed as much to the inherent geographic and economic advantages of Egypt over the other territories that were partitioned as to the personal cunning manoeuvrings of Ptolemy. Most significant urban growth and development in and around Alexandria had already peaked and then plateaued by the mid 2nd Century but this is a not unusual pattern of growth for comparable premodern cities, given Alexandria's modest size compared to Rome which was albeit more densely populated, this apparent cap on population size does not necessarily point to a decentralization, just that immigration was no longer in full swing.
During the reign of Ptolemy II the dynastic cult was established and further conquests in Upper Egypt up to the Dodekaschoinos in Lower Nubia were annexed further securing their position within Egypt. The establishment of the dynastic cult allowed him to gather united support from his Egyptian and Graeci-Macedonian subjects, as well as cultivating alliances within the royal court. This cult and its associated propaganda would become one of the pillars of Ptolemaic rule but although the practice of sibling marriage was intended to limit potential claimants to the throne it did not quite work out that way, instead co-rulers (and their children) ended up competing with each other, often at the instigation of royal favourites and advisors who wanted to bring about regime changes.
Ptolemy III was similarly successful in his wars with the Seleucids although many of his alleged conquests including his conquest of Ethiopia, India and Babylon are based in propaganda more than any reality.
All of this success on expanding borders and gaining control over increasing territories in the Mediterranean looks great on paper but the Ptolemaic kingdom's core was always Egypt and while the successes in Asia look promising on their own, they came at a steep cost which would only increase as the need arose. Exploiting the agricultural workforce in Egypt enabled them to pour a seemingly inexhaustible stream of resources into Alexandria and flex their muscle in the Mediterranean but this of course strained the tentative balance that had been struck between the native Egyptians and their Aegean conquerors to the breaking point.