It seems that from about the 1500s and onward, nobles and royals stopped building castles for both residential and defensive purposes and preferred to build palaces with little to no defensive utility for their residences while purely military installations such as star forts replaced the castles of the middle ages. When did this take place? When was the earliest residence only palace (ie. no arrow slits, bastions, turrets or any defensive structures) constructed in medieval Europe or Early Modern Europe?
The Italian War of 1494-1495 showed Europe the true power of modern artillery against the standard castle. The curtain walls made great targets, and collapsed under heavy fire, allowing infantry to enter. The origins of the bastion fort were found to be effective in Pisa in 1500 and Padua in 1509. The earthen/brick ramparts would not catastrophically fail under cannon fire. The bastion was further refined into star forts, which enhanced the angles for flanking fire for the defense.
A similar question from seven years ago covers much of this. https://www.reddit.com/r/AskHistorians/comments/3too5u/when_and_why_did_people_stop_building_castles/
Castles existed alongside normal unfortified residences throughout. In post-Roman times, manors were more economic than military, and usually lightly fortified if at all. The main form of defensive works were town walls. Then, as central order broke down and large parts of western Europe became zones for raiding - either by Saracens, Magyars or Vikings, or by local lords, castles became a thing. Most were private, but a lot - and most of the more impressive ones - were public works, intended as part of a network to limit raids or, later, a centres of royal or comital power. Then as first strong counts like the Baldwin dynasty in Flanders or the Plantagenets in Anjou exerted control, they tended to strict measures against unlicensed fortification - and the need diminished as raiding and feud became a thing of the past (discouraged by measures such as that taken by the Count of Flanders who had a peace-breaker boiled alive). Also, as power and access concentrated in princely courts, a noble found it more rewarding to invest in a town-house near the palace than in a stronghold. Castles are expensive to maintain, and decay quite rapidly (less the walls than the outworks and interior buildings).
Another factor was that as offensive techniques improved, castles became more expensive. Something like Caernarvon or Krak de Chevaliers was a major investment, far beyond the resources of a local baron. That said, simple 'peel towers' continued in use in areas like Scotland, as adequate against cattle raids, and marcher areas (Spain, the Balkans, the Baltic) continued the tradition.
Castles continued to be of military use well into the 17th century - many were held as strong points in the French and English civil wars. The walls had often decayed, but the ditch and site were still formidable. Even if cannon could bring down the curtain, the combination of a good ditch and a rubble mound still proved hard to crack. So at the conclusion of these wars a good many were 'slighted' - key sections demolished (often explosively, as at Corfe). Richelieu and Cromwell took a hard line on private fortifications.
So the trajectory is that castles came in around the 10th century, and were on the way out as private residences from the early 13th, but were still being built as public defences into the 14th and 15th, had a revival as strongholds in the unrest of the 16th and early 17th and then lapsed. In the 11th century there were thousands of private castles - most simple affairs; by the 13th there were hundreds of royal castles, by the 17th, dozens of towns fortified with trace italienne.