Robert Curthose’s inheritance of the Duchy of Normandy instead of the Kingdom of England was due in part to fears William the Conqueror held about his firstborn’s predisposition to rule, but appears to be overwhelmingly dictated by their personal turmoil.

After the Conqueror’s death, a fragile truce made between Robert and William Rufus broke down when landholders on both sides of the English Channel found their loyalties at odds given the heightening rivalry between the brothers. Many of the nobles that had once served William the Conquerer placed their support behind Robert given he was considered more easily manipulated than the steadfast William Rufus. Indeed, 6 of the 10 most powerful baronial landholders in England (as listed in the Doomsday Book) had thrown their support behind Curthose.

However as you’ve hinted at in OP the real rift between William the Conqueror and Robert Curthose was personal. While there aren’t a huge amount of primary sources to thoroughly explore their personal relationship, it appears clear that William Rufus was the Conqueror’s favourite son. A moment to accentuate this can be noted in a prank played by William Rufus and his younger brother Henry, who emptied a full chamber pot onto Robert’s head in boredom. While only a prank, Robert became so enraged that after brawling with his brothers he attempted to siege the keep in Rouen in Normandy in the Rebellion of 1088. He later unhorsed and humiliated William the Conqueror in an armed confrontation over Robert’s wanton actions on the continent, leaving the King bitter and alienated from his son. Moreover, William the Conqueror’s obvious favouritism towards his younger sons generated much discontent in Robert, who believed that power had not been distributed fairly amongst the brothers, which eventually led to outright rebellion and obviously pushed the Norman King to further distance himself and his personal legacy from his eldest son.