I was hoping - and still hope - that someone with more expertise on military matters would / will provide a really comprehensive answer to this question. But in the meantime, I can provide, faute de mieux, a brief summary of Macedonia's place in Hellenistic power politics.
Macedonia was much smaller and poorer than many parts of Alexander's vast empire. It had, however, several qualities that made it very attractive. First and foremost, as the ancestral kingdom of Philip and Alexander, Macedonia had real symbolic value to any Hellenistic dynast who could hold it. To be a Hellenistic king, after all, was to live in the shadow of Alexander. Ptolemy I had hijacked Alexander's body; but holding Alexander's throne was itself a powerful claim of legitimacy.
More generally, Macedonia had valuable resources of its own: extensive timber reserves, valuable gold and silver mines, and - most important of all - manpower. Though not nearly as populous as, say, Egypt or Mesopotamia, it supplied a steady stream of reliable recruits for the phalanx, and had superb native cavalry. Since Hellenistic kings tended to depend much more on imported Macedonian and Greek soldiers than on natives of the conquered territories, this was an extremely useful asset. Once the Antigonids finally established a secure dynasty in Macedonia, these resources allowed them to become serious players in Hellenistic power politics, capable of dealing on an equal footing with the larger but less well-integrated kingdoms of their rivals.
Macedonia was also an ideal base for controlling / influencing the wealthy and prestigious cities of Greece and western Asia Minor. In the half-century after Alexander's death, virtually every major player in the Hellenistic politics made a bid to gain influence over the Greek cities, whether by declaring them free (Antigonus I did this, and was soon imitated by the kings), by patronizing a famous shrine (as Seleucus did the Temple of Apollo at Didyma), or by forming special relationships with individual cities (as several kings did with Athens). A king based in Macedon was much better-placed to become the arbiter of Greece (one thinks of the famous "fetters" of Greece - three Macedonian strongholds at strategic points in the Greek peninsula) than a king in Alexandria or Seleucia.
The value of Macedonia to individual kings obviously varied with personal goals and Macedonia's place in the larger political landscape. Ptolemy I had little use for Macedonia once he had his Egyptian kingdom. But Antigonius I (who aspired to reunite Alexander's entire empire) and Seleucus I (who died trying to add Macedon to his sprawling domains) were obviously very motivated to assume the mantle of Philip and Alexander, as were all the adventurers (most famously, Pyrrhus) who risked everything in pursuit of the Macedonian throne.
This question seems like a straightforward binary, but it is actually as complicated as it is important for the early Hellenistic period.
However, before getting to this issue, it is important to recognize that, in 323, nobody was seriously thinking about the empire as a divided entity. Instead, the core debates in Babylon were about which of the potential heirs would take the throne of a united kingdom and, by extension, which faction would rule through them. In fact, in as much as the successors all aspired to be Alexander’s heir, they maintained a fiction that this was a single kingdom until years after the last Argead kings had died.
(Antigonus only officially used the title from 306, and the others followed suit.)
At the same time as all of this was happening, Alexander’s empire was in fact being torn apart, which gets more to the point of your question. Here the key is distinguishing a couple of points: first, that the Macedonians were not all of one mind and, second, that there is a difference between ideological significance and a practical significance.
To the first point, each of the Diadochoi (successors) had their own agendas and ideas about the relative importance of Macedonia and those ideas were further constrained by the appointments they received at Babylon (323) and at Triparadeisus (319) where the provinces were divvied out. It is a fairly standard assumption to say, as Worthington does in his recent biography, that Ptolemy put a priority on Egypt because of its wealth and defensible frontier and he never made a real attempt to take Macedonia even during his campaigns in the Aegean––after all, he already had Alexander’s body, wasn’t that better? On the other end of the spectrum, Antipater had been governor of Macedonia while Alexander was away and received it back in both conferences. Antigonus Monophthalmos, who had been governor of Cappadocia in Asia Minor, received Asia and set about securing all of the kingdom after Antipater died, but never returned to Macedonia, probably because he had a bigger piece of the pie in Asia.
Perhaps the most interesting example here is Seleucus who a) was one of the only high-ranking Macedonians to not put aside his Asian wife (Apame, the daughter the Sogdian Spitamenes, or according to Strabo 12.8.15 Artabazus), suggesting perhaps that he meant to use this relationship from an early date, but then became satrap in Mesopotamia, which was wealthy, stable...and utterly lacking in military forces for him to use. He established a kingdom anyway, despite periods where he was literally on the run without any territory. This kingdom was centered on Antioch on the Orontes and Seleucia on the Tigris, but ran from Miletus in Asia Minor to Bactria (modern Afghanistan). And yet, Seleucus was also assassinated after crossing over into Europe where he intended to seize Lysimachus’ kingdom after the battle of Corupedium (Appian, Syrian Wars 62). Did Seleucus see a political opportunity through his wife’s family? Probably, but his second wife was Stratonice, the daughter of Demetrius, meaning that she was the granddaughter of BOTH Antigonus and Antipater. Appian says that before he headed off on his final campaign, Seleucus gave Stratonice to his son Antiochus, who was then wasting away from love for her (Syrian Wars 59–60).
Putting these differing approaches together leads to the second issue. Macedonia did matter––probably to some because of nostalgia, the ache of wanting to go home, but to others because of its ideological significance. This was the kingdom of Alexander and the home of the people that had conquered Persia!
As u/toldinstone pointed out, every Hellenistic monarch with claims to Greekness, even if they weren't Macedonian (*cough*Pergamum*cough*) aimed to garner influence in the Greek cities around the Aegean, but they also competed in conspicuously declaring the freedom of the Greek cities (see, for instance, Antigonus' declaration at Tyre in 315 where he also declared Antipater's son Cassander a public threat in Diodorus Siculus 19.61). Add in that the early Hellenistic kingdoms amounted to a Greco-Macedonian minority ruling over indigenous populations, and there was a mistrust of the indigenous soldiers.
A good example: Polybius condemns the Ptolemies for empowering the Egyptians at the battle of Raphia in 217 by having them serve in the phalanx, saying that it led to revolt (5.107.1) Greece and Macedonia were thus the preferred sources for military manpower and even sources of colonists for the new Greek-style poleis that they were founding in their kingdoms.
However, these new kings all demonstrate greater interest in seizing and controlling land than they do in pining over Macedonia. Seleucus returns just forty(!) short years after Alexander had died. Of course, while all of this was happening, Pyrrhus of Epirus was challenging Macedonia for prominence in the region and Macedonia was about to be beset by the Gallic invasions of the late-280s and early-270s. Nevertheless, as u/toldinstone mentions, the Antigonid dynasty remained a legit power-player in the Hellenistic World until the early second-century BCE and even then Rome consciously divided it into four independent entities that were no longer allowed to come to together because of its potential to challenge Roman authority.
Tl;dr Macedonia was important and wealthy enough, but the contours of the political landscape after Alexander dictated whether, when, and how the Diadochoi aimed to take it.
Followup question. What are the best ancient sources for this period?