Well it depends on a lot of factors, not least of which, what kind of castle are we talking here? A simple ringwork or motte and bailey built upon an already present and usable hill, could be functional in weeks. Possibly even only a few days if the builders are particularly organized and the design is particularly small. The further you get away from that, the longer it will take. If the motte and bailey castle has a larger tower, or requires the hill to be made by hand first then the construction can be dragged out for months to a year. And these were probably the most common types of medieval castle.
But most people when they say castle mean the large stone structures. And that could take awhile, largely based on funding. For example, Chateau Gaillard was King Richard of England's pet project and he poured a lot of time and money into its construction. He helped with the planning, design, and he was on location for much of the construction period. And as such Gaillard was built in about two years, which was considered an unusually fast build time for a castle of its size and sophistication.
In most cases, the patron of the construction would not take such a hand's on approach and construction would drag on as funds ran out, or were diverted for some other important venture. In that case construction could drag on for much longer. Castle Dover took 12 years and cost roughly half the cost of Chateau Gaillard. While Crac des Chevaliers took decades, but not all at once. Different sections of the castle were constructed with sometimes years taking place where nothing was being built at all.