When you hear "pot" associated with leprechauns it doesn't mean one of the big iron cauldrons you often see in modern depictions. Earlier English language tradition seems to have rendered the original Irish word (which I think might be próca oir but there's a bit of a chicken and egg situation going on with that) pretty invariably as "treasure-crock". So more like a small clay vessel than a pot as such. Rather than having anything to do with rainbows, Leprechauns were said to either have buried these crocks themselves, the gold being theirs through their shoemaking, or else to have found stashes of loot left by soldiers after battles. I can't say for sure as to why they used crocks rather than boxes or chests or sacks. I can guess that a crock might've been easier for a short person to carry, or it would've been less conspicuous than the other options. It also might be that the average Irish home was more likely to have some crocks laying around than boxes or chests.

The earliest reference I can find to the specific phrasing "pot of gold" is in D. R. McAnally's Irish Wonders, published in 1888 (at least 200 years after the leprechaun is first referenced in English language literature), where he records a story told to him by an old woman from Crusheen. I don't know how popular the book was exactly but according to Mary Ketsin several editions of it were published. It might be that the specific phrasing "pot of gold" was popularised by this story.

Sources:

D. R. McAnally, Irish Wonders

Mary Ketsin, Irish Literature: Background and Guide to Books

William Allingham, The Leprechaun

William Butler Yeats, Fairy and Folk Tales of the Irish Peasantry