In antiquity, diamonds were extremely rare and extremely valuable. In Pliny the Elder's Natural History, book 37, he refers to them in this way:

The substance that possesses the greatest value, not only among the precious stones, but of all human possessions, is adamas; a mineral which, for a long time, was known to kings only, and to very few of them.

(There's some disputation about what exactly "adamas" refers to - it seems to be a range of clear, crystalline stones, including but not limited to diamond.)

At that time and through the sixteenth century, Europeans and Near Easterners had only one source for them: India, where they were first largely found in deposits in the Penner, Karnool, Godvari, and Makhnadi rivers, and eventually mined. Thought to have been formed by a hardening liquid - how else could they be so pale and clear? - they were prized for their ability to glitter in the light and throw rainbows around the room. They were evaluated roughly the same way that they are now, in terms of color (water), clarity, and size, with an unblemished and uncolored stone being the most valuable. In the European middle ages, there is evidence of diamonds being used in betrothal and marriage rings, contrary to the story that DeBeers entirely invented the idea: why not use the most valuable and indestructible stone to show the strength of your future relationship? Making that symbolism almost explicit, a poem written on the marriage of Constanzo I Sforza and Camilla D’Aragona in 1475 stated, "Two wills, two hearts, two passions are bonded in marriage by a diamond". There are extant diamond rings with loving inscriptions, and the 1505 will of Marion Chamber of Bury specifically lists a gold ring set with a diamond and a ruby as a "maryeng ryng". (Other rings made in this fashion may therefore also be wedding rings, possibly with the diamond symbolizing the groom and the ruby the bride.) By the seventeenth century, there are even fictional references to diamond rings for marriage in Cervantes and Voltaire.

Diamonds were found in eastern Brazil in 1725, and a relatively short frenzy began as people determined to strike it rich began digging everywhere to get more. Many didn't believe it, because it had been "known" for so long that Diamonds Are Found In India - but within a few years, Brazilian mines were shipping a dozen times as much diamond material back to Europe than the Indian mines were producing at that time. The Indian mines were beginning to slow down a lot at this time (although some continued to be operated, and there are some large diamond reserves there still), so finding new sources wasn't just, "woo hoo, I've got some diamonds!" but, "woo hoo, I can control the diamonds!" - this gave an even greater urgency to the search for the Next Big Find. By the beginning of the next century, Brazil was the leading source, and another boom occurred in the northern part of the country around the 1850s.

The next boom occurred in South Africa in 1870-1871, along the Orange River; after an initial mistaken pooh-poohing of the area's potential for exploitation by a British investigator, it was proven to yield a lot of diamonds. If you are a Frances Hodgson Burnett fan, which is statistically unlikely for this sub's userbase, this is where and roughly when Sara Crewe's father's friend struck it very big by discovering a new, deep vein. In real life, this is where the DeBeers syndicate comes in. A significant amount of diamonds were mined in Australia in the late nineteenth and early twentieth centuries. And after that, Russia - a few stones were found on their own in the nineteenth century, and then larger deposits in rivers, indicating the presence of good veins somewhere, leading up to the Revolution, which sort of put a spanner in the works. It wasn't until after World War II that effective mining of these began, but in the end this region became the second largest source in the world, so well done, lads.

But we're talking about price and popularity. As early as the first Brazilian diamond boom, there was a recognition that flooding the market was leading to reduced prices. The Portuguese made strong efforts to restrict the flow to other countries in order to keep the floor from dropping out, and it's highly possible that diamonds were being panned for and mined earlier than 1725, and sold in Goa to enter the market as "Indian diamonds". And because of these booms, the availability of diamonds did go way up, even if the prices didn't drop so far as to seriously devalue the stones: really large and elaborate diamond jewelry began to be found in royal courts, ballrooms, and opera houses across Europe (eighteenth century queens and their ladies in waiting might have entire stomachers set with diamonds). Diamonds were still highly valued for their own properties, but they were not seen as so excessively more special than rubies, sapphires, and emeralds. Rubies in particular were known to be much rarer and cost as much as diamonds at the time, and late nineteenth century articles on popular jewelry trends in trade magazines, like this one in The Jewelers' Circular and Horological Review, 1884, neither put diamonds on a pedestal alone nor consider them cheap and boring.

As far as I can tell, the Diamond Syndicate did not really start heavily engineering the market until Sir Ernest Oppenheimer became the chairman of De Beers in 1929. Diamond mining had always been extremely exploitative of its workers - the Brazilian and African mines ran on slave labor, or the next thing to it, and we shouldn't be surprised to learn that De Beers was started by Cecil Rhodes - but under Oppenheimer, De Beers and its syndicate began to exploit the market as well, releasing only a trickle of stones to dealers per year, in order to keep the value high. And because they created a stockpile by doing this, they didn't face any problems due to natural shortages or labor strikes, and could continue to do business at exactly the same volume.

In 1947, the ad copywriter Mary Frances Gerety came up with the line "a diamond is forever" for De Beers, now a classic. This has been made out to be part of the invention of the diamond engagement ring, but as mentioned before, the concept is much older. Well, both the concept of a diamond engagement ring, and the symbolism of diamond = forever = our love. However. Nineteenth century sources do not show engagement rings as having to be set with diamonds the way they are now. Ward and Lock's Home Book (1882) says that they can be plain gold, engraved with a word, or set with stones that form a word in acrostic; a story in Peterson's Magazine in 1875 uses a pearl engagement ring. But I would note that the same article in The Jewelers' Circular and Horological Review linked above notes that the diamond solitaire is the most popular engagement ring at that time, and a later article in the same volume says that while turquoises and pearls are generally popular, diamonds are "the rage" with those who could afford them. The shift to everyone expecting diamonds has as much to do with post-war prosperity and upward aspirations as it does De Beers itself.

Sources:

Diamonds: A Early History of the King of Gems, by Jack Ogden (Yale University Press, 2018)

Diamond Deposits: Origin, Exploration, and History of Discovery, by Edward I. Erlich and W. Dan Hausel (Society for Mining, Metallurgy, and Exploration, 2002)

The Last Empire: De Beers, Diamonds, and the World, by Stefan Kanfer (Macmillan, 1995)