Calling them stone "structures" is probably too generous. If you go to the spot now, you can still see stones at the top of one of the hills. As I recall, there are about a dozen of them visible, all about knee- to waist-high and mostly flattish on top. They did seem to be placed somewhat in a ring. They make nice benches.

The stones themselves only merited passing reference in Joseph Smith's own 1838 journal when he called them remains of a "Nephitish Alter an Tower"--referring to the peoples described in the Book of Mormon (who weren't around in Adam's time, but were roughly contemporary in time to the people of the Bible). He didn't mention Adam's altar in any of his own writings.

However others among Joseph Smith's contemporaries recalled a number of years later either seeing remains of the Nephite "tower" or of stones in a different spot that was identified as an altar built by the Biblical Adam that either resembled ruins of what used to be an old stone wall or something that looked like a three-tiered pulpit. It's unclear who made the initial identification of Adam's altar--the sources just don't indicate for certain if Joseph Smith himself identified it or if someone else gave it the name. This 1974 BYU Studies article (although kinda outdated) quotes just about every primary source that mentions the stones and offers something of a summary at the end of the article (see pp. 564-576).

There is speculation that the stones at Adam-ondi-Ahman are from the Mound Builders--probably because of the famous Cahokia Mounds near St. Louis and other mounds in Missouri--but as far as I know, there hasn't been any professional archaeological excavations in that spot.