This is a BIG topic. Very briefly: yes, this was a problem, and yes, lots of people (some very powerful) were concerned about it. I'll try to provide some basics based on how I present the information to my students but I hope others (specialists on Iconoclasm) will chime in as well.

First, let's look at the Second Commandment: "Thou shalt not make unto thee any graven image, or any likeness of any thing that is in heaven above, or that is in the earth beneath, or that is in the water under the earth: Thou shalt not bow down thyself to them, nor serve them: for I the LORD thy God am a jealous God, visiting the iniquity of the fathers upon the children unto the third and fourth generation of them that hate me." (Exodus 20: 4-5)

Pretty clear cut, right? Yet Christianity is defined by its images almost as much as it is by its texts. (Even Judaism, which is pretty consistently aniconic, has exceptions; there are images in floor mosaics in Palestine, Jewish catacombs in Rome, and synagogue decoration at Dura Europos, all of which question the assumption of consistent aniconic Jewish religious religious culture. But let's stick with Christianity...)

Some of Christianity's icon-worship is probably linked to Pagan traditions; e.g., house icons in the Late Antique period (Suchos and Isis, c. 200 Tempera on Board. 24x23” Formerly in Berlin, destroyed in WWII). Some of the earliest surviving Christian icons (like Christ and St Menas: https://en.wikipedia.org/wiki/File:Menas.jpg) have a similar aesthetic. There are also arguments for visual links between, e.g., how Mary is portrayed and how Isis was portrayed. That said, there is a significant gap (a couple hundred years) in the record. If you want more on this I believe the best place to look (on the home as a source of icon use) is: Thomas F. Mathews and Norman Muller, “Isis and Mary in early icons,” in Images of the Mother of God: Perceptions of the Theotokos in Byzantium, ed. Maria Vassilaki (Aldershot, 2005), 3-11.

One way some early images were justified was by removing them from the realm of human production. Thus some of the earliest (surviving) images used for veneration are acheiropoieta: images not made by human hands (so their believers claim). The so-called Shroud of Turin is probably the most famous example of this, but there are and were many, many other examples. The Veronica images (Christ's face) is another famous example. Legends surrounding the idea of an image of God made without human hands first date to the sixth century with the appearance of the Mandylion of King Abgar. According to the legend, Christ imprinted his features on a cloth after a messenger sent by the king failed to capture Christ’s portrait in a painted image. The image healed the king of an illness and subsequently copied itself onto a tile when it was walled up during a Persian invasion; these miraculous events underscored its supernatural provenance and powers. From its origins in the northern Syrian city of Edessa, the story and cult of the Mandylion image in the Byzantine East eventually passed to that surrounding the Veronica in the Latin West. (for more on this topic and related topics: Hans J. W. Drijvers, “The Image of Edessa in the Syriac Tradition,” 13-31; and Averil Cameron, “The Mandylion and Byzantine Iconoclasm,” 33-54, both in: Herbert L. Kessler and Gerhard Wolf, eds., The Holy Face and the Paradox of Representation, Bologna 1998; also Hans Belting, Likeness and Presence: A History of the Image before the Era of Art, especially chapter 11, “The ‘Holy Face’: Legends and Images in Competition,” 208-224 and Appendix 37, 541-544, Chicago 1994)

So early on it seems there were, among other factors, things like pre-existing traditions combined with miraculous stories that contributed to Christians' use of images, and subsequent justification for their use. But certainly there were opponents, and iconoclasm (the deliberate destruction of images) was not unheard of. So, e.g., we have this quote from Gregory the Great’s Letter to Serenus, Bishop of Marseilles, from 599 AD: "I note that some time ago it reached us that you, seeing certain people adoring images, broke the images and threw from the churches. And certainly we commend you for your zeal, lest something manufactured be adored, but we judge that you should not have broken these images. For a picture is displayed in churches on this account that those who do not know letters may at least read by seeing on the walls what they are unable to read in books."

Here is then a potential justification for image-use: they function like books for the illiterate. BUT not everyone thinks Gregory really meant this claim, or that this was used as a common or viable defense of images. (I don't have the literature around this debate before me at the moment, but I can try to dig it up.)

The problem became particularly heated in the Byzantine East, where the Iconoclast Controversy lasts from 726-843 in the Byzantine Empire. Some of the major events are:

726 Leo III supposedly ordered removal of the Christ icon from the Chalke Gate of the Imperial Palace at Constantinople

754 Council of Hieria during reign of Constantine V condemns use of icons

787 Second Council of Nicaea during the reign of Irene and Constantine VI (Iconoclasm reversed)

814 Leo V reinstated iconoclasm

843 Empress Theodora and Constantine VI end iconoclasm

Among some of the ways image veneration was justified was the idea that venerated passed from the image to the thing or idea being represented. The veneration is thus not to icon per se, but the person or God behind it.

Of course, there are also accounts that suggest people saw real power in the images per se, so again there's a question of what was said vs what people believed, etc. E.g., this account: "[A certain woman] depicted the saints Cosmas and Damian on the walls of her house, being as she had an insatiable desire of seeing them. [She then fell ill]. Perceiving herself to be in danger, she crawled out of bed, and upon reaching the place where these most wise saints were depicted on the wall, she stood up leaning on her faith as upon a stick and scraped off with her fingernails some plaster. This she put into water, and after drinking the mixture, she was immediately cured of her pains by the visitation of the saints." (Acts of the Seventh Ecumenical Council, 787; you can read these Acts online here )

In an effort to sum up a long-winded and perhaps not entirely coherent answer: yes, the 2nd Commandment posed problems for images. Christians dealt with these problems in a variety of ways, many involving some pretty convoluted mental gymnastics (I encourage you to check out the articles or primary sources to see the lengths to which image-defenders, or iconodules, went to legitimize image veneration). But generally image-veneration won out, and it wasn't until the Protestant Reformation that you again see a serious threat to the viability of image veneration in religious spaces.

I hope this helps!

EDIT: typos, minor clarification

EDIT 2: I'm just remembering that some of the justification for images also centered around belief in Christ's full humanity (i.e., the idea that he was both fully human and fully divine). If he was fully human then it was only natural that he could be portrayed via imagery, and denying that was to deny his human nature (and thus heretical to orthodox belief). There were also some links made between God's taking on of flesh (via Mary, who provided the flesh) and the figuring of Christ in paint, if I recall correctly. As I said, the topic is BIG and the justifications were many...