To start, they were only active for a short time. The title wasn't bestowed until early 1862, and six months later Ward, the commander, was dead.

Did this force suffer any notable defeat in the course of its service in suppressing the Taiping Rebellion?

Absolutely.

Prior to receiving this title, his army had a much worse record. They failed horribly at the start in assisting the Imperial forces to take Songjiang, a town which is today a southwestern suburb of Shanghai. They were under-prepared and the overall advance was poorly planned out on the Imperial side as well, so they can be forgiven for that. Shortly after, with more men and better preparations, they again made an attempt on Songjiang, and were successful, barely. If it was a major victory it was more in the sense that it drew attention to the Foreign Arms Corps, which brought them more men and more resources.

The very next month they suffered yet another major defeat when moving on Qingpu, the next town over. The Taiping forces knew they were coming and made quick work of the Foreign Arms Corps. For six months the Corps was basically nonexistent. Ward was getting over injuries sustained in the previous defeats, and half their recruits were killed in the previous battle. Not much of a victory to be claimed there.

Six months after that, they were once again defeated in trying to take Qingpu. This was the catalyst for Ward shifting the makeup of his army to include many more Chinese. We can say that this is the point that they became what was later called the Ever Victorious Army. They also took a whole year to train at this point, which is something they had lacked in the past. By 1862 they had better training, better supplies, and an extra degree of shock and awe on their side; To see Chinese soldiers in ridiculous clothing skilfully managing expensive arms played no small part in the few victories they had from this point on.

How was it able to maintain such a record if so?

So that was January. As I said, in March they were given the title of Ever Victorious Army, and in September their commander was dead and his replacement was unable to hold it together. We're looking at a period of only a few years of existence, and only a few months of having such a lofty title. There simply wasn't much time passing that made it hard to maintain the record. The height of their notoriety was a period of six months before it all fell apart. For six months they could maintain that record.

Or conversely, how did they improve over the years if not?

They simply didn't last years.

Tried to search around here, but only got this one asking on how Gordon is remembered in China

Not a lot of us talking about the Taiping Civil war here. I've answered a few, but to my knowledge no one has ever asked specifically about the Ever Victorious Army.

tl;dr: They were only ever-victorious if you ignore their prior major defeats, and the ultimate defeat 6 months after getting the name.