The problem with this quote is that it is very poorly sourced. It is a variation of some allegedly frequent comment by Charles V, but the first time this famous phrase appears is in a book from 1601 on speech by Girolamo Fabrizi d'Aquapendente.

According to Acquapendente (De locutione, page 64), the concrete phrasing would be this:

Unde solebat, ut audio, Carolus V Imperator dicere, Germanorum linguam esse militarem: Hispanorum amatoriam: Italorum oratoriam: Gallorum nobilem

Translation: Whence Emperor Charles V, as I hear, used to say that the language of the Germans was military; that of the Spanish was of love; that of the Italianswas rhetoric; that of the French was noble.

On the same page, Acquapendente makes a further comment, pointing out that a German person had heard some other variation of the statement. The comment from the alleged German man is also as poorly sourced as it can get, being tantamount to hearsay:

Alius vero, qui Germanus erat, retulit, eundem Carolum Quintum dicere aliquando solitum esse; Si loqui cum Deo oporteret, se Hispanice locuturum, quod lingua Hispanorum gravitatem maiestatemque prae se ferat; si cum amicis, Italice, quod Italorum dialectus familiaris sit; si cui blandiendum esset, Gallice, quod illorum lingua nihil blandius; si cui minandum aut asperius loquendum, Germanice, quod tota eorum lingua minax, aspera sit ac vehemens.

Translation: Indeed another one, who was German, narrated that the same Charles V sometimes used to say: if it was necessary to talk with God, he would talk speak Spanish, for the language of the Spanish carries graveness and majesty; if with friends, in Italian, for the dialect of the Italians was familiar; should someone be softened, in French, for no language is more tender; if to threaten or to speak harshly to someone, in German, for their whole language is menacing, rough and vehement.

Nothing can be found in any earlier sources like Gonzalo Fernández de Oviedo, Francisco López de Gómara, or any other notable chronicler from the time. However, from chronicler Prudencio de Sandoval, well informed though quite late we manage to know the languages he spoke, and it is quite coherent with Acquapendente:

Duke Charles knew well the Flemish, French, German, and Italian languages, and bad the Spanish one until he was a man. He understood some Latin.

Alonso de Santa Cruz, writing in 1550 (Chronicle of the Emperor, chapter VI), is of the same opinion about Charles V and how much of a polyglot he was, though he omits the German language:

And his great memory was known in the variety of languages he knew, that was: Flemish, Italian, French, Spanish, all of which he spoke as if he knew but one.

This array of languages matches fairly well what Acquapendente had stated, though throwing Flemish also into the mix. Charles V indeed spoke Spanish very poorly in his youth, and that was one of the main complaints of the Castilian Cortes when Charles took posession of the realms of the Crown of Castile and León. This lack of command of the Spanish language can be attested by a letter from Archbishop Alonso Manrique to Cardinal Cisneros dated 8 March of 1516:

His Highness does not know a word of Castilian, though he understands a little of it, a very grave thing that has already been noted, and the appropriate counsel has been giving, but until now nothing has been done of what would be convenient.

Despite this, Charles learned Spanish for good, and developped a particular liking to that language, to the point that when he was crowned emperor in Bologna in 1530 he made a very passionate speech in Spanish, much to the French ambassador's annoyance. Against the ambassador's annoyance, the now emperor responded in no ambiguous terms:

Understand me if you so wish, and don't expect from me any words but in my Spanish language, which is so noble that should be known and understood by all the Christian people.

Charles V was also very pragmatic on the matter of languages, as he clearly stated on the instructions he left to his son Philip in 1543, where we can read the following consideration:

being it necessary for everyone I think, my son, so it is to you more than to anyone, for you see over how many lands you shall lord, over how many lands, how different ones are from the others, and how different their languages; for which, if you want to have and enjoy them, imperative it is to be understood by them and to understand them, and for this nothing is more necessary and general than the Latin language. For which I strongly pray you to take the effort to learn it so much as to dare speak it fluently; it would not be bad to know some of the French one too

So, lacking concrete evidence of Charles' famous alleged quotation, we have some solid glimpses of the languages he knew, and what he actually thought of Spanish and of Latin, one being his preferred language later in life, and the other one the language he considered most useful, as it was de facto the lingua franca of the educated people throughout Europe.