It's one of the most difficult definitions to make, so I won't.

The historical context is that between 1000-1250 knights developed as an aristocratic class (rather than being simply a description of a class of soldiers, milites, they became chevalier). Part and parcel of this was the codification of what values pertained to being a knight, and therefore a member of the aristocracy, and what activities and values were expected of a knight: fighting in tournaments, granting ransoms, courtly love - the period c.1150-c.1220 is when the values of chivalry really begin being consciously developed.

In this period, while expensive, it was feasible to become a knight and the 'land-glut' of the late eleventh- and twelfth-centuries meant that there was plenty of incentive for landholders to grant out holdings to support and maintain knights for themselves (alongside the bulk of their knights who would serve in the household as retainers).

During the twelfth-century knights began to assume lots of administrative burdens (such as being sheriffs and bailiffs) for the English crown. This increased in intensity into the thirteenth-century when the Crown administration in localities was solely run by knights and for which knighthood was a pre-requisite. The cost of the ceremonies and activities of knighthood, such as fighting in tournaments, began dramatically increasing. The result was astounding, people (including important landholders) said, 'Sod this', and stopped being dubbed as knights. The Crown was faced with a dearth of officers, and instituted a command that anyone with an annual income of £40 or more must become a dubbed knight. Knighthood became, increasingly, the preserve of the rich and powerful - knighthood moved from being a ubiquitous part of aristocratic life to becoming the marker of the aristocratic elite.

That they stopped being dubbed as knights, participating in tournaments, or being officers of the Crown did not mean that these non-dubbed warriors stopped ascribing to the values of knighthood, nor that they stopped performing as knights in war. However, they patently were not knights. Thus the term man-at-arms began to fill this gap, they were not commoners but they were not members of the highest ranks of society.

Men-at-arms is a weird and fluid term which is almost impossible to define, it was used by fourteenth- and fifteenth-century authors to address audiences which definitely included dubbed knights, and it was one which was used in practical documents to describe undubbed individuals or groups who were performing knightly military duties.

(Edit): I should add that men-at-arms also included mercenaries, and thus could cover those who were non-aristocratic but fought in a manner perceived to be aristocratic - or it could be used as a label to bash those who were doing damage by both lay ('[by] hardiness and pillaging [they] make themselves men-at-arms' - Philippe de Mézières) and ecclesiastical commentators. It was a catch-all term.

For more information see:

Crouch, D., The Birth of Nobility: Constructing Aristocracy in England and France, 900-1300, (Harlow, 2005).

Coss, P., The Knight in Medieval England, (Woodbridge, 1993). | The Origins of the English Gentry, (Cambridge, 2003).

Keen, M.H., Chivalry, (London, 1984).

Taylor, C.D., Chivalry and the Ideals of Knighthood During the Hundred Years War, (Cambridge, 2013).