The word capital is a Latinate version of the word chapter (which exhibits a French sound change), both from Latin caput 'head'. Standardized capitalization like you see in modern English is a recent development. Early manuscripts usually wrote letters in a uniform size and even without much in the way of meaningful punctuation, but there was a tradition of making the letters that began a new section (or chapter) larger or more visually salient than others. This could take many forms, with the emphasized letters being decorated with some extra flourishes, written slightly larger, and/or written in red, and some texts would emphasize the first word or even whole first line of a new section. The Ostromir Gospel, the oldest dated East Slavic text, is a nice example of the practice. On its first page, the very first letter is lavishly illustrated--this is not a quirk of the first page, but throughout the text the beginning of each new reading has a large, decorated letter, and within readings you will see smaller decorated letters to mark the start of new sections within a reading, or to set off instructions about the day associated with the following reading. I'm not sure exactly when the tradition of capital letters to set off sections developed, since my paleography class focused on Medieval Slavic traditions and it definitely predates the 10th century, but the name definitely comes from their association with marking chapters.