I know there is a specific term for the artwork behind doing those things, just cant remember what its called.
An illuminated capital/initial. Those which depict a specific person therein are referred to as historiated capitals.
Historiated or illuminated initials are the usual term, historiated meaning specifically a initial with a scene or narrative images in it. I assume that you are talking about the illuminated manuscript ones; depending on the context they can be used to introduce a new paragraph or a new section/subsection of a text. For instance, the beginning of books of the bible, or of psalms in a psalter, are both places one might find historiated or illuminated initials although the actual number of these and their elaboration would vary a great deal depending on the fashion and circumstances of the buyer. This leaf is a nice example of a historiated initial with some less elaborate initials below it:
http://www.getty.edu/art/gettyguide/artObjectDetails?artobj=136157&handle=li