This is not my direct area of expertise, and I'm sure others will bring in more sources, but during the Great Depression the Federal Writer's Project recorded over two thousand interviews with former slaves. You can learn more about the project, and access information from the Library of Congress collection titled Born in Slavery: Slave Narratives from the Federal Writers' Project, 1936-1938 .

You may be using a search tool or search terms that are leading you astray in some fashion, because there's a fairly large number of slave narratives that are fairly accessible either in published form or via digitized archives--as one respondent notes, the Library of Congress has quite a few resources. The WPA-era project has already been mentioned. You should also consider famous autobiographical narratives or memoirs of slaves that are so well-known that they might at first not seem like "slave narratives" per se--Fredrick Douglass' autobiography, Solomon Northrup's 12 Years a Slave, or Oladauh Equiano's memoir and others like it that were published or circulated by British and American abolitionists from the end of the 18th Century into the middle of the 19th Century.

You ask about narratives from "even earlier", and it may be based on your OP that you are looking specifically for narratives from slaves who worked on the White House. The earlier you go, the harder it gets to find not just first-person narratives by slaves or ex-slaves but frankly first-person narratives by most people in the early modern Atlantic world. Autobiographies or memoirs as we imagine them today were a relatively uncommon thing as a literary genre prior to the 18th Century. You may want to consider here using works by historians who focus on the lives of particular slaves or particular groups of slaves. Roger Aden's 2014 Upon the Ruins of Liberty focuses on the contested effort to build an exhibit at the site of Washington's Philadelphia house (its ruins are right in the same area as Liberty Bell Pavilion and Independence Hall) discussing the slaves he kept at this property. Josephine Pacheco's book The Pearl (2010), about an 1848 escape attempt by 76 slaves near to Washington DC, shows another way to get at slave testimonies, namely, those given during court proceedings--obviously under duress and tightly limited to the matters being adjudicated, but a window into the lives of individuals nevertheless.

When I was working on a certificate in Public History, my class did learning exercises in researching enslaved narratives. A lot of State libraries and Historical Societies have online archives for enslaved narratives.

Here is a database in my State that allows you to search enslaved narratives:

https://www.virginiamemory.com/collections/aan/search-the-narrative

Thou might also try Documenting the American South from UNC Chapel Hill: https://docsouth.unc.edu/index.html

Lots of narratives of slaves and is keyword searchable.

This is fantastic! Thanks!