You're right, of course; the economics of academic book publishing are such that a great number of titles are priced beyond the reach of individual purchasers, because they are, in fact, targeted at academic libraries. For the same reason, many publishers wont even make Kindle editions affordable, for fear of undercutting their own market.

There is no straightforward solution to your problem – no central source of rentable academic titles that covers the whole market, that is. That said, many individual publishers, such as Taylor & Francis here in the UK, do now offer a (not that affordable in itself) rental option on many books and academic papers. If you have ever attended a college, especially one that has a university press, it is also worth investigating what sort of resources it makes available to alumni – I have signed up to the alumni portal of my alma mater, and through that can get free digital access to all the books published by its university press, and also the articles available via JSTOR, MUSE, Emerald and Sage.

Finally, there actually are some rental services offering academic titles more generally out there. These can be very pot luck – the market is huge, their reach slight – but I have found the one offered by Scribd at £10 a month to have the widest variety of academic titles to rent. Scribd works much better if you are interested in browsing some interesting-sounding titles looking for something to read than if you are desperate to find this one book that is urgently needed for some project, but it's definitely a start.

Besides the options mentioned here, if your local public library in the US does inter-library loans (ILL) you may be able to request copies of the books that way. What will be available depends on what networks they borrow through.

For example I live in Massachusetts. There is a state wide ILL network that includes public libraries, the libraries at the state universities, and a number of smaller college libraries. You could get just about any book if one of the libraries had it, the book would be delivered to your local library for pickup and return.

At the moment a number of libraries are not participating due to COVID, but once things return to more normal will be available as sources for books.

I understand outside the US libraries are a bit different, but similar arrangements may be available.

You should look at the booklist on the wiki https://www.reddit.com/r/AskHistorians/wiki/books Many are not particularly expensive (and there are also several free resources listed)