The current law is the Presidential Succession Act of 1947, from the Truman administration.

I'm not sure how authoritative we should consider the Senate's own Web page at https://www.senate.gov/artandhistory/history/minute/Presidential_Succession_Act.htm . I suspect it was just based off of the source it cites, Feerick, John D. From Falling Hands: The Story of Presidential Succession. New York: Fordham University Press, 1965.

It suggests that it may have been because Truman at the time had good relations with Speaker of the House Sam Rayburn, and bad relations with President Pro Tempore Kenneth McKellar.

According to Christopher M. Davis, "The President Pro Tempore of the Senate: History and Authority of the Office", Analyst on Congress and the Legislative Process, Congressional Research Service, September 16, 2015, https://www.senate.gov/CRSpubs/6497e221-2551-4be5-bfc8-fa4837867771.pdf , the practice of the President pro tem being the most senior member of the majority party (and therefore likely to be a doddering geezer) only became the firm rule about 1945, though it had been becoming more common.

That report also notes that, after the 25th Amendment allowed the election of a new vice president to fill a vacancy, the President pro tem's elevation is unlikely except for an "unprecedented national catastrophe", so perhaps there's little pressure to change it.