These characters were not on the original 10-key touch-tone phones (1964). They were added in 1968 with the introduction of the 12-key Western Electric model WE-2500. These keys were added to give phone menu systems an out-of-bounds character so that it was possible to encode variable length messages.

The internal mechanics of the touch tone system made it simple to add the two extra keys. Touch-tone numbers are encoded as a pair of tones played simultaneously that indicate which column and which row. The top row of keys all play a 697 Hz tone and the left column plays a 1209 Hz tone, so if you hit the one key it will send both a 697Hz and 1209Hz down the line. There was already an oscillator there for the fourth row, so adding those other two keys was very little additional hardware.

Sadly, they decided to arrange the keys with the one at the top. Adding machines had always put the one at the bottom because you use it more, and computer keypads took the design from the adding machines. This used to cause confusion when switching between computers and touchtone phones.

Computer keyboards have '*' and '#' because in 1961 Teletypes had them.... because in 1890s the Remington typewriter had them, because before that they were used in handwritten communication.

Interestingly, the Remington typewriters did not have a 1/! key, even into the second half of the 20th century. You got an exclamation point by typing "." then backspacing and typing an apostrophe. To get the digit 1 you were expected to type a lower case L.