It was mostly just done by eye. The typical torpedo of the time was powered primarily by compressed air. This could be heated by burning fuel to increase the pressure of the gas, and hence increase the speed of the torpedo. However, this method of propulsion requires the venting of the compressed air, and so the torpedo leaves a trail of bubbles behind it. These could easily be spotted by a ship's lookout.
Some ships, mostly escort destroyers, had hydrophones. These were an early form of passive sonar, and were basically an underwater microphone. These could pick up the screw noise from the torpedo, which is quite distinctive.