Probably to seek shelter in the international quarter of the city. Before the Japanese entered the city a number of foreigners, headed by John Rabe (a Nazi party member) established an international safe zone around of 3 square miles. Over 200,000 Chinese civilians sought shelter in the safe zone during its existence. While Japanese troops were given free reign to commit atrocities elsewhere they were somewhat more retrained when it came to dealing with civilians sheltered in the safe zone.
That is not to say civilians in the safety zone remained unmolested, the Japanese would enter the zone, carry off hundreds civilians and rape or kill them. The international safety zone is credited with saving the lives of tens of thousands of civilians.