I was looking at Google maps and noticed that a lot of the borders between countries in Africa are straight lines on the map, as opposed to the jaggedy borders found almost anywhere else. Why is that?
This is a complicated and very broad question, so any answer will inevitably involve a lot of simplification. In general, early states on all continents were often divided by natural barriers, such as rivers, large lakes, and mountain ranges, as these all often constituted serious obstacles to movement. Indeed, most of the national boundaries in Europe are geographical boundaries as well. Germany and Poland are divided by the Oder River. Spain and France by the Pyrenees Mountains. Greece and Turkey by the Maritsa River. Several of these boundaries, such as the France-Spain example above, were formed over the course of centuries, while others are formed by much more recent treaties, such as the German-Poland border established by the Potsdam Agreement in 1945.
Both of the Americas, Africa, and Australia, were colonized by European powers, starting in the mid 1400s and lasting until the mid 1900s. Most of these colonies obtained independence during the 1900s, and only a handful of ‘overseas territories’ remain in the hands of their colonial overlords. As over a dozen European states divided up faraway continents between themselves, they conquered the lands of indigenous peoples of those areas, and drew borders dividing their colonies chiefly in order to avoid conflict with other powers. The earliest such division to be formally made was the Treaty of Alcáçovez in 1479, between Castile (modern Spain) and Portugal, which declared that Castile could not establish new colonies south of the Canary Islands. Hundreds of other such divisions would be made in the ensuing centuries, including what is now perhaps the most famous colonial border in 1818. The London Convention determined that the border between the United States and Canada (then a British colony) should be extended along the 49th parallel. This created by far the longest straight border in the world.
But not all such colonial treaties produced straight lines. Indeed, geography continued to be used as a reference for colonial borders, and these borders typically remained either unchanged or minimally changed following the independence of those colonies. While Chad and Libya are divided by two perfectly straight lines cutting through the Sahara Desert, the DR Congo is divided from Tanzania by the massive Lake Tanganyika. This trend of "some natural and some artificial" borders holds true on all continents settled by Europeans. The America-Mexico border is determined by both the Colorado River and Rio Grande River, as well as five perfectly straight lines. There is even one case in Africa (at least that I’m aware of) of pre-colonial borders surviving to the present day. Rwanda has notably ‘jagged’ borders descended directly from a centuries old Kingdom there. Rwanda was subjugated by the German Empire following the Berlin Conference (explained by /u/souHad) (EDIT: souHad's comment has been removed. Berlin was a very broad conference focused on formalizing African colonial borders between the 7 powers colonizing it, which occured in 1884), and later by Belgium following the First World War. In 1962, Rwanda regained its independence as a republic and kept the same borders as the old Kingdom.
There are straight treaty-determined borders in Europe too, but they are not as common as on other continents. The most obvious of these is the Russia-Poland border, a patchwork of a few dozen linear borders,that mostly form clean north-south divide. This was established by treaty following the defeat of the Nazis at the end of World War Two in Europe. A similar example exists between in a small section of the Ukraine-Romania border just north of the Suceava River, and most of the Norway-Sweden border.
There's a good answer to the Africa part of your question here:
edit: credit where credit is due - /u/khosikulu
One of the more vexing problems that cartographers faced when desiring to convey abstract geographic data on a flat piece of paper is that ... um guess what, the earth is not anywhere flat, which means that there are no "straight lines" possible on the earth's surface. Mapmakers gradually accepted that the earth is spherical, then they refined their understanding to be an oblate spheroid, and then with more precise measuring devices have determined that it is ever-so-slightly pear-shaped, with certain small irregularities. Even to define what constitutes the proper "surface" or zero-level, at any given point, involves some contention due to local gravitational anomalies. Treating this subject comprehensively is far beyond this subreddit, the entire history and technological development is fascinating but requires volumes. Even today there are hundreds of people in full time career positions, "geodicists", whose entire job consists of further refining our understanding of this problem.
Just to throw in one anecdote ... the border between Ethiopia and Eritrea was drawn on a flat map in an office in London after the first world war, as a "straight line". That map had been created using a "mercator projection" which is one method of schematically warping "what is on the ground" to a flat piece of paper. Later on others tried to draw the same "straight line" on a map drawn according to the transverse mercator schema. On the actual surface of the earth, there is a sliver of land between those two "straight lines" as much as 2 km wide. Most of that land is extremely arid, supporting no vegetation at all, devoid of springs, oasis, or other water. Near the end of the 1900's, the two nations got into an argument about which curved line was the correct one. They fought a war in which over 120,000 men died, and did not resolve the issue, there are still military fortifications and outposts ready to resume hostilities at a moment's notice....