It really wasn't effective at all.

Originally, enemy reconnaissance planes that encountered each other would sarcastically wave and smile back and forth, lacking any real way to engage in combat. There are reports of planes ramming into one another in an attempt to bring each other down, but this generally resulted in both planes crashing. Not very efficient.

Soon they started bringing grenades and other munitions to throw, or used their own pistols or rifles to attack enemy planes. These tactics proved to be extremely inaccurate and were rendered useless when the first machine gun was developed that could shoot synchronized through the propeller. This invention is widely credited with creating the basis for modern dog fighting.

The Synchronization gear article on wiki has some interesting information on it.

This page has some really good info about aerial combat in general, as well as the first reported aviation kills using handheld and front-mounted machine guns.

edit: Added links to some sources. And punctuation.

During the first few months of the war pilots were strictly used in a reconnaissance form. Generally there would be two people in a plane. The pilot and the map maker. Like stated in another comment, at first the pilots would simply wave at each other and that was about it. Later there started to be more animosity as the fighting on the ground got more fierce. Leading to the pilots to be a little more angry in the air. At this point the map makers started to carry weapons, starting with rocks and grenades, eventually leading to pistols and rifles. The passengers would be the ones using the weapons, so even if there was a kill, it would have gone to the passenger rather than the pilot. William Mitchell discusses the growth of using weapons on planes in his Memoires of World War I. He talks about the growth of using small armaments to actual rifles and machine guns. He doesn't directly discuss any deaths from the small caliber weapons. But he does make mention of how the planes would get bullet holes in them and occasionally ricochets and debris from the plane could cause the pilots or map makers to need some medical service. Arch Whitehouse discusses a little more of the heated battles, including Bloody April, but he does make some mention of using small arms in air combat in Decisive Air Battles of the First World War but it is limited to saying that they happened and no real casualty count from it.

So, the real dog fights started when people started carrying heavy machine guns. Once the passengers were able to use swivel guns, that is when pilot deaths really started to rise. The actually death count to becoming an "ace" didn't actually start until the middle of the war when the pilot himself had control over machine guns mounted on the wings. These were still wildly inaccurate. Eventually a dutch pilot was able to develop a machine gun that fired at a rate that would allow the barrels to be behind the propellers. This made it so the pilots could be more accurate. The Red Baron said the first time he saw one plane doing this, "it looked like it was breathing fire" (from The Red Baron book).

Both sides would steal other's planes and steal their research. So it was a slow fight to take the skies.

If you have any other questions I would be glad to elaborate more.

Edit:1. I was wrong about some timeline stuff. Working on fixing it. 2. Adding sources

Jean Navarre's first aerial victory, in 1915, was over a German Aviatik B.1 brought down by his observer, a Lt. Jean Robert (or Roberts) with three (!) shots. The German plane landed behind French lines and was captured mostly intact ( High Flew the Falcons, Herbert Molloy Mason, Jr.)

He also relates that Navarre chased after what he thought was a zeppelin armed with a kitchen knife, but this turned out to be just a zeppelin-shaped cloud - this is also mentioned by René Chambe in Au Temps des Carabines - although he states that it may have been a joke by Navarre. Early war French observers were often armed with Berthier Mousquetons or Winchester repeaters.

BBC states that the first air-to-air victory with a hand gun took place in October 1914:

On 5 October, 1914, the first aeroplane in the world to be shot down from another aeroplane was a German two-seater Aviatik, piloted by Feldwebel Wilhelm Schlichting, with Lieutnant Fritz von Zangen as his observer.

It was brought down over Rheims, France. Sergeant Joseph Frantz (pilot) and Caporal Louis Quénault (observer) of the French Air Service were returning from a mission in a Voisin Type 3 (pusher), when they spotted and fired on the German aircraft. Quénault's Hotchkiss machine gun fired about 48 rounds (two clips) before the gun jammed. At this point von Zangen, the German observer, fired at them with his rifle. Quénault returned fire with his carbine, hitting the pilot. The plane, out of control, crashed to the earth and was destroyed.

This was witnessed by French troops on the ground, and thus became the first confirmed air-to-air combat victory.

There are stories of at least one French pilot returning to base with a hole in his fuselage where he had been hit by a brick (!) thrown by a German pilot, but I can't find a useful source for this. Other weapons included heavy steel darts (flechettes), hand grenades, grappling hooks on ropes, shotguns, pistols, and middle fingers.

Not WWI, but WWII

There's the story of Owen J. Baggett who may have been the only person in history to down a Japanese Airplane with a M1911 Pistol.

Given the source I would take it with a grain of salt. But from Wikipedia:

On March 31, 1943, the squadron was instructed to destroy a bridge at Pyinmana[4] but before reaching their target, the B-24 bombers were attacked by Japanese fighter planes. Baggett's plane was badly hit, and the crew were ordered to bail out. As the B-24 exploded, the Japanese pilots then attacked U.S. airmen as they parachuted to earth, one plane approaching Baggett within feet, and then, nose-up and in an almost-stall, the pilot opened his canopy. Two of Baggett's crew members were killed and Baggett was wounded, who played dead in his harness, hoping the Japanese would leave him alone. Baggott claims to have shot the Japanese pilot with his pistol,[7][8][9] becoming legendary as the only person to down a Japanese airplane with a M1911 pistol.[1][4][10][11][12] He was later captured by the Japanese, and held for over two years.[1][3][13] After 30 months captivity, he and 37 other POWs were liberated by 8 OSS agents who parachuted into Singapore. [14]

EDIT: A better source: an article from Air Force Magazine with more details of the incident.