The first machine gun fired from the air was demonstrated in 1912, when a Lewis Gun was fired from the air at stationary targets at the U.S. Army Aviation School in Maryland.

I don't know if any weapons were mounted on any aircraft at the outbreak of war, but if not, the situation was quickly remedied. The first air-to-air kill was performed on Aug. 25, 1914, when Russian pilot Pyotr Nikolayevich Nesterov rammed an Austrian aircraft after trying to shoot it with a pistol. Nesterov was killed when both aircraft crashed.

A Voisin III, a French bomber, scored the first air-to-air kill not involving ramming when a flight on Oct. 5, 1914 shot down a German aircraft with machine guns. The Germans were armed as well, returning fire with rifles. When the Voisin's machine gun jammed, the French aircraft (there were two men aboard) turned to a rifle of their own.

The first purpose-built fighter aircraft ─ defined as an aircraft designed to shoot down other aircraft ─ was the Vickers Gunbus, which was armed with a single 7.7mm Lewis Gun. The Gunbus was a pusher aircraft; it had its propeller in the rear. This meant the machine gun was in the nose of the aircraft, operated by the plane's observer.

The Gunbus entered service on the front Feb. 5, 1915 when RAF Squadron No. 2 received its first models.

A year ago, in response to a similar question, /u/Lord_Bob offered this video, which features an interview with Charles Euan Cuthbert Rabagliati, who is generally given credit for shooting down a German Aviatik as an observer of the Royal Flying Corps' No. 5 Squadron with a .303 rifle in 1914.

Follow-on:

I remember when I was a kid, reading about how early guns on WW1 planes weren't real great, and that some pilots would literally try to throw bricks at each other's propeller...and that several planes were "shot down" this way.

My question is, is there any truth to any of that?

In WWI, the big problem with mounting fixed machine guns on the front of an airplane was the propeller. Mounting a machine gun onto the front of an aircraft and firing would mean that you were just going to shoot yourself out of the sky. This meant that early aircraft were limited to fighting eachother in a few ways-

1. Firing small arms (rifles) at opposing aircraft. This was incredibly innaccurate.
2. Push Props- With propellers mounted at the back of a plan, a machine gun was free to face forward.

In December 1914, A french aviator named Roland Garros had his propeller blades armoured and mounted a machine gun to the front of the plane. The idea here was that instead of eating through the propeller, the bullets would just deflect. Garros was able to shoot down two planes like this until he was eventually shot down in April 1915 over enemy lines. The Germans captured both him and his plane, though he did try to burn the plane.

  The real shift to WWI fighter aircraft as we conventionally understand them was the invention of the synchronization gear, which was developed by Anthony Fokker, a dutch aircraft designer in the employ of Germany. This synchronized the machine gun to the propeller, which timed the machine gun to shoot through the gaps in a spinning propeller. It was introduced to the front the summer of 1915 with the Fokker Eindecker Monoplane. This led to the "Fokker Scourge," a period of a few months where Allies had no real answer to the eindecker. Eventually, the French and British started fielding the Neiuport 11 and the Airco DH2. Neither plane had a synchronization gear, but instead either mounted the machine gun above the propeller, or was a push prop. 

As far as sources go, There's The Canvas Falcons, by Stephen Longstreet. There's also a fantastic Nova special that tracks the WWI arms race in the air, called the First Air War.