The "military" of the IDF was definitely not created until after the British Mandate terminated. I will focus on the Haganah, however, since other groups weren't the backbone of the IDF that came.

As /u/MI13 pointed out, I have written answers on this in the past, and you should check out that answer. Hopefully this answer will be a little less all over the place.

Pre-1948 Israeli Paramilitary Organizations

The first paramilitary organization established by the Jewish population of Palestine (the geographical region under the Ottoman Empire, not a formal administrative unit) was called Bar-Giora in 1907, and it formed HaShomer (the guard) in 1908. HaShomer, as put by Benny Morris in Righteous Victims, was:

...a semi-clandestine armed organization whose sole purpose was to guard the settlements. The motto of Bar-Giora and HaShomer was: "In blood and fire Judea fell; in blood and fire shall Judea rise," from a poem praising the rebels against Rome.

The group essentially contracted out, and formed "conquest groups", it would clear out and begin cultivating (with armed defense) newly bought land before permanent settlers took over. The group never reached more than 100 members or so, with 300 Jewish hired hands, and they were often fairly aggressive in their "defending", arguing that Arab workers should not be allowed to work on the farms. They eventually fell out of favor with settlements for their policies. Next to form of the serious groups was the one that lasted far, far longer, and had far more impact: the Haganah.

In March/April of 1920, the Haganah was born, made up of veterans of HaShomer and also defenders of the Pale of Settlement previously. In the backdrop of this, and when British authorities took virtually no steps to stop the riots (drawing comparisons to Russian police during pogroms), Jews formed the Haganah (full name: Irgun Hahaganah, meaning defense organization). The Haganah was essentially a group that went around defending in localized fashion for the most part, armed with very little. They were headed by a 28 year old "veteran" of Jewish battalions, Yosef Hecht. Not much is known about him, he kept so secret that we barely have any information on him from the books I've seen. Unlike HaShomer, Haganah was not a private organization and was open for nearly any Jew to join, and it was under the authority of the Histadrut (the Jewish National Labor Federation). In May 1921, riots again broke out, and Haganah had its first real test. The Haganah had buried eight rifles, twenty pistols, and five hand grenades from the local branch in the dunes to the north, and couldn't find them. Some 300 Haganah "members", mostly youths with no training or experience, faced the Arab rioters with sticks. Usually operations involved things like 5 or 6 Haganah members wielding pistols (with or without ammunition) trying to get Arabs who were angry or rioting to stop or be scared away.

It wasn't until 1929 that the organization became a real, organized "defense" organization. After all, between 1922 and 1929 were fairly calm years, and very quiet ones at that. By 1929, Haganah had what was likely a few thousand members spread throughout the region, including some 300 in Haifa (of whom almost all were part-time members and only half could be armed). Drills, hiking tours, and training sessions/scenarios were discussed, but they were typically laughable in terms of actual value: members were asked things like "What would you do if an Arab came at you up front and two more on the left?" It wasn't an organization with military-like conduct, to be sure. Just to be clear, Haganah was considered illegal under the British Mandate, and so they were an organization in secrecy, which didn't help their attempts to organize and be effective.

In 1929, the Western Wall riots broke out. I'll stay away from why, because that's a story in and of itself, but the Haganah was to face its second major test here. During the first four days of rioting, the task of defending Jews fell almost entirely on the Haganah. The British had 142 policemen (half of which were in Jerusalem alone), assisted by 77 army troops, 169 Jewish and 1,063 Arab policemen. The high commissioner and police chief were out of the country, so command wasn't too centralized. The defense by the Haganah, which was organized around the individual commanders in each area, failed in pretty much all cases. Even in the best case, Haifa, 3,000 Jews were driven from their homes with many of them being looted. Elsewhere, like in the worst case of Hebron, Arab mobs essentially had free rein, committing the Hebron Massacre. The rioters were only stymied by a brave British policeman who used all his ammunition and even fought rioters with his bare fists, alongside some handful of assisting Arab/Jewish policemen (some historians say Arab policemen allegedly only fired into the air, and didn't fire on the rioters), and by the fact that many Arabs (fearing for their Jewish neighbors) sheltered and hid their Jewish neighbors in what would be one of the most touching acts of courage, in my humble opinion, in the history of the conflict.

1929 was a turning point for the Haganah. Its failures were noted by the Histadrut and heavily criticized (though the leadership praised the Haganah publicly for its courage), and Hecht was forced to resign, despite only being a titular scapegoat for a much-decentralized organization. The Zionist Executive took over the Haganah, which would essentially be led by David Ben-Gurion (Israel's first and probably most famous Prime Minister) by 1935. In light of 1929, the Haganah set out to rebuild and reorganize in much, much better fashion.

First, the Haganah set out in search of funds. By the end of 1929 (the riots happened around August for reference) they had managed to gather 800,000 Palestinian pounds; no small sum considering the average Arab peasant family had an average annual income of 27 Palestinian pounds. They used the money to rebuild destroyed settlements, add patrol roads, fences, shelters, searchlights, and so on. They analyzed the failures that had led to the 1929 riots spiralling so far out of their control, and focused on getting better training, better intelligence, and communications between the headquarters and the localized commands. In 1931, Saul Avigur was appointed head of the organization, right as Palestine was about to undergo an economic boom (while the rest of the world was in the Great Depression's grip), mostly thanks to new and huge numbers of immigration. Here, the struggle of whether or not the organization was to remain localized or a central one came to the fore, since many Haganah local commands would hide their arms counts and resources from central command since they didn't trust central command to efficiently work. In 1934, Avigur wrote a document titled "The Foundations of Defense", where he emphasized that the organization would be run like a military, would be centralized, and would be the sole body responsible for Jewish defense in the region. While in practice this was hard to implement, and laziness reigned supreme, this would mark the first real attempt to centralize by the head of the Haganah that we know of. However, since there was no "mortal threat" to a state, and since a Jewish state seemed so far off, from 1930 to 1935 there was a general focus on just gathering arms. Agents went to Belgium, France, and Italy, where the first two had lax arms controls and the last was happy to help (being a Fascist government hoping the help would hurt the British) supply the Jewish group. In 1935-1937 the Italians even hosted the first naval training in their port of La Spezzia for the Haganah. Arms began to be imported: almost all needed repair and maintenance, and ammunition was short, so an arms industry was set up. By 1935, the Haganah was producing more than a hundred hand grenades per day, for example. Training continued in the amount that it had before, Saturday mornings and one evening a week. However they became formal occasions, and foreign training manuals helped devise a training course of 120 hours a week. 1/4 would be drills with fieldcraft, minor tactics, weapons handling, and firing 10-15 rounds a week (huge achievement to have ammunition for that). However, other training would be maneuvers (carrying bricks to simulate arms) and planning sessions.

CONTINUED BELOW

I asked a similar question a while back and got a great answer from /u/tayaravaknin. You might enjoy reading his replies while waiting for responses in this thread.