You notice the UK's budgets are presented in a red briefcase, so when did this become common practice? Why did we start using a red briefcase, and what form was the budget presented in before?
The briefcases used by ministers in the UK's government are actually known as ministerial boxes. They are all red in color (except for the black ones used by party whips) and built by the same company, Barrow and Gale, in London. Their design and coloring has remained relatively the same since the 1860s. The boxes are used by the ministers to securely hold their documents, since they are built to withstand lots of damage.
The box you see outside in the photoshoots is the one that the Chancellor of the Exchequer uses to hold his documents. The budget, being one of his documents, historically has been held in the case with the rest.
Basically to answer the first question, the budget has been kept in the briefcase since the briefcases came into use. Sorry that I can't answer what was used before it. I am not entirely knowledgeable on this subject.
Edit: Chancellor of the Exchequer, not Minister