To answer this question, we would first have to discuss another one: What do you believe mathematics to be? This is the so called Ishango bone. It is believed to be older than 20.000 years. Right away you'll notice the carvings which seem to be arranged in groups. Those groups have different numbers of elements. Maybe somebody used it to count something, maybe members of groups, maybe animals hunted, maybe days before it rained for the next time ect.. Would that already be mathematics?

I assume, you'd say no. But how about this: Jean de Heinzelin (Ishango. In: Scientific American 206 (1962), S. 105–116) found some patterns, such as that the left column consists of all the prime numbers between 10 and 20, the groups in the right column consist of the numbers 10+1 and 10 -1, as well as 20+1 and 20-1, some pairs in the middle column consist of a number and its double. This could of course be all completely coincidental. But it could also mean that whoever made this bone had a basic knowledge of arithmetic operations.

But I assume, you would not count this as a mathematical problem, because even in this case, it would just be the solution, and the bone would be lacking the formulation of a problem in verbal form.

What you were probably looking for is this: In general, it is stated that the oldest mathematical problems are Babylonian. If we ignore how the number system evolved and things were counted by Babylonians (if you want to know, see here, start on p. 33.], the first thing that comes close to what you are looking for, would be something called W19408,76. If you google this easy-to-remember name, you'll find a clay tablet, or broken remains of a clay tablet to be exact, from around 3200 BC. On it, you'll find the calculations of the area of two fields, done with the help of the so called agrimensors' method. (cf. Robson).

So does this answer your question? It might. It might not:

This text is often claimed to be the oldest mathematical problem in the world. But just as in the case of the Ishango bone (if we assume the markings to be something more than counting), it does not contain the problem itself, only the solution. So what is the difference?

More importantly, do you consider this to be mathematics? The agrimensor's rule is an approximation, so this tablet is clearly utilitarian. In fact, almost everything from Babylon is. There are however, some tablets with problems, which seem to be utilitarian at first sight, nevertheless, the dimensions e.g. of the fields given are completely unrealistic, so those calculations can be interpreted as non-utilitarian (or pure mathematics?).

But here is the thing: Calculations obviously are something that we would naturally considers to be a part of mathematics. But does a calculation in itself already constitute mathematics? Isn't mathematics more, because it also serves to provide insides about the properties of objects in a formal sense or constructs overarching theoretical structures? Is it important to have a proof that a mathematical procedure is correct in order for it to be mathematics? Because we only have calculations from Babylonian times. No theory, not a singular proof. (cf. Hoyrup's dicussion here) This may seem like nitpicking, but it actually makes quite a difference: If you feel, that there is a significant difference between calculations, like the Babylonians did them, and mathematics, as a theoretical discipline that has (at least in some cases) practical applications, this is not the oldest mathematical problem ever written down, but simply a calculation by a technique that the scribe happened to know. In that case, the oldest mathematical problem would stem from ancient Greece.