"Early" is a pretty tricky time frame here. Can you give some more guidance? For example, the "earliest" mathematical work (e.g., from Ancient Mesopotamia) that we have found was recorded on clay tablets. The tools that mathematicians used have varied dramatically by place and time, from clay tablets to the abacus to the chalk board to the modern computer.