Slaves in the early 1500s were not particularly lucrative, because no one had ever thought of using slaves as mass labor to produce a cash crop. This model was actually fairly novel, and was only developed by the Portuguese after discovered that, one, sugar would grow well in tropical climes like those found in Azores or Madeira, and, two, it was easier to control slaves if they were taken from their homeland. This combination -- the cash crop of sugar and the access to and transportation of labor -- was the essential piece of the model of plantation slavery that we're familiar with in much of the Americas.

The Portuguese were the first to be heavily involved in the slave trade -- the British and the Dutch would be involved later, and the Spanish were never particularly involved and preferred to buy their slaves from the Portuguese (although the Portuguese had to obtain special licences from the Spanish mainland in order to trade with Spanish colonies, which lengthened the process and encouraged smuggling). Through their feitoria trading posts, which the Portuguese had established in the Kongo Empire and in what is now Angola, the Portuguese had more access than anyone else to Africans. This meant that prospective slave owners did not have to pay the additional price of a tariff, wait for an official recognized trading ship with the proper permits, or pay a smuggler extra money to arrive before the official ships.

Brasil was also optimally placed, relative to Africa, so that slave ships could move from Brasil and to Africa, and vice versa. The trade winds allowed ships to skirt the Brazilian coastline, docking at various ports, before taking off over the Atlantic towards Africa. It was even easier to go from Africa to Brasil.

So the combined factors of (1) being able to implement plantation slavery before anyone else, (2) being heavily involved in the slave trade and being able to bypass tariffs, and (3) having low transportation costs, allowed Brazilian slave owners to buy slaves cheaply. This meant that, unlike the British, they were less concerned with making sure that slaves lived long enough to reproduce. Their greater access to Africans, relative to the Spanish, also allowed for a terrifying scale unseen in any of the other cities, polities, or later, colonies, in the Americas. Because the slaves were so replaceable, and because sugar was so lucrative, owners didn't really care that their investments would likely die after two years of hard labor, often losing limbs to processing mills, dying of disease, dying of malnutrition, or dying due to injury. All so Europeans could have their sugar.

TLDR: Sugar. It was sugar.

Slaves in Brazil were growing sugar, for one. Sugarcane is very difficult ti grow, and harvesting it is quite dangerous. You have to cut the stalk and the remaining pieces are sharp enough to cleanly impale a person if they fall on it. In addition to the working conditions, the Brazilian slaves also had to contend with a host of tropical diseases that weren't present in places like the Southern US. Slaves died with much higher numbers in Brazil forcing slave owners to replace their workforces more frequently, meaning that Brazil needed to import more slaves.

Essentially, the work was more difficult, more dangerous, and the environment was more hostile. All this came together to create a "perfect storm" of making a slave's like absolutely horrible.