Every time I wear my Confederate flag shirt my dad gave me at school, I always get ugly looks and get called "white trash" even though I'm Hispanic..
What's your assumption on why it's negatively approached?
I'm sure that to you, it's just a shirt your dad gave you. The thing is that it's also the flag of the Confederate States of America, a rebellion dedicated to the preservation and perpetuation of the enslavement of black Americans by white Americans with all the tortures and suffering that entailed. This association goes back for as long as the flag has existed, since it was literally designed for that job, just as the American flag was designed to symbolize the United States.
The very short version is that you're shirt which symbolizes this:
The new [Confederate] constitution has put at rest, forever, all the agitating questions relating to our peculiar institution — African slavery as it exists amongst us — the proper status of the negro in our form of civilization" and further that "Our new government is founded upon exactly the opposite idea ["equality of the races"]; its foundations are laid, its cornerstone rests upon the great truth, that the negro is not equal to the white man; that slavery — subordination to the superior race — is his natural and normal condition. This, our new government, is the first, in the history of the world, based upon this great physical, philosophical, and moral truth.
There is a great deal more to the case than that one quote, but it's a good start. For further reading, I can't recommend this post by /u/georgy_k_zhukov enough.
I can't speak to what everyone you see thinks when they see you in the shirt, but think about it like you would if you saw a person at school in a t-shirt with a band name on it. Or a jersey for a sports team. You'd probably think they're big fans of the band or team, right? I personally brand a small part of my online presence with a headshot of a Lego figure of Abraham Lincoln because I love Lego and the president. And also because I'm a dork.
You don't wear merdhancise branded with things you hate. We pick out such clothing, jewelry, and other cultural markers to express our beliefs and identities. By wearing them, we send the message that this is us, who we are or want to be and what we think is right and good. The people you see at school probably haven't read that quote -I didn't until I was in university- but they know the general facts it expresses and connect them with you.
I understand it's not a shirt you personally picked out and you don't mean those things by it, but that's something that other people can't easily know just by looking at you. It sucks that you're being treated unfairly because of that. This is just how human cultures work; we have symbols and we connect them to ideas and people. Sometimes that means we make mistakes, whether it's in misjudging people who are wearing gifts not of their own choosing or the choosing to display symbols we don't fully understand, like the stereotypical college student who gets a tattoo they can't read and then finds out it doesn't say what they thought.
All this said, if you have questions about anything in Zhukov's posts or the Confederacy, slavery, and so forth in general, you are in the right place and many of us are happy to help. It's literally what we're here for.