I used to get that a lot when I used to be the type to give cold, icy silences when I was angry. So every time I'd be relaxing or quiet, my partner would say, "What are you upset about?" lol
I no longer give the silent treatment to anyone and I seldom get angry so that's not a problem for me anymore. If you're not the type who goes silent when you're angry or depressed, maybe your partner is going on the basis of past partners or even the way a parent reacted when they were sad/angry/depressed. I think you need to talk it out and tell them that if you are upset or depressed, you'll communicate with them.