The Grammy winner responded to criticism that her outfit this year was "trash," reminding people that she didn't even attend