Same Matrix? Shame on You!