Can we see under the hook of deep models?