This paper probes the behaviors of DNNs in generalizations. We find that the information of out-of-domain generalization encodes more “linearly” on the intermediate layers than other layers for VLCS and PACS, but not so for RotatedMNIST.
Out-of-Distribution Failure through the Lens of Labeling Mechanisms: An Information Theoretic Approach
Introducing additional labels can help OOD generalization performance.
On the data requirements of probing
This paper uses learning theory to recommend suitable dataset sizes to reliably reproduce probing findings.
Neural reality of argument structure constructions
This paper adopts psycholinguistic studies on neural language models to analyze where the meanings are carried.
Cool Books in 2021
Here are some books I found interesting in 2021.
- 西方现代思想讲义（刘擎）
- Capital and Ideology (Piketty)
- Keep Sharp: Build a Better Brain at Any Age (Gupta)
- The Nature of Expertise (Chi etal)
Quantifying the Task-Specific Information in Text-Based Classifications
In this paper, we estimate the task-specific information in text-based classification datasets.
An unsupervised framework for tracing textual sources of moral change
In this paper, we propose an unsupervised framework that infers the source text with prominent influence on the moral time course.
What do writing features tell us about AI papers?
This paper studies the writing features and their indications towards conference / workshop appearances in AI venues.
How is BERT surprised? Layerwise detection of linguistic anomalies
This paper uses Gaussian mixture model to detect how contextualized LMs show surprisal given different types of anomalies.