Language Model Evaluation Beyond Perplexity

Introduction

Neural language modelsIn this work, we do not use the term language model to refer to cloze language models such as BERT Devlin et al. (2019), which do not give us a distribution over strings. have become shockingly good at modeling natural language data in recent years Merity et al. (2017); Conneau and Lample (2019); Radford et al. (2019). Thus, to test just how well neural language models capture language NLP researchers have started to look beyond standard evaluation metrics such as perplexity, endeavoring to understand which underlying attributes of human language these models are learning. To this end, a nascent literature has emerged that focuses on probing language models Belinkov and Glass (2019), i.e., determining whether models encode linguistic phenomena. For the most part, these works have been limited to analyses of sentence-level phenomenon, such as subject–verb agreement Gulordava et al. (2018) and garden path effects van Schijndel and Linzen (2018) among a myriad of other properties (Blevins et al., 2018; Chowdhury and Zamparelli, 2018, inter alia).

In this work, we attempt to understand which macro-level phenomena of human language today’s language models reflect. That is, we pose the question: Do neural language models exhibit the statistical tendencies of human language? Phenomena that can be measured at this level provide an alternate view of a model’s comprehension; for example, rather than exploring whether morphological agreement is captured, we look at whether our models learn the trends across a corpus as a whole, e.g., the token rank–frequency (Zipf’s) relationship. In comparison to standard probing techniques, this framework does not require we know a priori how linguistic phenomena should manifest themselves. That is, when there is no law stating the theoretical tendencies of an attribute of natural language or we have reason to believe our language domain does not follow such a law, we can use the statistical tendencies present in empirical data as our baseline. This characteristic both allows us to assess a model’s fit to highly corpus-dependent distributions—like the length distribution—and mitigates the biases introduced by our own preconceptions regarding properties of natural language.Such biases are naturally introduced by many probing techniques that e.g., draw conclusions from hand-constructed challenge tasks.

More concretely, our paper describes an experimental design and accompanying hypothesis tests to determine precisely whether text generated from language models follows the same empirical trends as human language. Our experiments reveal that adherence to natural language tendencies varies widely with both model architecture and generation strategy, e.g., Fig. 1 shows varying degrees of adherence to the empirical type–token relationship, an artifact that perplexity alone could not reveal. Our findings suggest this framework is a valuable tool for gaining a deeper understanding of where today’s language models are succeeding and failing at capturing human language.

Language Models

Language models are probability distributions over natural language sentences. We define the support of a language model $p_{\scaleto{{\boldsymbol{\theta}}}{4pt}}$ with parameters ${\boldsymbol{\theta}}$ as

where $\mathcal{V}$ is the model’s vocabulary and tokens eos and bos demarcate the beginning and end of a string, respectively, and $\mathcal{V}^{*}$ is the Kleene closure of $\mathcal{V}$. In this paper, we term vocabularies consisting of words closed and those consisting of BPE tokens Sennrich et al. (2016) open.

In the case when $p_{\scaleto{{\boldsymbol{\theta}}}{4pt}}$ is locally normalized, which is the predominant case for language models, $p_{\scaleto{{\boldsymbol{\theta}}}{4pt}}$ is defined as the product of probability distributions:

Statistical Tendencies of Language

Human languages are thought to exhibit statistical tendencies, several of which are explicitly quantified by laws Altmann and Gerlach (2016). In this section, we review a subset of these distributions–both with and without well-established forms—over which we subsequently perform analyses.

where $\zeta(s)=1/\sum_{k=1}^{\infty}k^{-s}$ is the normalizing constant of our probability mass function (pmf). The adherence of language to Zipf’s law has been widely studied and is considered one of the canonical laws of quantitative linguistics Baroni (2009); Li et al. (2010); Moreno-Sánchez et al. (2016).

Estimating $s$ from an observed set of rank–frequency pairs can be done using standard estimation techniques. Here we use the maximum-likelihood estimateDerivation in App. A. We may also estimate $s$ using, e.g., least squares over the original or log–log transform of our distribution. However, it has been empirically observed that least-squares estimates under this paradigm are not reliable Clauset et al. (2009) and further, directly incorporate assumptions that contradict power law behavior Schluter (2020). (MLE), employing numerical optimization to solve for $s$ since the MLE of the discrete power law lacks a closed form solution.

Type–Token.

Heaps’ law Herdan (1960), also known as the type–token relationship, states that the number of additional unique tokens (i.e., number of types) in a document diminishes as its length increases. Formally, we can express the expected number of types $u(\cdot)$ as a function of the length $l(\cdot)$ of the string $\mathbf{y}$ via the relationship $u(\mathbf{y})\propto l(\mathbf{y})^{\beta}$ where $\beta<1$ is a free parameter. Types may be, e.g., unigrams or bigrams.

The above formulation of Heaps’ law lacks an obvious probabilistic interpretation. However, if we frame Heaps’ law as modeling the expected value of the number of types for any given length document, then we can model the relation as a Poisson process, where the marginal distribution over document length follows Heaps’ proposed power law. Specifically, we model the number of types for a document of a given length as a non-homogeneous Poisson process (NHPP; Ross, 1996) where our rate parameter $\lambda(l(\mathbf{y}))$ is Heaps’ power law relation. The probability that there are $k$ types in a document of length $t$ is then

for $\lambda(l(\mathbf{y}))=\alpha\cdot l(\mathbf{y})^{\beta}$. Similarly to Eq. 4, we can fit parameters $\alpha,\beta$ using MLE (see App. A).

2 Other Tendencies

Natural language has other quantifiable distributions, e.g., over document length or unigrams. While there may not exist well-established laws for the behavior of these (often highly corpus-dependent) distributions, we can observe their empirical distributions w.r.t. a corpus. We review a few here and leave the exploration of others to future work.

Using notation from earlier, we estimate the pmf of the distribution over the length of documents in a corpus $\mathcal{C}$ as

Unigram.

Notably, the rank–frequency law of § 3.1 leaves the categorical distribution over words unspecified, i.e., it defines the frequency for the$k^{\text{th}}$ ranked word without specifying the word itself. In order to make explicit comparisons, we define the unigram distribution w.r.t. corpus $\mathcal{C}$ as

Stopwords and Symbols.

Certain percentages of words in a string consist of either symbols, i.e., numbers and punctuation, or stopwords, i.e., common words such as “that” or “so” that primarily serve a syntactic function. We can model this percentage as a (continuous) random variable $S$ and estimate its probability density function (pdf) as

Statistical Distances

In this work, we aim to quantify the degree to which the linguistic distributions of text generated from language models match—or differ from—those of natural language. To this end, we propose the use of several probability metrics Mostafaei and Kordnourie (2011); Rachev et al. (2013) as our notion of statistical distance.Some of these metrics are formally pseudo-distances, as they are not necessarily symmetric. For each of these metrics, we present nonparametric statistical significance tests, i.e.,author=clara,color=orange,size=,fancyline,caption=,]not a precise def of nonparametric tests, should we tests that may be used when the underlying distribution of observed data is not known.

Perhaps the simplest method for measuring the distance between two random variables is through differences in expectations, e.g., means or variances. (Semi-)distances of this nature are formally called primary metrics. To estimate this distance, we can use observations from random samples $\mathcal{S}_{1}$ and $\mathcal{S}_{2}$, e.g., $\mu_{1}-\mu_{2}\approx\phi(\mathcal{S}_{1},\mathcal{S}_{2})=\hat{\mu}(\mathcal{S}_{1})-\hat{\mu}(\mathcal{S}_{2})$.

Observing a value of $\phi(\mathcal{S}_{1},\mathcal{S}_{2})\neq 0$ on its own is not enough to confirm a difference between $\mu_{1}$ and $\mu_{2}$; we need to assess whether the observed distance is significantly above or below . Formally, our null and alternative hypotheses are:

In a nutshell, a permutation test provides a simple method for constructing the sampling distribution of a test statistic $\phi$ through empirical observations. The method uses the value of $\phi$ over all possible rearrangements of the observed data points to represent the distribution of the test statistic under the null hypothesis. Using this distribution, we can determine the probability of observing a value of the test statistic (or a more extreme value), which if low, may give us reason to reject a specific null hypothesis. In this work, we only consider statistics $\phi(\cdot,\cdot)$ over two samples. We provide pseudocode for this case in App. B.When the number of possible permutations of the data is computationally prohibitive, we may instead use a MC sampling approach, where we sample from the set of possible permutations Good (2000).

2 Simple Metrics

Primary metrics provide only a weak measure of the sameness of random variables as they are completely dependent on a single statistic of a distribution.author=clara,color=orange,size=,fancyline,caption=,]check or reword On the other hand, we know a random variable can be completely described by its distribution function. As such, we turn to simple metrics of distance between random variables.

Given cumulative distribution functions (cdfs) $P_{1}$ and $P_{2}$ over one-dimensional random variables, the Kolmogorov–Smirnov (KS) metric is

where $D\in$ and $D(\cdot,\cdot)=0$ indicates the distributions are identical. However, not all random variables can be described in terms of a cdf. For categorical distributions where the support of our random variable is not ordinal, the natural counterpart to the KS metric is the Chi-square distance. This metric has a number of drawbacks (discussed in App. C)—primarily that its value can be hard to interpret and so we instead turn to the total variation distance (tvd)—a widely used metric of distance between probability distributions.

Given two pmfs $p_{1}$ and $p_{2}$, we define tvd as

where similarly to the KS metric, tvd is bounded above by 1 and a value of indicates identical distributions. In our setting, we consider two use cases for the KS metric and tvd: as distance metrics between an empirical and theoretical distribution (one-sample) and between two empirical distributions (two-sample). The corresponding hypotheses that we can test with these metrics are:

O ne-Sample Case: (12) T wo-Sample Case: (13) where in the two-sample case, the exact form of $p$ does not need to be known. These hypotheses require the following tests.

The KS test Smirnov (1948) is a nonparametric goodness-of-fit test originally designed to assess the fit of a continuous cdf to empirically-observed data; the two-sample version tests whether two samples come from the same distribution. The method has since been extended to discrete distributions and is regarded as one of the most widely applicable nonparametric goodness-of-fit tests for comparing two distributions Horn (1977); Moreno-Sánchez et al. (2016). The test uses the KS metric $D$ as its test statistic; under our null hypothesis, $D$ converges to 0 almost surely in the limit as our number of samples $n\rightarrow\infty$ by the Glivenko–Cantelli theorem.Also known as the “fundamental theorem of statistics.” We may reject the null hypothesis if our test statistic is greater than the critical value, which is computed based off of our sample size and a desired significance level.Under the null hypothesis, our text statistic $D$ follows a Kolmogorov distribution. In the two sample case, the critical value is dependent on the size of both samples.

A Test for tvd.

Unlike the KS metric, we do not have a (theoretical) limiting distribution for tvd between samples from the same distribution that holds for all density functions Devroye and Győrfi (1990). However, we can construct this distribution using resampling techniques. Formally, when $\mathcal{S}_{1}$ and $\mathcal{S}_{2}$ are drawn from the same distribution $p$—where $p$ need not be known—then the test statistic tvd($p_{\scaleto{\mathcal{S}_{1}}{4pt}},p_{\scaleto{\mathcal{S}_{2}}{4pt}}$) follows the sampling distribution $\mathcal{Z}_{p}$, i.e., $\textsc{tvd}(p_{\scaleto{\mathcal{S}_{1}}{4pt}},p_{\scaleto{\mathcal{S}_{2}}{4pt}})\sim\mathcal{Z}_{p}$. The distribution of $\mathcal{Z}_{p}$ can be computed using permutations of our samples, in the same manner as defined in § 4.1.

Experiments

We use the above framework to assess the degree to which language models learn various distributions of natural language, i.e., we report metrics outlined in § 4 measured over the distributions and quantities defined in § 3. We compare samples generated from language models to a reserved test set taken from the same corpus as the model’s training data. Each set contains 1 million samples.Due to our large sample sizes, we should anticipate that our results will almost always be significant, even when effect sizes are trivially small. As such, we will almost assuredly reject our null hypotheses that model-generated samples come from the same distribution as natural language ones. While in this light, the presentation of hypothesis tests in § 4 may seem pointless, we provide them for cases where generating many samples for each model setting is computationally prohibitive. We tokenize all samples using the Moses decoder toolkit Koehn et al. (2007). All text is lower-cased and only complete unigrams are considered, i.e., when BPE is used, only the detokenized unigram is considered. Length of a string is computed as the number of tokens separated by whitespace. Note that when reporting the KS metric ($D$), we always report the metric between (a) an empirical cdfauthor=clara,color=orange,size=,fancyline,caption=,]do we need to clarify how this is done? computed over the respective model-generated samples and (b) a reference cdf, where $D_{p}$ indicates direct comparison with empirical cdf of the test set. $D_{p_{\scaleto{{\boldsymbol{\theta}}}{4pt}}}$ and $D_{\hat{p}}$ indicate comparison with cdfs of a parametric distribution, whose parameters are estimated on the model and test set, respectively.

Simulating Corpora from Language Models.

Given the distribution $p_{\scaleto{{\boldsymbol{\theta}}}{4pt}}$, we may exactly compute statistics and distributions for language models over the entire set $\mathcal{Y}$, weighting examples by the probability assigned to each string; however, doing so is infeasible due to the size of the output space and non-Markovian structure of most neural models. Rather, we turn to sampling to create a representative set $\mathcal{S}=\langle\mathbf{y}^{(1)},\dots,\mathbf{y}^{(N)}\rangle$ from $p_{\scaleto{{\boldsymbol{\theta}}}{4pt}}$. We explore three sampling schemes: ancestral random sampling (Random), nucleus sampling (Nucleus), and beam sampling (Beam).The latter two sampling designs do not result in samples drawn according to our original $p_{\scaleto{{\boldsymbol{\theta}}}{4pt}}$. As such, the schemes lead to two “new” distributions, $p_{\scaleto{{\boldsymbol{\theta}}}{4pt}}^{(n)}$ and $p_{\scaleto{{\boldsymbol{\theta}}}{4pt}}^{(b)}$, respectively.

In ancestral random sampling, $\mathbf{y}^{(i)}$ are constructed iteratively according to the distribution

where $y_{0}=\textsc{bos}$. Under the local normalization scheme of Eq. 2, sampling according to Eq. 14 is equivalent to sampling $\mathbf{y}^{(i)}$ directly from $p_{\scaleto{{\boldsymbol{\theta}}}{4pt}}$. In nucleus sampling, our distribution is truncated to the most probable items covering portion $n\in(0,1]$ of the probability mass. Formally, we now sample

where $\mathcal{V}_{n}(p)\subseteq\bar{\mathcal{V}}$ is the smallest subset such that $\sum_{y\in\mathcal{V}_{n}(p)}p(y)\geq n$ and $Z\mathrel{:\mkern-0.25mu=}\sum_{y\in\mathcal{V}_{n}(p)}p(y)$