1.1 Proposed workflow using transformer models

Our proposed workflow for entity resolution is given in Figure 1, which uses transformer models as well as traditional generative or discriminative models.

While the workflow is applicable to both the supervised and unsupervised settings, i.e. with or without a sample of labelled matches, we shall focus on the supervised setting in this paper. Transformer models are considered for embedding and classification. Notice that cross-encoder models with text as input are not relevant for scoring given numerical embeddings, whereas how to use transformer models to achieve scoring-classification from texts directly remains an open question in the unsupervised setting.

A fundamental difficulty for modelling arises from the following fact about entity resolution. While matched items refer to real entities, a pair of non-matches does not correspond to a single real entity. For instance, when two records refer to two different persons, there is no person associated with both records. It follows that there is no naturally defined population of non-matches corresponding to the population of matches, which complicates the construction of training data for supervised learning.

To deal with this difficulty, it is common to create the training and test sets in a way that mirrors how the models will later be applied to out-of-sample items. Suppose that, for each out-of-sample record in $A$, one would conduct blocking to gather $k$ candidate records in $B$ for scoring and classification. We then construct candidate records in the same manner for the sampled file-$A$ records, and partition those file-$A$ records into training and test sets together with their associated candidate records. Thus, all candidate pairs associated with a given file-$A$ record remain in the same partition. Duplication of candidate records for different records in $A$ is allowed if it is also allowed for the out-of-sample records in application.

Notice that, if none of the valid matched records is selected among the $k$ candidates, the corresponding file-$A$ record cannot be linked correctly. Hence the choice of $k$ induces a floor of missing match rate (MMR). In practice, for scoring and classification, it is natural to select the $k$ most likely link candidates for blocking, i.e. the records in $B$ with the most similar texts, and to choose $k$ such that the MMR floor is considered acceptable. Selecting the most similar candidates also provides more difficult non-matches for supervised learning than selecting non-matches randomly.

In addition to the MMR, another central performance measure is the false linkage rate (FLR). We use the FLR and MMR, following the definitions of Lee et al. (2022), for the evaluation and comparison of the different methods considered in this paper. Their formal definitions in the present setup are given in Section 1.3. Notice that both are well-defined whether or not one employs additional treatments of duplicated candidate records for scoring and classification.

The details of the steps in the proposed workflow will be studied in this paper, and the relevant methodological choices exemplified and compared.

1.2 Link classification errors by transformer models

The estimation of FLR and MMR for link classification based on transformer models requires special attention.

If one uses only a pretrained transformer model that is fixed independently of the labelled data, then, under the SRSWOR assumption stated in Section 1.3, the observed errors in the labelled file-$A$ sample may be used to estimate the FLR and MMR that would arise over the corresponding finite population of file-$A$ records.

The matter becomes more complicated when the transformer model is fine-tuned using the labelled sample. An obvious approach is to split the labelled sample into a training set for fine-tuning and a test set for estimating the errors of the resulting subsample-tuned model. This is valid if one is willing to apply that subsample-tuned model directly to the out-of-sample items. It is not sufficient, however, if the subsample training is used only to determine the hyperparameters of fine-tuning, after which the transformer model is fine-tuned again using the entire labelled sample and this final model is applied to the remaining records. The errors observed on the original test set then refer to a different fitted model from the one eventually used for entity resolution.

We shall study this issue later using the DBLP-Scholar and Abt-Buy datasets, illustrating both the statistical problem and the practical difficulties that arise. Below we briefly explain why the issue also presents difficulties for several existing approaches to error assessment.

Bayesian uncertainty propagation. In Bayesian approaches to entity resolution, one may model the key values or their disagreement patterns and generate link sets according to the posterior distribution of the matches, conditional on the observed keys; see, for example, Tancredi & Liseo (2011); Sadinle (2017); Steorts et al. (2016). It is unclear how such prior-posterior uncertainty propagation can be carried through a fine-tuned transformer model, which does not provide the likelihood function required for the corresponding Bayesian calculation. Moreover, repeatedly generating link sets while refitting or evaluating a large transformer model would be computationally expensive.

Auditing. A practical approach in the unsupervised setting of entity resolution is to draw an audit sample from the classified links in order to estimate the FLR (Christen, 2012); see also Binette et al. (2024) for a more elaborate proposal. In the supervised setting, the labelled match sample can be used to estimate both FLR and MMR when the classification model is fixed independently of that sample. The situation changes when the same labelled observations are used for fine-tuning. Auditing additional out-of-sample records could provide information about the final model, but identifying their true matches may require substantial clerical effort.

Design-based sample inference. Zhang et al. (2025, 2026) develop unbiased estimation of algorithmic prediction or classification errors under finite population sampling. In principle, this provides a way to accommodate models trained using sampled observations. In practice, however, the required calculations can be costly for transformer models. Moreover, although link classification can be viewed as a special case of classification, unbiased estimation of classification errors in this framework relies on randomised prediction or classification, whereas deterministic classification is generally preferred for record linkage.

1.3 Notations and rest of paper

Let us now introduce the basic notations related to the workflow of entity resolution, formalising the concepts discussed above and preparing for the investigations later.

Without loss of generality, let us use the standard setup of record linkage across two files. Let $A = \{a_1, \ldots, a_{n_A}\}$ and $B = \{b_1, \ldots, b_{n_B}\}$ be two files containing $n_A$ and $n_B$ records, respectively. Let entity resolution, or record linkage, be based on $\{\tau_i : i \in A \cup B\}$, where $\tau_i$ is the text associated with record $i$, which may or may not consist of structured fields, as exemplified in Table 1.

Let $M$ be the set of matches, $M \subset A \times B$, and denote a match by $(i,j) \in M$, or equivalently $y_{ij}=1$, if records $i$ and $j$ refer to the same entity. A non-match is denoted by $(i,j) \notin M$, or $y_{ij}=0$. Let $A(M)$ and $B(M)$ denote the records in $A$ and $B$, respectively, belonging to $M$.

Let $A_s\subseteq A$ denote the sampled file-$A$ records. For simplicity of exposition in this paper, we assume that $A_s$ is obtained from $A$ by simple random sampling without replacement (SRSWOR). Let

$$s_M=\{(i,j)\in M:i\in A_s\}$$

denote the labelled match relation induced by these sampled records. Let $A(s_M)$ denote the records in $A_s$ having at least one match in $s_M$, and let $B(s_M)$ denote the corresponding matched records in $B$. Thus, $A_s\setminus A(s_M)$ contains the sampled file-$A$ records that are non-matchable.

In practice, one first draws the file-$A$ records in $A_s$ and then identifies all their known matches by any means necessary. This formulation samples file-$A$ records rather than individual matched pairs. The distinction becomes relevant when a file-$A$ record has more than one valid partner in $B$.

To limit the number of comparisons for the models, let $C_i$ denote the block of candidate records for each $i \in A$ according to a given blocking scheme, where $C_i \subset B$, and $C_i \cap C_j$ may be non-empty for $i \neq j \in A$. Let $k$ be the size of $C_i$ in the case $|C_i| \equiv k$.

By applying a given model of link classification to $$\{(i,j) : i \in A_s,\; j \in C_i\},$$ denote by $\ell_i=1$ if $i \in A_s$ is linked and $\ell_i=0$ if it is not linked. One can estimate the FLR and MMR when the model is applied to all the record pairs $$\{(i,j) : i \in A,\; j \in C_i\},$$ as $$\begin{equation} \label{eq:FLR:MMR} \widehat{\mathrm{FLR}} = 1- \frac{ \sum_{i \in A(s_M)} \ell_i y_{ij_i} }{ \sum_{i \in A_s} \ell_i }, \qquad \widehat{\mathrm{MMR}} = 1- \frac{ \sum_{i \in A(s_M)} \ell_i y_{ij_i} }{ |A(s_M)| }, \end{equation} \qquad(1)$$ where $j_i$ refers to the linked record in $B$ given $\ell_i=1$, and $y_{ij_i} \equiv 0$ if $i \in A \setminus A(M)$.

For supervised learning including non-matches in addition to matches, let $(A_1,A_2)$ denote a partition of $A_s$, where

$$A_1\cup A_2=A_s, \qquad A_1\cap A_2=\emptyset.$$

We use $A_1$ for training and $A_2$ for testing. Let

$$s_r=\{(i,j)\in s_M:i\in A_r\}, \qquad r=1,2,$$

denote the labelled match relations induced by these two sets of file-$A$ records. Thus, all known matches belonging to the same file-$A$ record remain on the same side of the training-test partition.

According to the setup in Section 1.1, the corresponding candidate non-matches associated with matchable records are

$$\bigcup_{i\in A_1\cap A(s_M)} \{(i,j):j\in C_i,\ y_{ij}=0\}$$

for training and

$$\bigcup_{i\in A_2\cap A(s_M)} \{(i,j):j\in C_i,\ y_{ij}=0\}$$

for testing. For non-matchable sampled file-$A$ records, all candidate pairs are non-matches, giving

$$\bigcup_{i\in A_r\setminus A(s_M)} \{(i,j):j\in C_i\}, \qquad r=1,2.$$

Notice that the setup of supervised learning above can vary with the blocking scheme. In particular, given the fixed block size $k$, we consider two schemes.

Under Scheme-I, the retained labelled match is inserted into each candidate block, whereas under Scheme-II the candidate block is determined entirely by similarity and may therefore fail to contain a valid match. The difference between the two schemes decreases as the block size $k$ increases. Scheme-I makes more efficient use of the labelled matches for learning, while Scheme-II is more closely related to the application of the models to out-of-sample records.

In the experiments below we use Scheme-II to construct the candidate blocks. For supervised fitting, however, one known valid file-$B$ partner is retained as the positive example for each matchable file-$A$ record, whether or not that particular retained partner occurs in $C_i$. The non-match examples are drawn from $C_i$, with every known valid partner excluded from the non-match class. At evaluation, $C_i$ is not augmented in this way: a predicted link must be selected from the retrieved candidate block, and failure to retrieve any valid partner contributes to the floor MMR.

Section 2 describes the datasets and experimental setup used throughout the empirical analysis. We study the proposed workflow using pretrained transformer models in Section 3. Section 4 considers the fine-tuning of transformer models, including the related assessment of link classification errors. Section 5 provides the final remarks and discusses topics for future research and development.

2.1 Datasets and preprocessing

We use the DBLP-Scholar and Abt-Buy datasets from the Magellan repository (Konda et al., 2016). The two datasets differ considerably in both scale and the form of their textual information. DBLP-Scholar contains bibliographic records, whereas Abt-Buy contains product records collected from two online retailers. Our evaluation reformulates these benchmark datasets as record-level candidate retrieval and linkage problems, with FLR and MMR calculated over file-$A$ records and their candidate sets. These error rates are therefore not directly comparable to the pair-level precision, recall or F1 scores reported under the standard entity-matching classification protocol in, for example, DeepMatcher and Ditto (Li et al., 2020; Mudgal et al., 2018). Table 2 summarises the data used in the experiments.

For DBLP-Scholar, file $A$ is the DBLP file and file $B$ is the Scholar file. The original files contain the fields title, authors, venue and year. The gold-standard mapping contains 5,347 pairs involving 2,408 distinct DBLP records and 5,218 distinct Scholar records. A DBLP record may therefore have more than one valid Scholar partner. Of the 2,616 DBLP records, 2,408 have at least one gold-standard match.

For Abt-Buy, file $A$ is the Abt file and file $B$ is the Buy file. The original Abt file contains the product name, description and price, while the Buy file additionally contains the manufacturer. The gold-standard mapping contains 1,097 pairs. All 1,081 Abt records have at least one known match in the Buy file, and some records have more than one valid partner.

For semantic blocking, we use the title field for DBLP-Scholar and the product-name field for Abt-Buy. These fields contain no missing values in either dataset. The other common text fields remain available for the field-level comparisons used by the traditional scoring models. The complete gold-standard mappings are retained when determining whether a candidate set contains a valid match and when evaluating a predicted link.

For constructing supervised match examples, one valid file-$B$ partner is retained for each matchable file-$A$ record. Thus, there are 2,408 retained positive pairs for DBLP-Scholar and 1,081 for Abt-Buy. Importantly, the other known valid partners are not treated as non-matches. At evaluation, a predicted link is considered correct if the selected file-$B$ record is any known valid partner of the corresponding file-$A$ record.

2.2 Training and candidate construction

The experimental unit is a file-$A$ record together with its candidate file-$B$ records, rather than an individual candidate pair. This distinction is important because all the pairs associated with a given file-$A$ record must remain together when the data are divided into training and test sets.

For the main experiments, the file-$A$ records are randomly divided 50–50 into training and test sets. In DBLP-Scholar this gives 1,308 records in each half. In Abt-Buy, 540 records are assigned to training and 541 to testing. The primary results reported in Sections 3 and 4 use random seeds 42, 43 and 44 and are averaged over the three resulting splits. In Tables 5–8, numbers in parentheses are Monte Carlo standard errors, calculated as the sample standard deviation of the three seed-level error rates divided by $\sqrt{3}$. The FLR and MMR are calculated separately for each split before averaging. With only three replications, these standard errors describe observed split-to-split variation and do not support precise interpretation of small method differences. Seeds 45–51 are used later where additional Monte Carlo replications are required.

Candidate blocks are constructed according to Scheme-II in Section 1.3. First, the pretrained bi-encoder all-MiniLM-L6-v2 is used to embed the blocking text of every record in files $A$ and $B$. For each record $i\in A$, the records in $B$ are ranked by cosine similarity of their embeddings, and the $k$ most similar records form $C_i$. We consider primarily $k=5$ and $k=50$, while $k=1$ is also useful for examining the blocking floor.

The same procedure is applied independently to every file-$A$ record. Consequently, the same file-$B$ record may occur in the candidate blocks of several file-$A$ records. This mirrors how semantic blocking would be applied to records outside the labelled sample and avoids imposing an artificial one-to-one restriction at the blocking stage.

The labelled candidate pairs are then constructed for supervised learning. Each retained valid partner provides one positive example for its matchable file-$A$ record. This retained positive is supplied for supervised fitting whether or not that particular partner occurs among the retrieved top-$k$ candidates. The non-match examples are drawn from the retrieved candidate block: candidate-derived pairs that are not known valid partners are treated as negatives, while every additional gold-standard partner of the same file-$A$ record is excluded from the non-match class. Thus, one retained positive may be accompanied by several retrieved hard non-matches rather than by only one negative example. At evaluation, the candidate block is not augmented with the retained positive, so a record cannot be linked correctly when none of its valid partners has been retrieved. The resulting non-matches are therefore relatively difficult examples selected by semantic similarity rather than arbitrary pairs drawn from $A\times B$.

For DBLP-Scholar, the training and test sets also contain file-$A$ records that have no gold-standard partner. Their candidate pairs are all non-matches. This is useful because such records occur in the application of record linkage and a classification procedure should be able to leave them unlinked. For Abt-Buy, every file-$A$ record has at least one known valid partner.

For discriminative link classification, the candidate pairs belonging to the training half are further divided 80–20 into fitting and validation parts. The fitting part is used to estimate the model and the validation part to choose the link-classification threshold by maximising the F1 score. The outer test records are not involved in this choice.

2.3 Models and implementation

We consider three ways of progressing from the candidate pairs to link classification, corresponding to the workflows introduced in Figure 1.

In the traditional workflow, field-level text comparisons are obtained with the Jaro–Winkler metric. In the proposed workflow, these are replaced by cosine similarities calculated from bi-encoder embeddings. Unless otherwise stated, the bi-encoder is all-MiniLM-L6-v2 (Reimers & Gurevych, 2019). Section 3.1 explains how these comparison outcomes enter the subsequent scoring models.

Two traditional scoring models are used in the main comparison. The generative model follows the MEC approach of Lee et al. (2022), with the match and non-match densities of the continuous comparison outcomes estimated by kernel density estimation. For every MEC fit, the per-field match and non-match densities are estimated with sklearn.neighbors.KernelDensity using a fixed bandwidth of 0.05. The bandwidth is held fixed across fields, datasets, random splits and block sizes rather than reselected from the data. The discriminative model is logistic regression. A radial-kernel SVM, XGBoost and a small multilayer perceptron are considered later as a sensitivity analysis.

For direct classification from paired texts, we use the cross-encoder ms-marco-MiniLM-L-6-v2. Both transformer models are first considered in their pretrained form. Their fine-tuning is then investigated in Section 4.1, where we give the corresponding training settings and explain how the candidate blocks are treated before and after fine-tuning.

The model inputs are constructed as follows. All text fields are lower-cased and stripped of leading and trailing whitespace, with missing values represented by empty strings. For DBLP-Scholar, the fields are used in the order title, authors, venue and year. For Abt-Buy, the fields are product name followed by description; price and manufacturer are not used. For cross-encoder classification, the fields of each record are concatenated in this order with the literal separator [SEP] , and the two resulting record strings are supplied as the paired input to the cross-encoder. The tokenizer then applies the configured maximum sequence length of 128 tokens. For bi-encoder scoring, by contrast, each field is encoded separately with all-MiniLM-L6-v2, and the comparison vector for a candidate pair consists of the fieldwise cosine similarities in the same order. Semantic blocking uses only the title embedding for DBLP-Scholar and the product-name embedding for Abt-Buy. During bi-encoder fine-tuning, each training record is serialised using the same full-record field order and [SEP] separator; after fine-tuning, however, all records are re-embedded field by field for blocking and cosine scoring.

For discriminative models, including the cross-encoder, a validation subset of the training pairs is used to choose the threshold for declaring a link by maximising the F1 score. The particular classification rules are described together with the corresponding models below, since they form part of the methodological comparison rather than merely an implementation detail.

The objective of these experiments is not to identify the best possible model or hyperparameter configuration for either benchmark dataset. Rather, the common setup enables us to examine how embedding, scoring, classification and fine-tuning can be combined within the proposed workflow. Accordingly, the reported error rates are intended to illustrate the behaviour of this workflow under a common experimental setup, rather than to establish state-of-the-art entity-matching performance or a benchmark ranking against published systems such as DeepMatcher (Mudgal et al., 2018), Ditto (Li et al., 2020), or the method of Peeters & Bizer (2021).

2.4 Evaluation

The unit of link classification is again a file-$A$ record. Given the candidate block $C_i$, a scoring model ranks or scores its candidate file-$B$ records. If a link is declared, let $j_i$ denote the selected record. For models that use a classification threshold, no link is declared when the best candidate does not satisfy that threshold. The generative MEC calculations considered below instead select the highest-ranked candidate according to the estimated density ratio.

A declared link is counted as correct whenever $(i,j_i)$ is one of the known gold-standard pairs for record $i$. This is important for both datasets because the gold-standard relation need not be one-to-one. Selecting a valid partner other than the particular partner retained for supervised fitting is therefore still a correct link.

The two error measures are the FLR and MMR defined in Equation (1). It is useful to state their empirical calculation explicitly. Let $D$ be the number of links declared among the test file-$A$ records, let $C$ be the number of those declarations that select a valid gold-standard partner, and let $M$ be the number of matchable file-$A$ records in the test set. Then

$$\widehat{\mathrm{FLR}} = 1-\frac{C}{D}, \qquad \widehat{\mathrm{MMR}} = 1-\frac{C}{M}.$$

Thus, an incorrect declared link contributes to both error measures: it is a false link among the declarations and, for a matchable file-$A$ record, it also means that the true match has not been recovered. An undeclared matchable record contributes to the MMR but not to the numerator or denominator of the FLR. For DBLP-Scholar, a declared link for a non-matchable file-$A$ record is necessarily a false link and therefore contributes to the FLR.

Blocking imposes an additional lower bound on the MMR. If none of the valid partners of a matchable file-$A$ record occurs in $C_i$, no scoring or classification model operating on that block can link the record correctly. We refer to the proportion of such records as the floor MMR and examine it explicitly in Section 3.1.

Unless otherwise stated, the tables in Sections 3 and 4 report the mean FLR and MMR over the three primary random splits. In the later investigation of training-set fractions and subsample error assessment, seeds 42–51 provide ten Monte Carlo replications. There we additionally report Monte Carlo standard errors, and, where relevant, relative Monte Carlo standard errors, in order to distinguish variation across random splits from the differences between modelling procedures.

The replication counts were not selected to attain a prespecified hypothesis-testing power or precision target. They reflect the computational cost of repeated fine-tuning and the descriptive purpose of the experiments. Accordingly, the three-split comparisons in Tables 5–8 are used to identify broad patterns rather than to resolve small numerical differences; their Monte Carlo standard errors are reported to show the scale of split-to-split variation. The ten-replication studies in Section 4.3 provide a more precise assessment of training-fraction sensitivity and subsample error estimation, but they are likewise interpreted cautiously when the Monte Carlo standard error is of the same order as the observed difference or bias.

3.1 Embedding and scoring

For scoring by generative or discriminative models, one needs numerical representation of key-value comparisons, denoted by $\gamma_{h,ij}$ for each field $h$ of $\tau_i$ and $\tau_j$, where $h=1,\ldots,H$. The Jaro–Winkler metric is a possibility for the traditional workflow, whereas one can use the cosine similarity measure of bi-encoder embeddings for the proposed workflow.

By the MEC approach using generative models, one would obtain an estimated density ratio for the record pair $(i,j)$, which is given as

$$\begin{equation} \label{eq:density-ratio} r_{ij} = \prod_{h=1}^{H} \frac{ f(\gamma_{h,ij}\mid y_{ij}=1) }{ f(\gamma_{h,ij}\mid y_{ij}=0) }. \end{equation} \qquad(3)$$

The product form in Equation (3) assumes that the field-level comparison outcomes are conditionally independent given the match status $y_{ij}$. This assumption may be violated for related bibliographic fields, such as title, authors and venue. In that case, the product of the field-specific density ratios need not equal the likelihood ratio based on their joint distribution.

Notice that discrete comparison outcomes, i.e. agreement or not, are common for record linkage traditionally, in which case one would replace the density ratio by a probability ratio. Given continuous comparison outcomes $\gamma_{h,ij}$, we simply apply kernel density estimation (Silverman, 1986) to estimate the two densities in Equation (3), separately, in the training set of matches and non-matches that are constructed as described in Section 2.2.

Meanwhile, discriminative models target the probability of match given the key-value comparison outcomes, denoted by

$$\begin{equation} \label{eq:match-probability} p_{ij} = \Pr\left( y_{ij}=1 \mid \gamma_{1,ij},\ldots,\gamma_{H,ij} \right). \end{equation} \qquad(4)$$

We use logistic regression as a basic discriminative model. Other models are possible, such as support vector machine (SVM), gradient-boosted trees (XGBoost) and multilayer perceptron (MLP) neural network. A given discriminative model can be estimated from the training set of matches and non-matches as a single dataset.

Using the labelled records in the DBLP-Scholar and Abt-Buy datasets, we consider blocking scheme-II (Section 1.3) with fixed size $k$ for each record in $A(s_M)$. This induces a floor MMR, since a file-$A$ record cannot be linked correctly when none of its valid matched records is included in the candidate block.

Table 4 shows the floor MMR for the two datasets, where the candidates for each record in $A(s_M)$ are selected from $B$ based on the bi-encoder cosine similarity. At $k=1$, it is 0.022 for the former and 0.278 for the latter, which means that about 97.8% of the matchable file-$A$ records have a valid match as the top candidate in the DBLP-Scholar dataset, whereas about 72.2% do so in the Abt-Buy dataset. This indicates that entity resolution is more difficult in the latter case.

Meanwhile, both the floor MMRs become practically acceptable once $k$ is increased to 50, especially because one would have to set $k=4535$ to remove the floor (i.e. making the floor MMR zero) for the DBLP-Scholar dataset, or $k=748$ for the Abt-Buy dataset. Because blocking uses only the title field for DBLP-Scholar and the product-name field for Abt-Buy, these floor MMRs also reflect that single-field retrieval design. Incorporating additional fields, such as the product description in Abt-Buy, could potentially lower the blocking floor, but multi-field blocking is not evaluated in the present study.

3.2 Comparison of embedding and scoring methods

The comparison outcomes $\gamma_{h,ij}$ are calculated for the candidate pairs constructed as described in Section 2.2, either as the Jaro–Winkler metric or the cosine similarity of bi-encoder embeddings. Given $\{\gamma_{h,ij}\}$, we compare the resulting link classification errors under generative and discriminative scoring models. The FLR and MMR are calculated on the held-out file-$A$ records according to Section 2.4.

The distinction between the two scoring approaches is worth explaining. For MEC under the generative model, we link each file-$A$ record $i$ to the candidate $j\in C_i$ with the highest density ratio $r_{ij}$ in Equation (3). Thus, the best available candidate is always selected. For a discriminative model one could similarly select the candidate with the highest match probability $p_{ij}$ in Equation (4). However, this would force a link even when all the candidate probabilities are small.

We therefore consider a threshold-based classification rule for the discriminative models. As described in Section 2.2, a validation subset of the training pairs is used to choose the threshold that maximises the F1 score. A link is declared only when the highest-scoring candidate exceeds this threshold; otherwise the file-$A$ record is left unlinked. This provides a useful contrast with the generative procedure because it allows the model to abstain when the candidate evidence is weak. We do not investigate alternative cross-validation schemes or decision criteria here, since our purpose is to illustrate the role of discriminative scoring-classification within the proposed workflow rather than to optimise the procedure for these two datasets.

Table 5 reports the test set FLR and MMR for four combinations of embedding and scoring methods, given two block sizes $k$. The results are averaged over three random 50–50 training-test splits, with Monte Carlo standard errors in parentheses. Comparisons of close means should be read in light of this split-to-split variation.

Regarding bi-encoder embeddings instead of the traditional character-string comparisons, the effects are not obvious for the DBLP-Scholar dataset where the texts are less messy (Table 1). The bi-encoder yields lower FLRs in all four comparisons, but does not consistently reduce the MMR. In contrast, the improvements are clearly visible for the Abt-Buy dataset with more messy texts. Both the FLR and MMR are reduced by the bi-encoder representation for either scoring model and either block size.

These results illustrate the potential advantages of transformer-based embeddings for text-based entity resolution. Notice that the performance of the bi-encoder can be further changed by fine-tuning, which will be considered in Section 4.

Regarding scoring-classification by generative or discriminative models, the results here show that the discriminative logistic regression model yields lower FLRs than the generative kernel density model, whereas the latter yields lower MMRs. The distinction is especially clear for the Abt-Buy dataset. This reflects the different link-classification rules used here: we always link the top-ranked candidate under the generative model, whereas a validation split is used to determine a threshold for the discriminative model, which may abstain from declaring a link. Accordingly, the FLR/MMR contrast in Table 5 should not be interpreted as an intrinsic difference between generative and discriminative model families; it also reflects the different decision rules applied to their scores.

Without advocating for one modelling approach against the other generally, we note that scoring-classification by discriminative models is worth considering in practice, although such models have been less common in record linkage traditionally.

For completeness, we also apply the pretrained cross-encoder directly to the candidate pairs obtained from bi-encoder blocking. Since the cross-encoder operates on paired record texts rather than on the numerical comparison outcomes used by the traditional scoring models, we report its results together with those of the fine-tuned cross-encoder in Table 7. This allows the effect of task-specific fine-tuning to be assessed separately from that of using the cross-encoder architecture itself.

Regarding the proposed workflow, using the bi-encoder model for embedding and the traditional generative or discriminative models for automated scoring-classification can reduce the number of candidate pairs that need to be processed by a cross-encoder, compared with applying the cross-encoder to every candidate block. This provides the basis for examining the accuracy-cost tradeoff after fine-tuning, compared with either the traditional workflow without transformer models or the transformer-only workflow.

3.3 Sensitivity analysis

Here we conduct sensitivity analysis of a methodological choice not covered above, which exemplifies the relevant considerations in practice and helps to prepare the ground for fine-tuning transformer models later.

Instead of logistic regression, one can use many other discriminative models. Table 6 compares it to a radial-kernel SVM, an XGBoost and a small MLP net, all based on the same bi-encoder embeddings. The SVM is fitted with an RBF kernel and probability estimation enabled; XGBoost uses 200 trees, maximum depth 4, learning rate 0.1 and log-loss as the evaluation metric; and the MLP has hidden layers of sizes 64 and 32 and a maximum of 300 iterations. The random-split seed is passed to each classifier, and otherwise the defaults of the pinned library versions are used. The FLR and MMR are obtained in the same way as above, and the results of logistic regression are reproduced from Table 5 for easy comparison. The models exhibit different tradeoffs between the two error measures. For DBLP-Scholar, logistic regression has low FLR in both block-size configurations, while several models have lower observed MMR. The FLR estimates for logistic regression and SVM at $k=5$ are close relative to their Monte Carlo standard errors. For Abt-Buy, the FLR and MMR comparisons vary with the model and block size, with appreciable split-to-split variation in several entries. For XGBoost on Abt-Buy, the validation-selected decision threshold lies near 0.45 on the 0.05 grid, so small numerical differences in the embeddings, such as those between CPU and GPU computation, can move it by one grid point and shift the FLR and MMR in opposite directions by more than 0.01. Logistic regression provides a simple discriminative model and will be used as the default for the investigation later.

4.1 Fine-tuning transformer models

Using the training-test construction described in Section 2.2, we now consider task-specific fine-tuning of the bi-encoder and cross-encoder models. The same random splits and treatment of multiple valid partners are retained, so that the results can be compared directly with those obtained using the pretrained models.

There is one important distinction between the two fine-tuning experiments. After fine-tuning the bi-encoder, the records are re-embedded and the $k=50$ candidate blocks are reconstructed from the fine-tuned embeddings before the traditional scoring models are evaluated. For the cross-encoder, the candidate blocks obtained from the pretrained bi-encoder are held fixed. Thus, the cross-encoder comparison isolates the effect of fine-tuning the classifier rather than changing the candidate set at the same time.

The hyperparameters for learning transformer models involve at least four choices: the warmup step, typically set to around 10%; the learning rate, typically of the order $10^{-5}$; the number of epochs, typically small; and the batch size, typically 16 or 32. While some experimentation is needed to settle the hyperparameters in practice, it is rare to apply cross-validation inside the training set formally due to the computation cost this would require. Notice that the computation cost differs for bi-encoder and cross-encoder. Once the embeddings from a bi-encoder have been obtained, similarity calculation and subsequent scoring are computationally cheap. In contrast, the cross-encoder must process each pair of texts directly whenever it is applied.

The bi-encoder model all-MiniLM-L6-v2 is trained for one epoch with ContrastiveLoss, using 10% linear warmup, batch size 16 and learning rate $2\times10^{-5}$. The contrastive loss encourages the embeddings of matched record pairs to be closer and those of non-matched pairs to be further apart. The fine-tuned bi-encoder model can then be applied to the records in $A\cup B$ to yield the fine-tuned embeddings, from which the per-field cosine similarities are calculated as before.

The cross-encoder ms-marco-MiniLM-L-6-v2 is trained for five epochs with 10% linear warmup, batch size 16, learning rate $2\times10^{-5}$, and a maximum sequence length of 128 tokens. The training pairs are labelled as matches or non-matches for binary link classification.

For comparison, we also evaluate the pretrained cross-encoder without updating any of its model parameters. The same 50–50 outer split and 80–20 pair-level validation split are used, but the training portion is not used to fit the cross-encoder. It is retained only so that the link-classification threshold can be selected on exactly the same validation partition as for the fine-tuned model. In both cases, the threshold is selected from $0.10,0.15,\ldots,0.90$ by maximising the validation F1 score, and the direct output of CrossEncoder.predict() is used without an additional sigmoid transformation.

While the hyperparameters above may not be optimal for these models, they suffice for the methodological investigations below, which is our purpose in this paper rather than to find the best procedures for the two datasets at hand.

The bi-encoder and cross-encoder are fine-tuned for one and five epochs, respectively. Thus, the fine-tuned comparisons in Table 7 concern the specified training configurations, rather than an epoch-matched comparison of model architectures. The contribution of training duration to the observed differences is not assessed separately.

Table 7 compares the link classification performance with or without fine-tuning of the transformer models. The results for the traditional models with pretrained bi-encoder embeddings are reproduced from Table 5 for easy comparison. All results are averaged over three random 50–50 splits of the training and test sets.

The cross-encoder results show a clear improvement from fine-tuning. For DBLP-Scholar, the pretrained cross-encoder has an FLR of 0.061 and an MMR of 0.026, compared with 0.018 and 0.014 after fine-tuning. For Abt-Buy, the corresponding error rates fall from 0.267 and 0.278 to 0.041 and 0.152. Fine-tuning therefore reduces both error rates for the cross-encoder in both datasets.

Compared with the traditional models based on the fine-tuned bi-encoder, the cross-encoder has a lower observed FLR than MEC in both datasets. For DBLP-Scholar, the mean MMRs for MEC and the cross-encoder are 0.011 and 0.014, respectively; this small difference should be interpreted cautiously given the Monte Carlo standard errors. For Abt-Buy, their mean MMRs are likewise close (0.145 and 0.152), while their FLRs differ more substantially (0.124 and 0.041). Logistic regression has higher observed MMR in both datasets, while its FLR is closer to that of the cross-encoder for Abt-Buy.

We also examined the sensitivity of the fine-tuned cross-encoder to the block size. For Abt-Buy, reducing the block size from $k=50$ to $k=5$ increases both the FLR and MMR, from 0.041 and 0.152 to 0.056 and 0.219, respectively. For DBLP-Scholar, the effect is less uniform: the FLR increases from 0.018 to 0.022, whereas the MMR changes only slightly, from 0.014 to 0.013.

Changing $k$ affects more than the number of non-match examples available during fine-tuning. It also changes the candidate set available at evaluation and hence the probability that at least one valid partner is retrieved. This is particularly relevant for Abt-Buy, where the floor MMR is appreciably higher at $k=5$ than at $k=50$. The comparison should therefore be interpreted as sensitivity to the block-size configuration as a whole, rather than as an isolated effect of the number of training non-matches.

This points to another aspect of the cost-accuracy tradeoff for cross-encoder models. Larger candidate blocks provide more retrieved hard non-matches for supervised learning and can increase the chance that a valid partner is available for classification, but they also increase the number of record pairs to which the cross-encoder must be applied. The appropriate block size therefore depends on candidate recall, linkage error and computational cost together. Notice also that the test-set performance of a fine-tuned cross-encoder would not necessarily carry over unchanged if a different blocking procedure were employed for out-of-sample records.

Meanwhile, the workflow using the fine-tuned bi-encoder produces a different pattern across the two datasets. Since the candidate blocks are reconstructed after fine-tuning, this comparison reflects both the changed embedding representation and the candidate blocks obtained from it. For DBLP-Scholar, the workflow using the fine-tuned bi-encoder yields lower MMRs for both the generative and discriminative models, but higher FLRs. For Abt-Buy, it yields considerably lower values of both error rates for both traditional models. For the discriminative logistic regression model, the observed Abt-Buy FLR is 0.053, compared with 0.041 for the fine-tuned cross-encoder. This motivates examining whether traditional-model scoring can handle some cases without cross-encoder evaluation, which we investigate next.

4.2 Combining models for scoring-classification

Since bi-encoder blocking does not distinguish different blocks of text pairs, which are all sent to the cross-encoder model for link classification, a natural idea is to use the traditional models to separate the harder and easier cases, each composed of $(i,C_i)$, so that only the harder cases are passed on to the costly cross-encoder link classification, while the easier cases can be classified by the traditional models directly.

Given a traditional discriminative model for scoring in Equation (4), trained on the fine-tuned bi-encoder embeddings, define a corresponding discrimination gap for each case $(i,C_i)$, which is given as

$$\begin{equation} \label{eq:discrimination-gap} \Delta_i = p_{(i1)} - p_{(i2)}, \end{equation} \qquad(5)$$

where $p_{(i1)} \geq p_{(i2)} \geq \cdots \geq p_{(ik)}$ are the order statistics of $\{p_{ij}:j\in C_i\}$. A smaller gap (5) for a file-$A$ record suggests lower confidence by the traditional model to separate its best file-$B$ candidate from the runner-up, in which sense link classification of this case is harder than another case with a larger discrimination gap.

For the comparison below, the $k=50$ candidate blocks are held fixed at those obtained from the pretrained bi-encoder, so that the logistic regression and cross-encoder are evaluated on the same candidate records. The logistic regression scores are recalculated using the fine-tuned bi-encoder embeddings. The quartiles are then formed separately within each test set according to $\Delta_i$. These test-set quartiles are used only as an illustrative analysis of how linkage errors vary with case difficulty; they do not define a routing threshold for deployment. In an operational combined procedure, the routing rule, including any discrimination-gap threshold, would be selected using the training or validation partition and then applied unchanged to test or out-of-sample records.

Table 8 shows the FLR and MMR by the quartiles of $\Delta_i$, where Q1 refers to the smallest 25% discrimination gaps and the hardest cases accordingly, and Q4 refers to the 25% easiest cases. The FLR and MMR of the fine-tuned cross-encoder model are compared to those of logistic regression based on the fine-tuned bi-encoder embeddings.

The results support the discrimination gap as a useful measure for separating harder cases from easier ones, although the error rates do not decrease monotonically across every quartile. The distinction is particularly clear for Abt-Buy. In Q1, logistic regression has FLR 0.230 and MMR 0.706, compared with 0.121 and 0.400, respectively, for the cross-encoder. The advantage of the cross-encoder remains substantial in Q2. In Q3, however, both models have very low FLRs and logistic regression has the lower MMR. In Q4, logistic regression makes no errors, while the cross-encoder has only a very small MMR.

For DBLP-Scholar, neither model makes an observed error in Q4 across these splits. In Q1–Q3, the cross-encoder has lower mean MMR, while the small FLR differences in Q1–Q3 should not be treated as stable model orderings given the Monte Carlo standard errors. The results are consistent with concentrating cross-encoder evaluation among cases with smaller discrimination gaps, while leaving many of the easier cases to the traditional model.

This shows that, (i) the discrimination gap is a reasonable measure for separating the harder cases from the easier ones, and (ii) it can be more beneficial to apply the cross-encoder model to the harder cases in order to improve accuracy subject to a cost constraint, provided the traditional models can do a relatively good job for the easier cases. The Abt-Buy results illustrate this particularly clearly: the largest gains from the cross-encoder occur among the cases with the smallest discrimination gaps, whereas there is little benefit from sending the easiest cases to the more costly model.

Figure 2 shows the proposed workflow of entity resolution with the additional details of how the traditional scoring models can be combined with the transformer model. In particular, the workflow provides a way to control the accuracy-cost tradeoff by using the traditional generative or discriminative models to identify the hard cases for cross-encoder link classification, while leaving the rest of the link classification to the traditional model.

The potential computational saving arises because fine-tuning is performed once for a given labelled sample, whereas the subsequent cost of cross-encoder classification increases with the number of text pairs sent to it. Once the bi-encoder embeddings have been obtained, scoring by the traditional models is comparatively inexpensive. Routing only a subset of cases to the cross-encoder therefore reduces the number of pairwise cross-encoder evaluations. As a hardware-independent cost measure, Table 11 reports the realised number of cross-encoder pair evaluations for the two training-derived routing rules relative to sending all $k=50$ candidate pairs to the cross-encoder. The realised runtime saving will depend on the routing rule, block size, hardware and implementation, and is not benchmarked systematically in the present paper.

Although the benefits of creating separate case streams for link classification and the feasibility of using the traditional models for this task are clear by now, how to determine exactly which cases should be sent to the cross-encoder model must depend on the cost constraint and the acceptable accuracy in each application. It is also possible to consider more refined schemes than, say, choosing a single threshold of the discrimination gap.

For instance, by MEC using a generative model, one can estimate both the FLR and MMR for the classified link set, and use them to filter out the cases for the cross-encoder model. Or, to further reduce the cost, one may consider sending fewer candidates for each case passed on to the cross-encoder model, provided the left-out candidates have much lower match probabilities than the selected ones according to the scoring model, in which respect one may consider more refined discrimination measures than the gap (5) between the top two candidate match probabilities.

However, we notice that there is another issue related to the exact scheme of creating separate case streams, which concerns the assessment of linkage errors by transformer models, as we shall discuss in the next section. To demonstrate the combined procedure directly, Section 4.3.3 also reports pooled FLR and MMR under two routing rules whose discrimination-gap cutoffs are determined from the training records and then applied unchanged to the test records.

4.3 Linkage error assessment of transformer models

Assessing the link classification errors of the cross-encoder model faces two major issues. As we have explained in Section 1.2, the test errors only refer to the model fine-tuned on the training set, which causes the first issue if one would like to fine-tune the model on the whole labelled match sample using the hyperparameters settled on the training set, which is a very common practice in Machine Learning (e.g. Hastie et al., 2009). The second issue is related to the creation of separate case streams for link classification in the proposed workflow, which means, for example, that the cross-encoder model would only be applied to the selected harder cases instead of all the out-of-sample cases.

Below we investigate the first issue, before proposing a workable approach to linkage error assessment, even though it does not yield unbiased estimation of the FLR and MMR.