Multivariate goodness-of-fit and nonparametric two-sample testing are fundamental statistical problems. In the goodness-of-fit problem, observations are compared with a specified probability model; in the two-sample problem, two independent samples are compared without specifying their common distribution under the null hypothesis. Both problems arise in many scientific settings and have generated a large literature of procedures based on empirical distribution functions, nearest neighbors, graphs, kernels, energy-type distances, characteristic functions, and multivariate depth; see, for example, Thas (2010) and Rayner et al. (2009).

The practical difficulty is not a shortage of tests but the absence of a uniformly best procedure. Power can change substantially with the form of the alternative, dimension, sample size, and whether nuisance parameters are estimated. Consequently, an applied analyst may wish to compare several procedures, while a methodological researcher proposing a new statistic must evaluate it against established competitors over many alternatives. When implementations are distributed among packages with different interfaces, data conventions, output formats, and resampling schemes, such comparisons require substantial additional programming and can be difficult to reproduce.

The MDgof and MD2sample packages (Rolke, 2026b, 2026a) address this computational problem through a common framework for multivariate goodness-of-fit and two-sample testing. The principal contribution is a common computational framework in which existing and newly proposed statistics can be implemented, calibrated, compared, and benchmarked under common conditions. The framework supports continuous and discrete data, simple and composite null hypotheses, hybrid goodness-of-fit problems based on simulated reference samples, simultaneous inference, Monte Carlo power studies, and systematic benchmarking. The computational contribution is the integration of these tasks in a single extensible workflow, so that calibration and comparative simulation need not be reimplemented separately for each statistic.

Four design features are central. First, built-in procedures are exposed through common testing and power interfaces rather than unrelated method-specific functions. Second, user-defined statistics can be inserted into the same resampling and benchmarking machinery used by the built-in procedures. Third, the packages provide collections of simulation scenarios so that competing procedures can be evaluated under identical alternatives and sample sizes. Fourth, MDgof and MD2sample deliberately use parallel names for analogous user-facing operations. This is an interface-design choice: a user who has learned the workflow for one testing problem can transfer that knowledge to the other without learning a second vocabulary. Simultaneous attachment of both packages is not required; namespace-qualified calls such as MDgof::gof_test() and MD2sample::twosample_test() remain unambiguous.

Computationally intensive components use Rcpp (Eddelbuettel et al., 2024) and parallel processing where appropriate. The packages also give explicit attention to multivariate discrete data, for which general-purpose software is less extensive than for continuous data. This makes it possible to use the same overall computational strategy for continuous observations, discrete count representations, and simulation-based hybrid problems.

The present work builds on the computational approach developed for the univariate goodness-of-fit and two-sample problems in the R packages Rgof and R2sample (Rolke, 2025). MDgof and MD2sample extend this approach to multivariate problems, for which the available test statistics, data representations, and computational requirements are substantially different. The four packages nevertheless share a similar workflow for testing, power estimation, user-defined procedures, and comparative simulation.

The remainder of the paper is organized around the computational framework. Section 2 defines the statistical settings handled by the software. Section 3 summarizes the implemented procedures and their relation to existing software. Sections 4–7 describe simultaneous inference, the common testing interface, extensibility, power estimation, and benchmarking. Section 8 discusses computational performance, Section 9 gives representative applications, and the final section summarizes the role of the framework in methodological development and comparative simulation.

For multivariate goodness-of-fit, let 𝑿1,…,𝑿n be observations from an unknown distribution F. The null hypothesis is H0:F=F0, where F0 may be completely specified or may contain parameters estimated from the data. For multivariate two-sample testing, let 𝑿1,…,𝑿n∼F and 𝒀1,…,𝒀m∼G be independent samples. The null hypothesis is H0:F=G. The packages support both continuous multivariate observations and discrete data represented through support points and their observed counts.

A useful intermediate case is a goodness-of-fit problem in which evaluation of F0 is difficult or unavailable but simulation from the model is feasible. A Monte Carlo sample from the fitted or specified null model can then be compared with the observed sample using a two-sample statistic. We refer to this as the hybrid problem. It is particularly useful for models in which generation is straightforward but high-dimensional distribution-function or likelihood calculations are computationally prohibitive.

When parameters of F0 are estimated from the observed data, calibration must reproduce the estimation step. A permutation calibration that treats the fitted distribution as fixed can be inappropriate. The framework therefore allows the user to provide an estimator and a null-data generator, yielding a parametric-bootstrap calibration in which parameters are re-estimated as required. This same separation between the test statistic and the data-generating mechanism is important for the power and benchmarking routines described below.

From a computational perspective, the three settings share the same basic pipeline: data→test statistic(s)→null calibration→p-value(s)→power and benchmarking. The software is organized around this pipeline. Differences among goodness-of-fit, two-sample, continuous, discrete, and hybrid problems are handled primarily through the data representation and null-generation mechanism rather than through unrelated user workflows.

The framework contains a broad collection of classical and recent multivariate procedures. Tables 1 and 2 summarize the methods currently available for goodness-of-fit and two-sample analysis, respectively. The purpose of including many methods is not to claim that one family dominates the others, but to make procedures based on different principles available under a common computational interface.

The implemented statistics span several methodological families. Empirical-distribution-function procedures include multivariate versions or approximations to Kolmogorov–Smirnov, Kuiper, Cramér–von Mises, and Anderson–Darling statistics (Anderson, 1962; Anderson & Darling, 1952; Cramér, 1928; Kolmogorov, 1933; Kuiper, 1960; Mises, 1928; Pettitt, 1976; Smirnov, 1939). For continuous multivariate goodness-of-fit, the empirical-distribution-function statistics are evaluated at the observed data points; this avoids the nd evaluations required by a full search over coordinate combinations, but it should be regarded as a computational approximation rather than the exact multivariate extension.

Other goodness-of-fit procedures use nearest-neighbor distances, density estimation, Rosenblatt-type transformations, spatial statistics, or discrepancies between observed and expected discrete counts (Bakshaev & Rudzkis, 2015; Bickel & Breiman, 1983; Fasano & Franceschini, 1987; Ripley, 1976). The two-sample collection includes nearest-neighbor, distance-based, kernel, graph, and Ball-divergence procedures, including methods of Aslan & Zech (2005), Baringhaus & Franz (2004), Biswas & Ghosh (2014), Gretton et al. (2012), Friedman & Rafsky (1979), Chen & Friedman (2017), and Pan et al. (2018). These established statistics serve both as practical testing procedures and as reference competitors for newly proposed methods.

Because no single multivariate test has uniformly high power, an analyst may apply several procedures and use the smallest individual p value as evidence against the common null hypothesis. The unadjusted minimum is not itself a valid level- α test because the individual statistics are dependent and the probability of at least one small p value exceeds α.

The packages therefore calibrate the minimum p value by resampling under the joint null hypothesis. If Pmin denotes the minimum of the individual p values and H0 is true, let Fmin be its null distribution function. The combined adjusted value is padj=Fmin(pmin⁡,obs). Because Fmin is generally unavailable analytically, it is estimated from resampled data generated under the same null model. The computational algorithm is:

For an example, see (Rolke, 2025). For a general discussion of p value adjustments based on resampling see (Westfall & Young, 1993).

MDgof::gof_test_adjusted_pvalue() and MD2sample::twosample_test_adjusted_pvalue() compute these adjusted p values. Both functions have the default argument B = c(5000, 1000). The first component specifies the number of simulation replications used to estimate the individual p values, and the second specifies the number used to estimate the null distribution function of their minimum. The two stages are performed serially rather than by nesting the full individual calibration inside every second-stage replication; the computational cost is therefore associated with the two successive stages rather than their product. Both provide a doMethods argument for specifying the tests to combine; doMethods="all" includes every method available for the given data type and dimension. The default selections are smaller subsets. For example, for discrete data MD2sample::twosample_test_adjusted_pvalue() combines Chi-square, Kolmogorov–Smirnov, Aslan–Zech, and Cramér–von Mises by default, whereas MD2sample::twosample_test() reports eight tests. For four-dimensional continuous data, the default MDgof::gof_test() selection reports qK, qCvM, and qAD. Specifying doMethods="all" in both testing routines also includes qKS and BR, allowing the same five methods to be compared. Method availability depends on data type and dimension; in particular, the chi-square implementations are limited to two dimensions. See Section 5 for the testing interfaces. For an example using this adjustment in simultaneous confidence bands, see (Aldor-Noiman et al., 2013).

The public interfaces separate three components that are often intertwined in method-specific implementations: the observed data, the statistic to be evaluated, and the mechanism used to generate or resample data under the null hypothesis. This separation is what allows the same computational machinery to serve built-in and user-defined statistics.

The two packages intentionally expose analogous workflows with parallel naming conventions. This reduces the learning cost for users moving between goodness-of-fit and two-sample problems. The packages need not be attached simultaneously; examples in this paper use namespace-qualified calls when the package identity is useful.

A user-defined statistic is supplied through TS; auxiliary objects or functions required by that statistic can be passed through TSextra. Once supplied, the statistic can participate in testing, power estimation, and benchmarking without requiring the user to implement separate Monte Carlo loops. This is the principal extensibility mechanism of the framework.

The illustrative power calculations in this section use B=250 Monte Carlo replications to keep computation times reasonable. For an estimated rejection probability π̂, the Monte Carlo standard error is approximately π̂(1−π̂)/B. With B=250, this is at most approximately 0.032 and is approximately 0.014 when the rejection probability is 0.05. The tabulated values illustrate the software and should not be interpreted as precise power estimates; larger simulation sizes are appropriate for substantive power comparisons.

A central purpose of the framework is to compare the power of competing methods, including methods supplied by the user, under common Monte Carlo conditions. In both packages this functionality is provided by run.studies(). By fixing the data-generating scenarios, sample sizes, type I error level, and simulation size across procedures, the framework removes a substantial source of incidental variation from comparative studies.

Both packages include a large number of case studies, that is, distributions for the null hypothesis and for an alternative hypothesis. They can be found in the routine case.studies(). The two packages provide 140 distinct case names, corresponding to 250 study configurations when continuous and discrete versions, parameter estimation, and five-dimensional cases are counted separately. Both packages include vignettes with detailed descriptions of the studies. For a listing one can run

The names are chosen to give some idea of the type of study. For example, in the first study the null hypothesis specifies independent uniform data, where the actual data is uniform with some data on the main diagonal in such a way that the marginals are still all uniforms. Many of the studies are based on copulas using the R package copula.

Let’s say we want to compare the power of the included test MDgof::newTS() with those included in the package for the case of continuous data without parameter estimation. We can run

res contains the detailed power estimates. The summary reports the average rank of each method across the selected scenarios, with larger values indicating better average performance.

Arguments common to MDgof::run.studies() and MD2sample::run.studies() are:

Arguments specific to MDgof::run.studies():

As an example, the package MDgof includes the routine newTS(). This runs a chi-square test with a 3 by 3 grid of bins, either equal space or equal probability. To see how these tests perform for continuous data without parameter estimation, we can run

The run.studies() routines can be used to

The method labels in the output abbreviate the procedures in Tables 1 and 2: KS (Kolmogorov–Smirnov), K (Kuiper), CvM (Cramér–von Mises), AD (Anderson–Darling), NN (nearest-neighbor), AZ (Aslan–Zech), BF (Baringhaus–Franz), BG (Biswas–Ghosh), MMD (maximum mean discrepancy), FR (Friedman–Rafsky), CF (Chen–Friedman), and BR (Bakshaev–Rudzkis). The q-prefixed labels identify goodness-of-fit EDF statistics; additional labels are defined by the respective package documentation.

All computations in this article were performed on an Intel Core i9-14900F system with 32 GB RAM running Windows 11 Home (build 26200), using R version 4.6.0 (2026-04-24 ucrt). The package versions were mvtnorm 1.4.2, copula 1.1.7, Ball 1.3.13, energy 1.7.12, GGally 2.4.0, and Rcpp 1.1.1-1.1. The versions of the packages presented here were MDgof 1.2.1 and MD2sample 1.4.2. The latter includes the corrected cross-sample distance guard in the Aslan–Zech statistic, which tests Dz(i,nx+j)>0 before evaluating log(Dz(i,nx+j)). To illustrate the computational costs, we include two examples.

The following examples illustrate the use of MDgof and MD2sample with real datasets. Their purpose is to demonstrate the software interface, including parameter estimation and adjusted p-values, rather than to provide comprehensive statistical analyses of the underlying scientific questions. The full code and fixed data used for these examples are included in the replication material.

Because the relative power of the procedures depends on the null distribution, the alternative, the dimension, and the sample size, the benchmark results should not be interpreted as general rankings. The simulation facilities provided by the packages allow users to compare procedures under scenarios relevant to their applications.

Multivariate goodness-of-fit and two-sample testing present a computational comparison problem as much as an implementation problem. The large number of available procedures reflects the fact that sensitivity to departures from the null varies with the geometry of the alternative, dimension, sample size, and treatment of nuisance parameters. Consequently, providing another isolated implementation of a test does not by itself solve the practical problem faced by analysts or the evaluation problem faced by researchers developing new methodology.

The contribution of MDgof and MD2sample is a common computational environment in which heterogeneous procedures can be applied, calibrated, compared, and benchmarked. Built-in statistics cover several methodological families and both continuous and discrete data. More importantly, user-defined statistics can enter the same testing, Monte Carlo power, simultaneous-inference, and benchmarking workflows. The deliberately parallel interfaces of the two packages make the computational structure of goodness-of-fit and two-sample studies similar from the user’s perspective even though their null-generation mechanisms differ.

The benchmark collections are not intended to identify a universally best test. Rather, they provide reproducible scenarios in which the strengths and weaknesses of competing statistics can be studied under common conditions. This is especially useful when a new statistic is proposed: the researcher can compare it with established procedures without writing a separate simulation infrastructure for every competitor.

There are also limitations. Some multivariate empirical-distribution-function statistics require computational approximations, and the cost of resampling-based calibration can be substantial for large samples or extensive benchmark grids. The framework therefore uses compiled code and parallel processing where appropriate, but computational burden remains method dependent. Future development can extend the collection of benchmark alternatives, incorporate additional testing families, and improve computational strategies for large or high-dimensional data.

In this sense, MDgof and MD2sample provide infrastructure for reproducible methodological comparison. They provide a common environment in which established and newly proposed multivariate goodness-of-fit and two-sample procedures can be implemented, evaluated, and compared under controlled simulation conditions.

Funding. No funds, grants, or other support was received.

Competing interests. The author has no relevant financial or non-financial interests to disclose.

Generative AI use. OpenAI ChatGPT (GPT-5.6 Sol) was used during manuscript preparation for language and organizational assistance, consistency checking, and software coding and debugging assistance. All resulting text, code, numerical results, and references were reviewed and verified by the author, who takes full responsibility for the content of the manuscript.

Data availability. The Fisher iris data are included with R. The English Premier League data used in Section 9.2 are publicly available from Football-Data.co.uk. The remaining numerical examples and simulation studies use data generated by the code and the MDgof and MD2sample packages described in the article.

Code availability. The MDgof and MD2sample packages are publicly available from CRAN. Code needed to reproduce the numerical examples is given in the article, and the benchmark-generating routines and case studies are included in the packages.