In a linear latent growth model parameterized by intercept (α) and slope (β) factors, those factors' relation is often of interest. The model typically captures this through their covariance parameter, which inherently a...In a linear latent growth model parameterized by intercept (α) and slope (β) factors, those factors' relation is often of interest. The model typically captures this through their covariance parameter, which inherently assumes linearity in their relation. However, this assumption may not always hold. For instance, α and β might be unrelated below a certain threshold along the α-axis but show a meaningful relation above it. That is, even though individual growth trajectories may follow a linear pattern over time, the relation between α and β can be nonlinear, potentially featuring distinct segments separated by a transition point. To address such relations, we propose a semiparametric approach that combines Bayesian P-splines for flexible nonlinear modeling of the α-β relation along with a segmented regression-based transition point detection method. This two-stage analytic approach provides for a more nuanced understanding of the α-β relation, including estimation of a potential transition point where the α-β relation structure fundamentally changes. Simulation results and an empirical data illustration support this approach's effectiveness with single transition point scenarios, offering deeper insights into aspects of the growth process.
Metamodels, or the regression analysis of Monte Carlo simulation results, provide a powerful tool to summarize simulation findings. However, an underutilized approach is the multilevel metamodel (MLMM) that accounts for...Metamodels, or the regression analysis of Monte Carlo simulation results, provide a powerful tool to summarize simulation findings. However, an underutilized approach is the multilevel metamodel (MLMM) that accounts for the dependent data structure that arises from fitting multiple models to the same simulated data set. In this study, we articulate the theoretical rationale for the MLMM and illustrate how it can improve the interpretability of simulation results, better account for complex simulation designs, and provide new insights into the generalizability of simulation findings.
Norms play a critical role in high-stakes individual assessments (e.g., diagnosing intellectual disabilities), where precision and stability are essential. To reduce fluctuations in norms due to sampling, normative studi...Norms play a critical role in high-stakes individual assessments (e.g., diagnosing intellectual disabilities), where precision and stability are essential. To reduce fluctuations in norms due to sampling, normative studies must be based on sufficiently large and well-designed samples. This paper provides formulas, applicable to any sample composition, for determining the required sample size for normative studies under the simplified parametric norming framework. In addition to a sufficiently large sample size, precision can be further improved by sampling according to an optimal design, that is, a sample composition that minimizes sampling error in the norms. Optimal designs are, here, derived for 45 (multivariate) multiple linear regression models, assuming normality and homoscedasticity. These models vary in the degree of interaction among three norm-predictors: a continuous variable (e.g., age), a categorical variable (e.g., sex), and a variable (e.g., education) that may be treated as either continuous or categorical. To support practical implementation, three interactive Shiny apps are introduced, enabling users to determine the sample size for their normative studies. Their use is demonstrated through the hypothetical planning of a normative study for the Trail Making Test, accompanied by a review of the most common models for this neuropsychological test in current practice.
Multilevel compositional data, such as data sampled over time that are non-negative and sum to a constant value, are common in various fields. However, there is currently no software specifically built to model compositi...Multilevel compositional data, such as data sampled over time that are non-negative and sum to a constant value, are common in various fields. However, there is currently no software specifically built to model compositional data in a multilevel framework. The package implements a collection of tools for modeling compositional data in a Bayesian multivariate, multilevel pipeline. The user-friendly setup only requires the data, model formula, and minimal specification of the analysis. This article outlines the statistical theory underlying the Bayesian compositional multilevel modeling approach and details the implementation of the functions available in , using an example dataset of compositional daily sleep-wake behaviors. This innovative method can be used to robustly answer scientific questions from the increasingly available multilevel compositional data from intensive, longitudinal studies.
The appeal of lagged-effects models, like the first-order vector autoregressive (VAR(1)) model, is the interpretation of the lagged coefficients in terms of predictive-and possibly causal-relationships between variables...The appeal of lagged-effects models, like the first-order vector autoregressive (VAR(1)) model, is the interpretation of the lagged coefficients in terms of predictive-and possibly causal-relationships between variables over time. While the focus in VAR(1) applications has traditionally been on the strength and sign of the lagged relationships, there has been a growing interest in the residual relationships (i.e., the correlations between the innovations) as well. In this article, we will investigate what residual correlations can and cannot signal, for both the discrete-time (DT) and continuous-time (CT) VAR(1) model, when inspecting a CT process. We will show that one should not take on a DT perspective when investigating a CT process: Correlated (i.e., non-zero) DT residuals can flag omitted common causes and effects at shorter intervals (which is well-known), but-when having a CT process-also effects at longer intervals. Furthermore, when inspecting a CT process, uncorrelated (i.e., zero) DT residuals do not imply that the variables have no effect on each other at other intervals, nor does it preclude the risk of having omitted common causes. Additionally, we will show that residual correlations in a CT model signal omitted causes for one or more of the observed variables. This may bias the estimation of lagged relationships, implying that the found predictive lagged relationships do not equal the underlying causal lagged relationships. Unfortunately, the CT residual correlations do not reflect the magnitude of the distortion.
Seemingly routine data-preprocessing choices can exert outsized influence on the conclusions drawn from randomized controlled trials (RCTs), particularly in behavioral science where data are noisy, skewed and replete wit...Seemingly routine data-preprocessing choices can exert outsized influence on the conclusions drawn from randomized controlled trials (RCTs), particularly in behavioral science where data are noisy, skewed and replete with outliers. We demonstrate this influence with two fully specified multiverse analyses on simulated RCT data. Each analysis spans 180 analytical pathways, produced by crossing 36 preprocessing pipelines that vary outlier handling, missing-data imputation and scale transformation, with five common model specifications. In Simulation A, which uses linear regression families, preprocessing decisions explain 76.9% of the total variance in estimated treatment effects, whereas model choice explains only 7.5%. In Simulation B, which replaces the linear models with advanced algorithms (generalized additive models, random forests, gradient boosting), the dominance of preprocessing is even clearer: 99.8% of the variance is attributable to data handling and just 0.1% to model specification. The ranges of mean effects show the same pattern (4.34 vs. 1.43 in Simulation A; 15.30 vs. 0.56 in Simulation B). Particular pipelines-most notably those that standardize or log-transform variables-shrink effect estimates by more than 90% relative to the raw-data baseline, while pipelines that leave the original scale intact can inflate effects by an order of magnitude. Because preprocessing choices can overshadow even large shifts in statistical methodology, we call for meticulous reporting of these steps and for routine sensitivity or multiverse analyses that make their impact transparent. Such practices are essential for improving the robustness and replicability of behavioral-science RCTs.
Despite the popularity of structural equation modeling in psychological research, accurately evaluating the fit of these models to data is still challenging. Using fixed fit index cutoffs is error-prone due to the fit in...Despite the popularity of structural equation modeling in psychological research, accurately evaluating the fit of these models to data is still challenging. Using fixed fit index cutoffs is error-prone due to the fit indices' dependence on various features of the model and data ("nuisance parameters"). Nonetheless, applied researchers mostly rely on fixed fit index cutoffs, neglecting the risk of falsely accepting (or rejecting) their model. With the goal of developing a broadly applicable method that is almost independent of nuisance parameters, we introduce a machine learning (ML)-based approach to evaluate the fit of multi-factorial measurement models. We trained an ML model based on 173 model and data features that we extracted from 1,323,866 simulated data sets and models fitted by means of confirmatory factor analysis. We evaluated the performance of the ML model based on 1,659,386 independent test observations. The ML model performed very well in detecting model (mis-)fit in most conditions, hereby outperforming commonly used fixed fit index cutoffs across the board. Only minor misspecifications, such as a single neglected residual correlation, proved to be challenging to detect. This proof-of-concept study shows that ML is very promising in the context of model fit evaluation.
Growth mixture models (GMMs) are popular approaches for modeling unobserved population heterogeneity over time. GMMs can be extended with covariates, predicting latent class (LC) membership, the within-class growth traje...Growth mixture models (GMMs) are popular approaches for modeling unobserved population heterogeneity over time. GMMs can be extended with covariates, predicting latent class (LC) membership, the within-class growth trajectories, or both. However, current estimators are sensitive to misspecifications in complex models. We propose extending the two-step estimator for LC models to GMMs, which provides robust estimation against model misspecifications (namely, ignored and overfitted the direct effects) for simpler LC models. We conducted several simulation studies, comparing the performance of the proposed two-step estimator to the commonly-used one- and three-step estimators. Three different population models were considered, including covariates that predicted only the LC membership (I), adding direct effects to the latent intercept (II), or to both growth factors (III). Results show that when predicting LC membership alone, all three estimators are unbiased when the measurement model is strong, with weak measurement model results being more nuanced. Alternatively, when including covariate effects on the growth factors, the two-step, and three-step estimators show consistent robustness against misspecifications with unbiased estimates across simulation conditions while tending to underestimate the standard error estimates while the one-step estimator is most sensitive to misspecifications.
Second-order latent growth models (LGMs) have garnered considerable attention and are increasingly utilized in longitudinal data analyses of latent constructs comprised of multiple items. The growth parameter estimates i...Second-order latent growth models (LGMs) have garnered considerable attention and are increasingly utilized in longitudinal data analyses of latent constructs comprised of multiple items. The growth parameter estimates in these models are intrinsically linked to the model identification methods. Latent-standardization (identification) methods, in which the latent variable is standardized at a reference time point (e.g., eta-1), yield theoretically unique and interpretable growth parameters. Traditional latent-standardization methods indirectly standardize eta-1 the first-order component of the second-order LGM by constraining item intercepts and/or loadings. Such methods require a two-step modeling procedure and do not truly standardize eta-1. This article proposes a 1-stage method that indirectly standardizes eta-1 through the second-order component of the model by constraining the mean and variance of the level factor. This new single-step modeling method ensures eta-1 is truly standardized, with a mean of 0 and a variance of 1. Theoretical, simulated, and empirical comparisons are conducted across different latent-standardization methods, demonstrating the target accuracy and implementation simplicity of the proposed 1-stage method.
Single-case experimental designs (SCEDs) involve repeated measurements of a small number of cases under different experimental conditions, offering valuable insights into treatment effects. However, challenges arise in t...Single-case experimental designs (SCEDs) involve repeated measurements of a small number of cases under different experimental conditions, offering valuable insights into treatment effects. However, challenges arise in the analysis of SCEDs when autocorrelation is present in the data. Recently, generalized linear mixed models (GLMMs) have emerged as a promising statistical approach for SCEDs with count outcomes. While prior research has demonstrated the effectiveness of GLMMs, these studies have typically assumed error independence, an assumption that may be violated in SCEDs due to serial dependency. This study aims to evaluate two possible solutions for autocorrelated SCED count data: 1) to assess the robustness of previously introduced GLMMs such as Poisson, negative binomial, and observation-level random effects models under various levels of autocorrelation, and 2) to evaluate the performance of a new GLMM and a linear mixed model (LMM), both of which incorporate an autoregressive error structure. Through a Monte Carlo simulation study, we have examined bias, coverage rates, and Type I error rates of treatment effect estimators, providing recommendations for handling autocorrelation in the analysis of SCED count data. A demonstration with real SCED count data is provided. The implications, limitations, and future research directions are also discussed.
In this study, we extend the dynamic fit index (DFI) developed by McNeish and Wolf to the context of time series analysis. DFI is a simulation-based method for deriving fit index cutoff values tailored to the specific mo...In this study, we extend the dynamic fit index (DFI) developed by McNeish and Wolf to the context of time series analysis. DFI is a simulation-based method for deriving fit index cutoff values tailored to the specific model and data characteristics. Through simulations, we show that DFI cutoffs for detecting an omitted path in time series network models tend to be closer to exact fit than the popular benchmark values developed by Hu and Bentler. Moreover, cutoff values vary by number of variables, network density, number of time points, and form of misspecification. Notably, using 10% as the upper limit of Type I and Type II error rates, the original DFI approach fails to identify cutoffs for detecting an omitted path when effect size and/or sample size is small. To address this problem, we propose two alternatives that allow for the derivation of cutoffs using more lenient criteria. DFI extends the original DFI approach by removing the upper limit of Type I and Type II error rates, whereas DFI aims at maximizing classification quality measured by the Matthews correlation coefficient. We demonstrate the utility of these approaches using simulation and empirical data and discuss their implications in practice.
It is a well-known fact that for the bivariate normal distribution the ratio between the point-polyserial correlation (the linear correlation after one of the two variables is discretized into categories with probabilit...It is a well-known fact that for the bivariate normal distribution the ratio between the point-polyserial correlation (the linear correlation after one of the two variables is discretized into categories with probabilities ) and the polyserial correlation (the linear correlation between the two normal components) remains constant with keeping the 's fixed. If we move away from the bivariate normal distribution, by considering non-normal margins and/or non-normal dependence structures, then the constancy of this ratio may get lost. In this work, the magnitude of the departure from the constancy condition is assessed for several combinations of margins (normal, uniform, exponential, Weibull) and copulas (Gauss, Frank, Gumbel, Clayton), also varying the distribution of the discretized variable. The results indicate that for many settings we are far from the condition of constancy, especially when highly asymmetrical marginal distributions are combined with copulas that allow for tail-dependence. In such cases, the linear correlation may even increase instead of decreasing, contrary to the usual expectation. This implies that most existing simulation techniques or statistical models for mixed-type data, which assume a linear relationship between point-polyserial and polyserial correlations, should be used very prudently and possibly reappraised.
Numerous studies have shown that motor inhibition can be triggered automatically when the cognitive system encounters interfering stimuli, even a suspicious stimulus in the absence of perceptual awareness (e.g., the nega...Numerous studies have shown that motor inhibition can be triggered automatically when the cognitive system encounters interfering stimuli, even a suspicious stimulus in the absence of perceptual awareness (e.g., the negative compatibility effect). This study investigated the effect of temporal expectation, a top-down active preparation for future events, on unconscious inhibitory processing both in the local expectation context on a trial-by-trial basis (Experiment 1) and in the global expectation context on a block-wise basis (Experiment 2). Modeling of the behavioral data using a drift-diffusion model showed that temporal expectation can accelerate the evidence accumulation and improve response caution, regardless of context. Importantly, the acceleration is lower when the target is consistent with the suspicious response tendency induced by the subliminal prime than when the target is inconsistent with that, which is significantly correlated with the behavioral RTs (i.e., the compatibility effect). The results provide evidence for a framework in which temporal expectation enhances inhibitory control of unconscious processes. The mechanism is likely to be that temporal expectation enhances the activations afforded by subliminal stimuli and the strength of cognitive monitoring, so that the cognitive system suppresses these suspicious activations more strongly, preventing them from escaping and interfering with subsequent processing.
This study primarily investigates the impact of ignoring nonnormal distributions in RSEM models on the estimation of parameters in the second residual structure. The results of the simulation studies demonstrate that whe...This study primarily investigates the impact of ignoring nonnormal distributions in RSEM models on the estimation of parameters in the second residual structure. The results of the simulation studies demonstrate that when the RSEM model follows a nonnormal distribution, it is crucial to test and estimate the nonnormal distribution while constructing mixture RI-AR or mixture RI-CLPM models. This approach guarantees the unbiased estimation of autoregressive parameters and cross-lagged parameters in the second residual structure. If, during the construction of an empirical model, the nonnormal distribution of mixture RI-AR models or mixture RI-CLPM models is not taken into account, or if a normal distribution is assumed directly for analysis, the resulting parameter estimates for autoregressive parameters and cross-lagged parameters will be biased, leading to erroneous inferences.
Many applications of network modeling involve cross-sectional data of psychological variables (e.g., symptoms for psychological disorders), and analyses are often conducted using a regularized Gaussian graphical model (G...Many applications of network modeling involve cross-sectional data of psychological variables (e.g., symptoms for psychological disorders), and analyses are often conducted using a regularized Gaussian graphical model (GGM) employing a lasso, also known as the graphical lasso or . Appropriate methodology for handling missing data is underdeveloped while using glasso, precluding the use of planned missing data designs to reduce participant fatigue. In this research, we compare three approaches to handling missing data with glasso. The first resembles a two-stage estimation approach-borrowed from the covariance structure modeling literature-whereby a saturated covariance matrix among the items is estimated prior to using glasso. The second and third approaches use glasso and the expectation-maximization (EM) algorithm in a single stage and either use EBIC or cross-validation for tuning parameter selection. We compared these approaches in a simulation study with a variety of sample sizes, proportions of missing data, and network saturation. An example with data from the Patient Reported Outcomes Measurement Information System is also provided. The EM algorithm with cross-validation performed best, but all methods appeared to be viable strategies under larger samples and with less missing data.
The reciprocal relations between sleep and affect have been a common focus in psychological research. Researchers studying affective processes often collect data multiple times a day over several days. Subjective sleep q...The reciprocal relations between sleep and affect have been a common focus in psychological research. Researchers studying affective processes often collect data multiple times a day over several days. Subjective sleep quality, on the other hand, is generally measured once at the beginning of the day. This difference in measurement frequency creates a challenge when analyzing these data, because standard dynamic models are not equipped for this. Furthermore, many of the popular approaches are based on the assumption of stationarity, meaning that processes are assumed to continue throughout the night in the same way as throughout the day. In this paper, we introduce a dynamic structural equation model that incorporates reciprocal relations between momentary affect and daily measures of sleep, tackling both of these challenges and also incorporating individual differences in these relations. To demonstrate the practical applicability of this model, we make use of an empirical example of positive and negative affect. Furthermore, we aim to give researchers the means to adapt or build on this model to align it with different research questions and other asynchronously measured variables.
Ambulatory assessment has gained widespread popularity among researchers who study the dynamics of everyday experiences and behaviors, such as sleep patterns or emotional states. In this paper, we focus on the challenge...Ambulatory assessment has gained widespread popularity among researchers who study the dynamics of everyday experiences and behaviors, such as sleep patterns or emotional states. In this paper, we focus on the challenge that arises when we want to investigate the dynamic relations between variables measured at different frequencies. As a running example, we use a sleep quality variable measured once every morning and a momentary experience variable measured multiple times during the day for multiple days. We propose two = 1 models that imply different processes; the first focuses on dynamic relations from day to day between sleep quality and a daily factor of the momentary experience variable, and the second focuses on dynamic relations from one measurement occasion to the next, which prioritizes when the variables affect each other. Additionally, we introduce a combination of these two models and demonstrate their accuracy with a simulation study. An empirical = 1 example of daily sleep quality and momentary self-doubt demonstrates that dynamic relations exist between sleep quality and self-doubt at certain moments in the day and the daily factor of self-doubt. Researchers may adjust the proposed dynamic models to align with their own theories and to accommodate different data or research interests.
Predicting ordinal responses such as school grades or rating scale data is a common task in the social and life sciences. Currently, two major streams of methodology exist for ordinal prediction: traditional statistical...Predicting ordinal responses such as school grades or rating scale data is a common task in the social and life sciences. Currently, two major streams of methodology exist for ordinal prediction: traditional statistical models such as the proportional odds model and machine learning (ML) methods such as random forest (RF) adapted to ordinal prediction. While methods from the latter stream have displayed high predictive performance, particularly for data characterized by non-linear effects, most of these methods do not support hierarchical data. As such data structures frequently occur in the social and life sciences, e.g., students nested in classes or individual measurements nested within the same person, accounting for hierarchical data is of importance for prediction in these fields. A recently proposed ML method for ordinal prediction displaying promising results for nonhierarchical data is Frequency-Adjusted Borders Ordinal Forest (fabOF). Building on an iterative expectation-maximization-type estimation procedure, I extend fabOF to hierarchical data settings in this work by proposing Mixed-Effects Frequency-Adjusted Borders Ordinal Forest (mixfabOF). The proposed method is shown to achieve performance advantages over fabOF and other existing RF-based prediction methods in settings with high random effect variability. For other settings, mixfabOF performs similarly to fabOF and alternative RF-based prediction methods.
Wording effects, the systematic method variance arising from the inconsistent responding to positively and negatively worded items of the same construct, are pervasive in the behavioral and health sciences. Although seve...Wording effects, the systematic method variance arising from the inconsistent responding to positively and negatively worded items of the same construct, are pervasive in the behavioral and health sciences. Although several factor modeling strategies have been proposed to mitigate their adverse effects, there is limited systematic research assessing their performance with exploratory structural equation models (ESEM). The present study evaluated the impact of different types of response bias related to wording effects (random and straight-line carelessness, acquiescence, item difficulty, and mixed) on ESEM models incorporating two popular method modeling strategies, the correlated traits-correlated methods minus one (CTC[M-1]) model and random intercept item factor analysis (RIIFA), as well as the "do nothing" approach. Five variables were manipulated using Monte Carlo methods: the type and magnitude of response bias, factor loadings, factor correlations, and sample size. Overall, the results showed that ignoring wording effects leads to poor model fit and serious distortions of the ESEM estimates. The RIIFA approach generally performed best at countering these adverse impacts and recovering unbiased factor structures, whereas the CTC(M-1) models struggled when biases affected both positively and negatively worded items. Our findings also indicated that method factors can sometimes reflect or absorb substantive variance, which may blur their associations with external variables and complicate their interpretation when embedded in broader structural models. A straightforward guide is offered to applied researchers who wish to use ESEM with mixed-worded scales.
Many psychological phenomena can be understood as arising from systems of causally connected components that evolve over time within an individual. In current empirical practice, researchers frequently study these system...Many psychological phenomena can be understood as arising from systems of causally connected components that evolve over time within an individual. In current empirical practice, researchers frequently study these systems by fitting statistical models to data collected at a single moment in time, that is, cross-sectional data. This raises a central question: Can cross-sectional data analysis ever yield causal insights into systems that evolve over time-and if so, under what conditions? In this paper, we address this question by introducing Equilibrium Causal Models (ECMs) to the psychological literature. ECMs are causal abstractions of an underlying dynamical system that allow for inferences about the long-term effects of interventions, permit cyclic causal relations, and can in principle be estimated from cross-sectional data, as long as information about the resting state of the system is captured by those measurements. We explain the conditions under which ECM estimation is possible, show that they allow researchers to learn about within-person processes from cross-sectional data, and discuss how tools from both the psychological measurement modeling and the causal discovery literature can inform the ways in which researchers collect and analyze their data.