Multivariate Behavioral Research[JOURNAL]
Sun
200 papers
Permutation Tests for Assessing Potential Non-Linear Associations between Treatment Use and Multivariate Clinical Outcomes.
Ren B, Lipsitz SR, Fitzmaurice GM … +1 more , Weiss RD
Multivariate Behav Res · 2024 · PMID 37379399 · Full text
In many psychometric applications, the relationship between the mean of an outcome and a quantitative covariate is too complex to be described by simple parametric functions; instead, flexible nonlinear relationships can...
In many psychometric applications, the relationship between the mean of an outcome and a quantitative covariate is too complex to be described by simple parametric functions; instead, flexible nonlinear relationships can be incorporated using penalized splines. Penalized splines can be conveniently represented as a linear mixed effects model (LMM), where the coefficients of the spline basis functions are random effects. The LMM representation of penalized splines makes the extension to multivariate outcomes relatively straightforward. In the LMM, no effect of the quantitative covariate on the outcome corresponds to the null hypothesis that a fixed effect and a variance component are both zero. Under the null, the usual asymptotic chi-square distribution of the likelihood ratio test for the variance component does not hold. Therefore, we propose three permutation tests for the likelihood ratio test statistic: one based on permuting the quantitative covariate, the other two based on permuting residuals. We compare simulation the Type I error rate and power of the three permutation tests obtained from joint models for multiple outcomes, as well as a commonly used parametric test. The tests are illustrated using data from a stimulant use disorder psychosocial clinical trial.
Person Specific Parameter Heterogeneity in the 2PL IRT Model.
Perez AL, Loken E
Multivariate Behav Res · 2024 · PMID 37351913 · Publisher ↗
Following Kelderman and Molenaar's demonstration that a factor model with person specific factor loadings is almost indistinguishable from the standard factor model in terms of overall fit, we examined person specific me...
Following Kelderman and Molenaar's demonstration that a factor model with person specific factor loadings is almost indistinguishable from the standard factor model in terms of overall fit, we examined person specific measurement models in Item Response Theory, person specific discrimination and difficulty parameters were created by adding random variation at the item by person level. Using standard fitting algorithms for the 2PL IRT there was modest evidence of person- or item-level misfit using common diagnostic tools. The item difficulties were well-estimated, but the item discriminations were noticeably underestimated. As found by Kelderman and Molenaar, factor scores were estimated with less than expected reliability due to the underlying heterogeneity. The person specific models considered here are basically limiting cases of IRT models with multilevel, mixture, or differential item functioning structure. We conclude with some thoughts regarding real-world sources of heterogeneity that might go unacknowledged in common testing applications.
A Mixed-Effects Model in Which the Parameters of the Autocorrelated Error Structure Can Differ between Individuals.
Nestler S
Multivariate Behav Res · 2024 · PMID 37351912 · Publisher ↗
Research in psychology has seen a rapid increase in the usage of experience sampling methods and daily diary methods. The data that result from using these methods are typically analyzed with a mixed-effects or a multile...
Research in psychology has seen a rapid increase in the usage of experience sampling methods and daily diary methods. The data that result from using these methods are typically analyzed with a mixed-effects or a multilevel model because it allows testing hypotheses about the time course of the longitudinally assessed variable or the influence of time-varying predictors in a simple way. Here, we describe an extension of this model that does not only allow to include random effects for the mean structure but also for the residual variance, for the parameter of an autoregressive process of order 1 and/or the parameter of a moving average process of order 1. After we have introduced this extension, we show how to estimate the parameters with maximum likelihood. Because the likelihood function contains complex integrals, we suggest using adaptive Gauss-Hermite quadrature and Quasi-Monte Carlo integration to approximate it. We illustrate the models using a real data example and also report the results of a small simulation study in which the two integral approximation methods are compared.
On the Accuracy of Replication Failure Rates.
Schauer JM
Multivariate Behav Res · 2023 · PMID 37339430 · Publisher ↗
A prominent approach to studying the replication crisis has been to conduct replications of several different scientific findings as part of the same research effort. The reported proportion of findings that these progra...
A prominent approach to studying the replication crisis has been to conduct replications of several different scientific findings as part of the same research effort. The reported proportion of findings that these programs determined failed to replicate have become important statistics in the replication crisis. However, these "failure rates" are based on decisions about whether individual studies replicated, which are themselves subject to statistical uncertainty. In this article, we examine how that uncertainty impacts the accuracy of reported failure rates and find that the reported failure rates can be substantially biased and highly variable. Indeed, very high or very low failure rates could arise from chance alone.
A Time-Varying Dynamic Partial Credit Model to Analyze Polytomous and Multivariate Time Series Data.
Castro-Alvarez S, Bringmann LF, Meijer RR … +1 more , Tendeiro JN
Multivariate Behav Res · 2024 · PMID 37318274 · Publisher ↗
The accessibility to electronic devices and the novel statistical methodologies available have allowed researchers to comprehend psychological processes at the individual level. However, there are still great challenges...
The accessibility to electronic devices and the novel statistical methodologies available have allowed researchers to comprehend psychological processes at the individual level. However, there are still great challenges to overcome as, in many cases, collected data are more complex than the available models are able to handle. For example, most methods assume that the variables in the time series are measured on an interval scale, which is not the case when Likert-scale items were used. Ignoring the scale of the variables can be problematic and bias the results. Additionally, most methods also assume that the time series are stationary, which is rarely the case. To tackle these disadvantages, we propose a model that combines the partial credit model (PCM) of the item response theory framework and the time-varying autoregressive model (TV-AR), which is a popular model used to study psychological dynamics. The proposed model is referred to as the time-varying dynamic partial credit model (TV-DPCM), which allows to appropriately analyze multivariate polytomous data and nonstationary time series. We test the performance and accuracy of the TV-DPCM in a simulation study. Lastly, by means of an example, we show how to fit the model to empirical data and interpret the results.
Quantifying Evidence for-and against-Granger Causality with Bayes Factors.
Oravecz Z, Vandekerckhove J
Multivariate Behav Res · 2024 · PMID 37293977 · Publisher ↗
Testing for Granger causality relies on estimating the capacity of dynamics in one time series to forecast dynamics in another. The canonical test for such temporal predictive causality is based on fitting multivariate t...
Testing for Granger causality relies on estimating the capacity of dynamics in one time series to forecast dynamics in another. The canonical test for such temporal predictive causality is based on fitting multivariate time series models and is cast in the classical null hypothesis testing framework. In this framework, we are limited to rejecting the null hypothesis or failing to reject the null - we can never validly accept the null hypothesis of no Granger causality. This is poorly suited for many common purposes, including evidence integration, feature selection, and other cases where it is useful to express evidence against, rather than for, the existence of an association. Here we derive and implement the Bayes factor for Granger causality in a multilevel modeling framework. This Bayes factor summarizes information in the data in terms of a continuously scaled evidence ratio between the presence of Granger causality and its absence. We also introduce this procedure for the multilevel generalization of Granger causality testing. This facilitates inference when information is scarce or noisy or if we are interested primarily in population-level trends. We illustrate our approach with an application on exploring causal relationships in affect using a daily life study.
An Iterative Scale Purification Procedure on for the Detection of Aberrant Responses.
Qiu X, Huang SY, Wang WC … +1 more , Wang YG
Multivariate Behav Res · 2024 · PMID 37261427 · Publisher ↗
Many person-fit statistics have been proposed to detect aberrant response behaviors (e.g., cheating, guessing). Among them, is one of the most widely used indices. The computation of assumes the item and person paramet...
Many person-fit statistics have been proposed to detect aberrant response behaviors (e.g., cheating, guessing). Among them, is one of the most widely used indices. The computation of assumes the item and person parameters are known. In reality, they often have to be estimated from data. The better the estimation, the better will perform. When aberrant behaviors occur, the person and item parameter estimations are inaccurate, which in turn degrade the performance of . In this study, an iterative procedure was developed to attain more accurate person parameter estimates for improved performance of . A series of simulations were conducted to evaluate the iterative procedure under two conditions of item parameters, known and unknown, and three aberrant response styles of difficulty-sharing cheating, random-sharing cheating, and random guessing. The results demonstrated the superiority of the iterative procedure over the non-iterative one in maintaining control of Type-I error rates and improving the power of detecting aberrant responses. The proposed procedure was applied to a high-stake intelligence test.
Reorienting Latent Variable Modeling for Supervised Learning.
Jo B, Hastie TJ, Li Z … +3 more , Youngstrom EA, Findling RL, Horwitz SM
Multivariate Behav Res · 2023 · PMID 37229653 · Full text
Despite its potentials benefits, using prediction targets generated based on latent variable (LV) modeling is not a common practice in supervised learning, a dominating framework for developing prediction models. In supe...
Despite its potentials benefits, using prediction targets generated based on latent variable (LV) modeling is not a common practice in supervised learning, a dominating framework for developing prediction models. In supervised learning, it is typically assumed that the outcome to be predicted is clear and readily available, and therefore validating outcomes before predicting them is a foreign concept and an unnecessary step. The usual goal of LV modeling is inference, and therefore using it in supervised learning and in the prediction context requires a major conceptual shift. This study lays out methodological adjustments and conceptual shifts necessary for integrating LV modeling into supervised learning. It is shown that such integration is possible by combining the traditions of LV modeling, psychometrics, and supervised learning. In this interdisciplinary learning framework, generating practical outcomes using LV modeling and systematically validating them based on clinical validators are the two main strategies. In the example using the data from the Longitudinal Assessment of Manic Symptoms (LAMS) Study, a large pool of candidate outcomes is generated by flexible LV modeling. It is demonstrated that this exploratory situation can be used as an opportunity to tailor desirable prediction targets taking advantage of contemporary science and clinical insights.
Combining Item Purification and Multiple Comparison Adjustment Methods in Detection of Differential Item Functioning.
Hladká A, Martinková P, Magis D
Multivariate Behav Res · 2024 · PMID 37218672 · Publisher ↗
Many of the differential item functioning (DIF) detection methods rely on a principle of testing for DIF item by item, while considering the rest of the items or at least some of them being DIF-free. Computational algori...
Many of the differential item functioning (DIF) detection methods rely on a principle of testing for DIF item by item, while considering the rest of the items or at least some of them being DIF-free. Computational algorithms of these DIF detection methods involve the selection of DIF-free items in an iterative procedure called . Another aspect is the need to correct for multiple comparisons, which can be done with a number of existing methods. In this article, we demonstrate that implementation of these two controlling procedures together may have an impact on which items are detected as DIF items. We propose an iterative algorithm combining item purification and adjustment for multiple comparisons. Pleasant properties of the newly proposed algorithm are shown with a simulation study. The method is demonstrated on a real data example.
Go Multivariate: Recommendations on Bayesian Multilevel Hidden Markov Models with Categorical Data.
Mildiner Moraga S, Aarts E
Multivariate Behav Res · 2024 · PMID 37195880 · Publisher ↗
The multilevel hidden Markov model (MHMM) is a promising method to investigate intense longitudinal data obtained within the social and behavioral sciences. The MHMM quantifies information on the latent dynamics of behav...
The multilevel hidden Markov model (MHMM) is a promising method to investigate intense longitudinal data obtained within the social and behavioral sciences. The MHMM quantifies information on the latent dynamics of behavior over time. In addition, heterogeneity between individuals is accommodated with the inclusion of individual-specific random effects, facilitating the study of individual differences in dynamics. However, the performance of the MHMM has not been sufficiently explored. We performed an extensive simulation to assess the effect of the number of dependent variables (1-8), number of individuals (5-90), and number of observations per individual (100-1600) on the estimation performance of a Bayesian MHMM with categorical data including various levels of state distinctiveness and separation. We found that using multivariate data generally alleviates the sample size needed and improves the stability of the results. Moreover, including variables only consisting of random noise was generally not detrimental to model performance. Regarding the estimation of group-level parameters, the number of individuals and observations largely compensate for each other. However, only the former drives the estimation of between-individual variability. We conclude with guidelines on the sample size necessary based on the level of state distinctiveness and separation and study objectives of the researcher.
Touchstones of Equivalence and the Houdini Transformation.
Rovine MJ, McDermott PA
Multivariate Behav Res · 2024 · PMID 37191469 · Publisher ↗
Inspired by Peter Molenaar's Houdini transformation, we consider the idea of touchstones between different models. Touchstones represent instances where models that appear different on the surface can have equivalent cha...
Inspired by Peter Molenaar's Houdini transformation, we consider the idea of touchstones between different models. Touchstones represent instances where models that appear different on the surface can have equivalent characteristics. Touchstones can appear as identical tests of model parameters. They can exist in the mean structure, in the covariance structure, or in both. In the latter case, the models will generate identical mean and covariance structures and will fit the data equally well. After showing some examples of touchstones and how they result from constraints on a general model, we show how that idea can suggest Molenaar's Houdini transformation. This transformation allows one to take a latent variable model and derive an equivalent model comprised solely of manifest (observed) variables. As equivalent models, the parameters of one can be transformed into the parameters of the other.
Unique Variable Analysis: A Network Psychometrics Method to Detect Local Dependence.
Christensen AP, Garrido LE, Golino H
Multivariate Behav Res · 2023 · PMID 37139938 · Publisher ↗
The local independence assumption states that variables are unrelated after conditioning on a latent variable. Common problems that arise from violations of this assumption include model misspecification, biased model pa...
The local independence assumption states that variables are unrelated after conditioning on a latent variable. Common problems that arise from violations of this assumption include model misspecification, biased model parameters, and inaccurate estimates of internal structure. These problems are not limited to latent variable models but also apply to network psychometrics. This paper proposes a novel network psychometric approach to detect locally dependent pairs of variables using network modeling and a graph theory measure called weighted topological overlap (wTO). Using simulation, this approach is compared to contemporary local dependence detection methods such as exploratory structural equation modeling with standardized expected parameter change and a recently developed approach using partial correlations and a resampling procedure. Different approaches to determine local dependence using statistical significance and cutoff values are also compared. Continuous, polytomous (5-point Likert scale), and dichotomous (binary) data were generated with skew across a variety of conditions. Our results indicate that cutoff values work better than significance approaches. Overall, the network psychometrics approaches using wTO with graphical least absolute shrinkage and selector operator with extended Bayesian information criterion and wTO with Bayesian Gaussian graphical model were the best performing local dependence detection methods overall.
Modeling Heterogeneity in Temporal Dynamics: Extending Latent State-Trait Autoregressive and Cross-lagged Panel Models to Mixture Distribution Models.
Holtmann J, Eid M, Santangelo PS … +2 more , Kockler TD, Ebner-Priemer UW
Multivariate Behav Res · 2024 · PMID 37130226 · Publisher ↗
Longitudinal models suited for the analysis of panel data, such as cross-lagged panel or autoregressive latent-state trait models, assume population homogeneity with respect to the temporal dynamics of the variables unde...
Longitudinal models suited for the analysis of panel data, such as cross-lagged panel or autoregressive latent-state trait models, assume population homogeneity with respect to the temporal dynamics of the variables under investigation. This assumption is likely to be too restrictive in a myriad of research areas. We propose an extension of autoregressive and cross-lagged latent state-trait models to mixture distribution models. The models allow researchers to model unobserved person heterogeneity and qualitative differences in longitudinal dynamics based on comparatively few observations per person, while taking into account temporal dependencies between observations as well as measurement error in the variables. The models are extended to include categorical covariates, to investigate the distribution of encountered latent classes across observed groups. The potential of the models is illustrated with an application to self-esteem and affect data in patients with borderline personality disorder, an anxiety disorder, and healthy control participants. Requirements for the models' applicability are investigated in an extensive simulation study and recommendations for model applications are derived.
betaDelta and betaSandwich: Confidence Intervals for Standardized Regression Coefficients in R.
Pesigan IJA, Sun RW, Cheung SF
Multivariate Behav Res · 2023 · PMID 37096594 · Publisher ↗
The multivariate delta method was used by Yuan and Chan to estimate standard errors and confidence intervals for standardized regression coefficients. Jones and Waller extended the earlier work to situations where data a...
The multivariate delta method was used by Yuan and Chan to estimate standard errors and confidence intervals for standardized regression coefficients. Jones and Waller extended the earlier work to situations where data are nonnormal by utilizing Browne's asymptotic distribution-free (ADF) theory. Furthermore, Dudgeon developed standard errors and confidence intervals, employing heteroskedasticity-consistent (HC) estimators, that are robust to nonnormality with better performance in smaller sample sizes compared to Jones and Waller's ADF technique. Despite these advancements, empirical research has been slow to adopt these methodologies. This can be a result of the dearth of user-friendly software programs to put these techniques to use. We present the betaDelta and the betaSandwich packages in the R statistical software environment in this manuscript. Both the normal-theory approach and the ADF approach put forth by Yuan and Chan and Jones and Waller are implemented by the betaDelta package. The HC approach proposed by Dudgeon is implemented by the betaSandwich package. The use of the packages is demonstrated with an empirical example. We think the packages will enable applied researchers to accurately assess the sampling variability of standardized regression coefficients.
Pay Attention to the Ignorable Missing Data Mechanisms! An Exploration of Their Impact on the Efficiency of Regression Coefficients.
Chen L, Savalei V, Rhemtulla M
Multivariate Behav Res · 2023 · PMID 37039444 · Publisher ↗
The use of modern missing data techniques has become more prevalent with their increasing accessibility in statistical software. These techniques focus on handling data that are (MAR). Although all MAR mechanisms are ro...
The use of modern missing data techniques has become more prevalent with their increasing accessibility in statistical software. These techniques focus on handling data that are (MAR). Although all MAR mechanisms are routinely treated as the same, they are not equal. The impact of missing data on the efficiency of parameter estimates can differ for different MAR variations, even when the amount of missing data is held constant; yet, in current practice, only the rate of missing data is reported. The impact of MAR on the loss of efficiency can instead be more directly measured by the (FMI). In this article, we explore this impact using FMIs in regression models with one and two predictors. With the help of a application, we demonstrate that efficiency loss due to missing data can be highly complex and is not always intuitive. We recommend substantive researchers who work with missing data report estimates of FMIs in addition to the rate of missingness. We also encourage methodologists to examine FMIs when designing simulation studies with missing data, and to explore the behavior of efficiency loss under MAR using FMIs in more complex models.
Exploratory Bi-factor Analysis with Multiple General Factors.
Jiménez M, Abad FJ, Garcia-Garzon E … +1 more , Garrido LE
Multivariate Behav Res · 2023 · PMID 37038725 · Publisher ↗
Exploratory bi-factor analysis (EBFA) is a very popular approach to estimate models where specific factors are concomitant to a single, general dimension. However, the models typically encountered in fields like personal...
Exploratory bi-factor analysis (EBFA) is a very popular approach to estimate models where specific factors are concomitant to a single, general dimension. However, the models typically encountered in fields like personality, intelligence, and psychopathology involve more than one general factor. To address this circumstance, we developed an algorithm (GSLiD) based on partially specified targets to perform exploratory bi-factor analysis with multiple general factors (EBFA-MGF). In EBFA-MGF, researchers do not need to conduct independent bi-factor analyses anymore because several bi-factor models are estimated simultaneously in an exploratory manner, guarding against biased estimates and model misspecification errors due to unexpected cross-loadings and factor correlations. The results from an exhaustive Monte Carlo simulation manipulating nine variables of interest suggested that GSLiD outperforms the Schmid-Leiman approximation and is robust to challenging conditions involving cross-loadings and pure items of the general factors. Thereby, we supply an R package (bifactor) to make EBFA-MGF readily available for substantive research. Finally, we use GSLiD to assess the hierarchical structure of a reduced version of the Personality Inventory for DSM-5 Short Form (PID-5-SF).
On the Common but Problematic Specification of Conflated Random Slopes in Multilevel Models.
Rights JD, Sterba SK
Multivariate Behav Res · 2023 · PMID 37038722 · Publisher ↗
For multilevel models (MLMs) with fixed slopes, it has been widely recognized that a level-1 variable can have distinct between-cluster and within-cluster fixed effects, and that failing to disaggregate these effects yie...
For multilevel models (MLMs) with fixed slopes, it has been widely recognized that a level-1 variable can have distinct between-cluster and within-cluster fixed effects, and that failing to disaggregate these effects yields a conflated, uninterpretable fixed effect. For MLMs with random slopes, however, we clarify that two different types of slope conflation can occur: that of the fixed component (termed fixed conflation) and that of the random component (termed random conflation). The latter is rarely recognized and not well understood. Here we explain that a model commonly used to disaggregate the fixed component-the contextual effect model with random slopes-troublingly still yields a conflated random component. Negative consequences of such random conflation have not been demonstrated. Here we show that they include erroneous interpretation and inferences about the substantively important extent of between-cluster differences in slopes, including either underestimating or overestimating such slope heterogeneity. Furthermore, we show that this random conflation can yield inappropriate standard errors for fixed effects. To aid researchers in practice, we delineate which types of random slope specifications yield an unconflated random component. We demonstrate the advantages of these unconflated models in terms of estimating and testing random slope variance (i.e., improved power, Type I error, and bias) and in terms of standard error estimation for fixed effects (i.e., more accurate standard errors), and make recommendations for which specifications to use for particular research purposes.
CLC Estimator: A Tool for Latent Construct Estimation via Congeneric Approaches in Survey Research.
Marzi G, Balzano M, Egidi L … +1 more , Magrini A
Multivariate Behav Res · 2023 · PMID 37038660 · Publisher ↗
This article proposes the Shiny app 'CLC Estimator' -Congeneric Latent Construct Estimator- to address the problem of estimating latent unidimensional constructs via congeneric approaches. While congeneric approaches pro...
This article proposes the Shiny app 'CLC Estimator' -Congeneric Latent Construct Estimator- to address the problem of estimating latent unidimensional constructs via congeneric approaches. While congeneric approaches provide more rigorous results than suboptimal parallel-based scoring methods, most statistical packages do not provide easy access to congeneric approaches. To address this issue, the CLC Estimator allows social scientists to use congeneric approaches to estimate latent unidimensional constructs smoothly. The present app provides a novel solution to the challenge of limited access to congeneric estimation methods in survey research.
Which is Better for Individual Participant Data Meta-Analysis of Zero-Inflated Count Outcomes, One-Step or Two-Step Analysis? A Simulation Study.
Huh D, Baldwin SA, Zhou Z … +2 more , Park J, Mun EY
Multivariate Behav Res · 2023 · PMID 36952487 · Full text
Meta-analysis using individual participant data (IPD) is an important methodology in intervention research because it (a) increases accuracy and precision of estimates, (b) allows researchers to investigate mediators and...
Meta-analysis using individual participant data (IPD) is an important methodology in intervention research because it (a) increases accuracy and precision of estimates, (b) allows researchers to investigate mediators and moderators of treatment effects, and (c) makes use of extant data. IPD meta-analysis can be conducted either via a one-step approach that uses data from all studies simultaneously, or a two-step approach, which aggregates data for each study and then combines them in a traditional meta-analysis model. Unfortunately, there are no evidence-based guidelines for how best to approach IPD meta-analysis for count outcomes with many zeroes, such as alcohol use. We used simulation to compare the performance of four hurdle models (3 one-step and 1 two-step models) for zero-inflated count IPD, under realistic data conditions. Overall, all models yielded adequate coverage and bias for the treatment effect in the count portion of the model, across all data conditions. However, in the zero portion, the treatment effect was underestimated in most models and data conditions, especially when there were fewer studies. The performance of both one- and two-step approaches depended on the formulation of the treatment effects, suggesting a need to carefully consider model assumptions and specifications when using IPD.
Fitting Bayesian Stochastic Differential Equation Models with Mixed Effects through a Filtering Approach.
Chen M, Chow SM, Oravecz Z … +1 more , Ferrer E
Multivariate Behav Res · 2023 · PMID 36848197 · Full text
Recent advances in technology contribute to a fast-growing number of studies utilizing intensive longitudinal data, and call for more flexible methods to address the demands that come with them. One issue that arises fro...
Recent advances in technology contribute to a fast-growing number of studies utilizing intensive longitudinal data, and call for more flexible methods to address the demands that come with them. One issue that arises from collecting longitudinal data from multiple units in time is nested data, where the variability observed in such data is a mixture of within-unit changes and between-unit differences. This article aims to provide a model-fitting approach that simultaneously models the within-unit changes with differential equation models and accounts for between-unit differences with mixed effects. This approach combines a variant of the Kalman filter, the continuous-discrete extended Kalman filter (CDEKF), and the Markov chain Monte Carlo method often employed in the Bayesian framework through the platform Stan. At the same time, it utilizes Stan's functionality of numerical solvers for the implementation of CDEKF. For an empirical illustration, we applied this method in the context of differential equation models to an empirical dataset to explore the physiological dynamics and co-regulation between couples.
About
- Frequency
- Sun
- Papers found
- 200
- RSS feed
- Subscribe