Sunday, May 28, 2017

Testing Rasch model with MPlus

I am a big fan of the Rasch model and have already dedicated one post to Testing Rasch model in R with eRm. Today I would like to add one more post, because as much as I like R, for modelling latent variables I actually prefer MPlus. Hence, I want to show you how to test the Rasch model in MPlus.

For demonstrative purposes we will use the same data-set as in my previous Rasch model post. If you want to follow along on your own, then you can download the data and all input and output files here.

2-pl Model

We start with testing the less restrictive 2-pl model. In contrast to the Rasch model aka 1-pl model item discrimination parameter may vary in the 2-pl model.

We create the MPlus input file 00_2pl.inp and request the creation of an additional difftest file for later model comparison:

DATA:
    file = rasch.dat;
    
VARIABLE:
    names =
        item1-item10
        group
        ;

    categorical =
        item1-item10
        ;

    usevar =
        item1-item10
        ;

ANALYSIS:
    estimator = WLSMV;
    parameterization = theta;

MODEL:
    !freely estimate all 
    !discrimination parameters
    f1 by
        item1* item2-item10
        ;
        
    f1@1;
        
SAVEDATA:
    difftest = 00_2pl_diff.dat;


Running 00_2pl.inp with MPlus gives us the output (abr.):

[...]

MODEL FIT INFORMATION

Number of Free Parameters                       20

Chi-Square Test of Model Fit

          Value                             79.967*
          Degrees of Freedom                    35
          P-Value                           0.0000

*   The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
    for chi-square difference testing in the regular way.  MLM, MLR and WLSM
    chi-square difference testing is described on the Mplus website.  MLMV, WLSMV,
    and ULSMV difference testing is done using the DIFFTEST option.

RMSEA (Root Mean Square Error Of Approximation)

          Estimate                           0.046
          90 Percent C.I.                    0.033  0.060
          Probability RMSEA <= .05           0.658

CFI/TLI

          CFI                                0.977
          TLI                                0.971

Chi-Square Test of Model Fit for the Baseline Model

          Value                           2041.670
          Degrees of Freedom                    45
          P-Value                           0.0000
          
          
[...]

IRT PARAMETERIZATION IN TWO-PARAMETER PROBIT METRIC
WHERE THE PROBIT IS DISCRIMINATION*(THETA - DIFFICULTY)

 Item Discriminations

 F1       BY
    ITEM1              0.562      0.097      5.765      0.000
    ITEM2              1.039      0.130      8.016      0.000
    ITEM3              1.145      0.124      9.256      0.000
    ITEM4              0.973      0.107      9.101      0.000
    ITEM5              0.900      0.101      8.918      0.000
    ITEM6              1.303      0.155      8.381      0.000
    ITEM7              0.941      0.105      8.998      0.000
    ITEM8              1.010      0.114      8.847      0.000
    ITEM9              0.921      0.124      7.410      0.000
    ITEM10             0.907      0.107      8.497      0.000

 Item Difficulties
    ITEM1$1           -3.743      0.616     -6.074      0.000
    ITEM2$1           -1.935      0.182    -10.617      0.000
    ITEM3$1           -0.996      0.096    -10.321      0.000
    ITEM4$1           -1.507      0.141    -10.700      0.000
    ITEM5$1           -0.327      0.080     -4.073      0.000
    ITEM6$1           -0.249      0.066     -3.767      0.000
    ITEM7$1            0.110      0.075      1.464      0.143
    ITEM8$1            0.473      0.079      5.974      0.000
    ITEM9$1            1.287      0.135      9.536      0.000
    ITEM10$1           2.294      0.248      9.253      0.000

 Variances
    F1                 1.000      0.000      0.000      1.000


The 2-pl model fits the data reasonably well. Further, except for item 1 all item discrimination parameters are very similar, which somewhat points to an 1-pl model.

1-pl Model aka Rasch Model

Initial Model

To convert the 2-pl to the 1-pl model, we fix all discrimination parameters to be equal. We save this change in 01_1pl.inp. Further, we request a χ² difference test, to compare both models.

DATA:
    file = rasch.dat;
    
VARIABLE:
    names =
        item1-item10
        group
        ;

    categorical =
        item1-item10
        ;

    usevar =
        item1-item10
        ;

ANALYSIS:
    estimator = WLSMV;
    parameterization = theta;
    difftest = 00_2pl_diff.dat;

MODEL:
    !fix all discrimination parameters  
    f1 by
        item1* item2-item10 (fix)
        ;

    f1@1;
        
OUTPUT:
    mod;
        
SAVEDATA:
    difftest = 01_1pl_diff.dat;


Running 01_1pl.inp with MPlus gives us the output (abr.):

[...]

MODEL FIT INFORMATION

Number of Free Parameters                       11

Chi-Square Test of Model Fit

          Value                            100.372*
          Degrees of Freedom                    44
          P-Value                           0.0000

Chi-Square Test for Difference Testing

          Value                             23.670
          Degrees of Freedom                     9
          P-Value                           0.0049

*   The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
    for chi-square difference testing in the regular way.  MLM, MLR and WLSM
    chi-square difference testing is described on the Mplus website.  MLMV, WLSMV,
    and ULSMV difference testing is done using the DIFFTEST option.

RMSEA (Root Mean Square Error Of Approximation)

          Estimate                           0.046
          90 Percent C.I.                    0.034  0.058
          Probability RMSEA <= .05           0.683

CFI/TLI

          CFI                                0.972
          TLI                                0.971

Chi-Square Test of Model Fit for the Baseline Model

          Value                           2041.670
          Degrees of Freedom                    45
          P-Value                           0.0000
          
[...]

IRT PARAMETERIZATION IN TWO-PARAMETER PROBIT METRIC
WHERE THE PROBIT IS DISCRIMINATION*(THETA - DIFFICULTY)

 Item Discriminations

 F1       BY
    ITEM1              0.979      0.042     23.201      0.000
    ITEM2              0.979      0.042     23.201      0.000
    ITEM3              0.979      0.042     23.201      0.000
    ITEM4              0.979      0.042     23.201      0.000
    ITEM5              0.979      0.042     23.201      0.000
    ITEM6              0.979      0.042     23.201      0.000
    ITEM7              0.979      0.042     23.201      0.000
    ITEM8              0.979      0.042     23.201      0.000
    ITEM9              0.979      0.042     23.201      0.000
    ITEM10             0.979      0.042     23.201      0.000

 Item Difficulties
    ITEM1$1           -2.622      0.163    -16.107      0.000
    ITEM2$1           -1.993      0.126    -15.756      0.000
    ITEM3$1           -1.072      0.088    -12.123      0.000
    ITEM4$1           -1.503      0.102    -14.681      0.000
    ITEM5$1           -0.313      0.074     -4.214      0.000
    ITEM6$1           -0.283      0.074     -3.813      0.000
    ITEM7$1            0.108      0.073      1.466      0.143
    ITEM8$1            0.481      0.077      6.285      0.000
    ITEM9$1            1.246      0.093     13.463      0.000
    ITEM10$1           2.203      0.146     15.051      0.000

 Variances
    F1                 1.000      0.000      0.000      1.000
    
[...]

MODEL MODIFICATION INDICES

NOTE:  Modification indices for direct effects of observed dependent variables
regressed on covariates and residual covariances among observed dependent
variables may not be included.  To include these, request MODINDICES (ALL).

Minimum M.I. value for printing the modification index    10.000

                            M.I.     E.P.C.  Std E.P.C.  StdYX E.P.C.

BY Statements

F1       BY ITEM1          21.563    -0.619     -0.619       -0.442
F1       BY ITEM6          10.330     0.273      0.273        0.195

WITH Statements

ITEM10   WITH ITEM1        34.327    -0.635     -0.635       -0.635
ITEM10   WITH ITEM2        11.516    -0.285     -0.285       -0.285

Variances/Residual Variances

ITEM1                      21.560     1.323      1.323        0.676
ITEM6                      10.329    -0.626     -0.626       -0.320

[...]


The model still fits reasonably well and now all item discrimination parameters are equal. Looking at the χ² difference test, however, we see that the restrictions yielded a significant (p < .01) worse fit. The modification indices confirm what was already hinted in the results of the 2-pl model – item 1 does not fit into the general pattern.

Adjusting the Model

To test whether assuming an equal discrimination parameter for item 1 is at fault for the significant χ² difference test, we remove the restriction for item 1 and run the model again.

In file 02_1pl_partial.inp we change the model to:

[...]

MODEL:
    !fix all discrimination parameters,
    !but for item1
    f1 by
        item1* 
        item2-item10 (fix)
        ;

    !fix factor variance 
    !for identification
    f1@1;
    
[...]


This improves the fit and the χ² difference test is no longer significant:

[...]

MODEL FIT INFORMATION

Number of Free Parameters                       12

Chi-Square Test of Model Fit

          Value                             80.388*
          Degrees of Freedom                    43
          P-Value                           0.0005

Chi-Square Test for Difference Testing

          Value                              8.956
          Degrees of Freedom                     8
          P-Value                           0.3460

*   The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
    for chi-square difference testing in the regular way.  MLM, MLR and WLSM
    chi-square difference testing is described on the Mplus website.  MLMV, WLSMV,
    and ULSMV difference testing is done using the DIFFTEST option.

RMSEA (Root Mean Square Error Of Approximation)

          Estimate                           0.038
          90 Percent C.I.                    0.025  0.051
          Probability RMSEA <= .05           0.937

CFI/TLI

          CFI                                0.981
          TLI                                0.980

Chi-Square Test of Model Fit for the Baseline Model

          Value                           2041.670
          Degrees of Freedom                    45
          P-Value                           0.0000
          
[...]


Obviously, if we want the Rasch model to fit we have to remove item 1. We do so in 03_1pl_9items.inp:

DATA:
    file = rasch.dat;
    
VARIABLE:
    names =
        item1-item10
        group
        ;

    categorical =
        item2-item10
        ;

    usevar =
        item2-item10
        ;

ANALYSIS:
    estimator = WLSMV;
    parameterization = theta;

MODEL:
    !fix all discrimination parameters
    f1 by
        item2* item3-item10 (fix)
        ;

    !fix factor variance 
    !for identification
    f1@1;
                
OUTPUT:
    mod;


Which nicely fits the data as can be seen in the output (abr.):

[...]

MODEL FIT INFORMATION

Number of Free Parameters                       10

Chi-Square Test of Model Fit

          Value                             59.596*
          Degrees of Freedom                    35
          P-Value                           0.0059

*   The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
    for chi-square difference testing in the regular way.  MLM, MLR and WLSM
    chi-square difference testing is described on the Mplus website.  MLMV, WLSMV,
    and ULSMV difference testing is done using the DIFFTEST option.

RMSEA (Root Mean Square Error Of Approximation)

          Estimate                           0.034
          90 Percent C.I.                    0.018  0.049
          Probability RMSEA <= .05           0.964

CFI/TLI

          CFI                                0.987
          TLI                                0.987

Chi-Square Test of Model Fit for the Baseline Model

          Value                           1990.785
          Degrees of Freedom                    36
          P-Value                           0.0000
          
[...]

IRT PARAMETERIZATION IN TWO-PARAMETER PROBIT METRIC
WHERE THE PROBIT IS DISCRIMINATION*(THETA - DIFFICULTY)

 Item Discriminations

 F1       BY
    ITEM2              1.007      0.044     23.080      0.000
    ITEM3              1.007      0.044     23.080      0.000
    ITEM4              1.007      0.044     23.080      0.000
    ITEM5              1.007      0.044     23.080      0.000
    ITEM6              1.007      0.044     23.080      0.000
    ITEM7              1.007      0.044     23.080      0.000
    ITEM8              1.007      0.044     23.080      0.000
    ITEM9              1.007      0.044     23.080      0.000
    ITEM10             1.007      0.044     23.080      0.000

 Item Difficulties
    ITEM2$1           -1.964      0.125    -15.775      0.000
    ITEM3$1           -1.057      0.087    -12.150      0.000
    ITEM4$1           -1.481      0.100    -14.760      0.000
    ITEM5$1           -0.309      0.073     -4.215      0.000
    ITEM6$1           -0.278      0.073     -3.811      0.000
    ITEM7$1            0.106      0.072      1.466      0.143
    ITEM8$1            0.474      0.075      6.294      0.000
    ITEM9$1            1.228      0.091     13.519      0.000
    ITEM10$1           2.171      0.143     15.188      0.000

 Variances
    F1                 1.000      0.000      0.000      1.000
    
[...]


Testing the Model across Multiple Groups

Now that the models fits the data in general we move on to the most important feature of the Rasch model – invariant comparisons. I.e., the relative positions of the item difficulty parameters are not allowed to differ across groups. In our data-set with have one grouping variable (group).

At first, let’s try whether the model still fits when the data is split into groups. In the file 04_1pl_9items_groups.inp we request a multigroup analysis by defining the grouping variable. MPlus’ default for multigroup analysis is strong measurement invariance, i.e. factor loadings and location parameters for CFA or analogously discrimination and difficulty parameters for IRT are held equal across groups. Hence, to allow the item difficulty parameters to differ between groups we have to explicitly request their estimation for the second group.

DATA:
    file = rasch.dat;
    
VARIABLE:
    names =
        item1-item10
        group
        ;

    categorical =
        item2-item10
        ;

    usevar =
        item2-item10
        ;
        
    grouping =
        group(1 = group1 2 = group2)

ANALYSIS:
    estimator = WLSMV;
    parameterization = theta;

MODEL:
    !fix all discrimination parameters
    f1 by
        item2* item3-item10 (fix)
        ;

    !fix factor variance 
    !for identification
    f1@1;
    
    !fix factor mean 
    !for identification
    [f1@0];
    
    !freely estimate difficulties
    [item2$1-item10$1*];
    
model group2:
    !freely estimate difficulties
    [item2$1-item10$1*];
    
    !fix residual variance
    !to 1 (like in group1)
    item2-item10@1;
    
        
OUTPUT:
    mod;
    
SAVEDATA:
    difftest = 04_1pl_9items_groups_diff.dat;


This yields (abr.):

[...]

MODEL FIT INFORMATION

Number of Free Parameters                       19

Chi-Square Test of Model Fit

          Value                            106.507*
          Degrees of Freedom                    71
          P-Value                           0.0041

Chi-Square Contributions From Each Group

          GROUP1                            63.554
          GROUP2                            42.953

*   The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
    for chi-square difference testing in the regular way.  MLM, MLR and WLSM
    chi-square difference testing is described on the Mplus website.  MLMV, WLSMV,
    and ULSMV difference testing is done using the DIFFTEST option.

RMSEA (Root Mean Square Error Of Approximation)

          Estimate                           0.041
          90 Percent C.I.                    0.023  0.056
          Probability RMSEA <= .05           0.826

CFI/TLI

          CFI                                0.978
          TLI                                0.977

Chi-Square Test of Model Fit for the Baseline Model

          Value                           1658.288
          Degrees of Freedom                    72
          P-Value                           0.0000
          
[...]

IRT PARAMETERIZATION IN TWO-PARAMETER PROBIT METRIC
WHERE THE PROBIT IS DISCRIMINATION*(THETA - DIFFICULTY)

Group GROUP1

 Item Discriminations

 F1       BY
    ITEM2              0.960      0.043     22.366      0.000
    ITEM3              0.960      0.043     22.366      0.000
    ITEM4              0.960      0.043     22.366      0.000
    ITEM5              0.960      0.043     22.366      0.000
    ITEM6              0.960      0.043     22.366      0.000
    ITEM7              0.960      0.043     22.366      0.000
    ITEM8              0.960      0.043     22.366      0.000
    ITEM9              0.960      0.043     22.366      0.000
    ITEM10             0.960      0.043     22.366      0.000

 Means
    F1                 0.000      0.000      0.000      1.000

 Item Difficulties
    ITEM2$1           -1.673      0.147    -11.413      0.000
    ITEM3$1           -0.871      0.115     -7.551      0.000
    ITEM4$1           -1.322      0.129    -10.229      0.000
    ITEM5$1            0.012      0.104      0.115      0.908
    ITEM6$1           -0.024      0.104     -0.231      0.818
    ITEM7$1            0.267      0.106      2.528      0.011
    ITEM8$1            0.771      0.113      6.824      0.000
    ITEM9$1            1.582      0.140     11.319      0.000
    ITEM10$1           2.648      0.231     11.444      0.000

 Variances
    F1                 1.000      0.000      0.000      1.000

Group GROUP2

 Item Discriminations

 F1       BY
    ITEM2              0.960      0.043     22.366      0.000
    ITEM3              0.960      0.043     22.366      0.000
    ITEM4              0.960      0.043     22.366      0.000
    ITEM5              0.960      0.043     22.366      0.000
    ITEM6              0.960      0.043     22.366      0.000
    ITEM7              0.960      0.043     22.366      0.000
    ITEM8              0.960      0.043     22.366      0.000
    ITEM9              0.960      0.043     22.366      0.000
    ITEM10             0.960      0.043     22.366      0.000

 Means
    F1                 0.000      0.000      0.000      1.000

 Item Difficulties
    ITEM2$1           -2.529      0.214    -11.804      0.000
    ITEM3$1           -1.322      0.130    -10.132      0.000
    ITEM4$1           -1.746      0.150    -11.677      0.000
    ITEM5$1           -0.662      0.110     -6.003      0.000
    ITEM6$1           -0.556      0.109     -5.116      0.000
    ITEM7$1           -0.048      0.105     -0.461      0.644
    ITEM8$1            0.218      0.105      2.070      0.038
    ITEM9$1            0.989      0.119      8.344      0.000
    ITEM10$1           1.936      0.167     11.608      0.000

 Variances
    F1                 1.000      0.000      0.000      1.000
    
[...]


The model still fits the data well. In contrast to the earlier analyses we now see how much each group contributes to the χ² and we get separated parameter estimates for the groups.

Next, we fix the item difficulty parameters across groups to test whether they are equal and invariant comparisons hold true. In the file 05_1pl_9items_groups_final.inp we revoke the free estimation of the item difficulty parameters by commenting it out and instead allow the factor mean to be freely estimated in group 2. Fixing the difficulty parameters without freeing the mean would not only assume the same measurement model in both groups, but that the mean score in both groups is equal too.

model group2:
    !fix difficulties
    ![item2$1-item10$1*];
    !but free factor mean
    [f1*];
    
    !fix residual variance
    !to 1 (like in group1)
    item2-item10@1;


As a result we get (abr.):

[...]

THE MODEL ESTIMATION TERMINATED NORMALLY



MODEL FIT INFORMATION

Number of Free Parameters                       11

Chi-Square Test of Model Fit

          Value                            114.107*
          Degrees of Freedom                    79
          P-Value                           0.0060

Chi-Square Contributions From Each Group

          GROUP1                            67.243
          GROUP2                            46.864

Chi-Square Test for Difference Testing

          Value                              8.435
          Degrees of Freedom                     8
          P-Value                           0.3922

*   The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
    for chi-square difference testing in the regular way.  MLM, MLR and WLSM
    chi-square difference testing is described on the Mplus website.  MLMV, WLSMV,
    and ULSMV difference testing is done using the DIFFTEST option.

RMSEA (Root Mean Square Error Of Approximation)

          Estimate                           0.038
          90 Percent C.I.                    0.021  0.053
          Probability RMSEA <= .05           0.893

CFI/TLI

          CFI                                0.978
          TLI                                0.980

Chi-Square Test of Model Fit for the Baseline Model

          Value                           1658.288
          Degrees of Freedom                    72
          P-Value                           0.0000
          
[...]

IRT PARAMETERIZATION IN TWO-PARAMETER PROBIT METRIC
WHERE THE PROBIT IS DISCRIMINATION*(THETA - DIFFICULTY)

Group GROUP1

 Item Discriminations

 F1       BY
    ITEM2              0.960      0.043     22.366      0.000
    ITEM3              0.960      0.043     22.366      0.000
    ITEM4              0.960      0.043     22.366      0.000
    ITEM5              0.960      0.043     22.366      0.000
    ITEM6              0.960      0.043     22.366      0.000
    ITEM7              0.960      0.043     22.366      0.000
    ITEM8              0.960      0.043     22.366      0.000
    ITEM9              0.960      0.043     22.366      0.000
    ITEM10             0.960      0.043     22.366      0.000

 Means
    F1                 0.000      0.000      0.000      1.000

 Item Difficulties
    ITEM2$1           -1.779      0.135    -13.215      0.000
    ITEM3$1           -0.831      0.098     -8.501      0.000
    ITEM4$1           -1.272      0.109    -11.641      0.000
    ITEM5$1           -0.053      0.088     -0.601      0.548
    ITEM6$1           -0.021      0.088     -0.240      0.811
    ITEM7$1            0.379      0.089      4.273      0.000
    ITEM8$1            0.763      0.093      8.184      0.000
    ITEM9$1            1.551      0.112     13.849      0.000
    ITEM10$1           2.535      0.166     15.311      0.000

 Variances
    F1                 1.000      0.000      0.000      1.000

Group GROUP2

 Item Discriminations

 F1       BY
    ITEM2              0.960      0.043     22.366      0.000
    ITEM3              0.960      0.043     22.366      0.000
    ITEM4              0.960      0.043     22.366      0.000
    ITEM5              0.960      0.043     22.366      0.000
    ITEM6              0.960      0.043     22.366      0.000
    ITEM7              0.960      0.043     22.366      0.000
    ITEM8              0.960      0.043     22.366      0.000
    ITEM9              0.960      0.043     22.366      0.000
    ITEM10             0.960      0.043     22.366      0.000

 Means
    F1                 0.000      0.000      0.000      1.000

 Item Difficulties
    ITEM2$1           -2.317      0.148    -15.660      0.000
    ITEM3$1           -1.369      0.108    -12.714      0.000
    ITEM4$1           -1.810      0.121    -14.953      0.000
    ITEM5$1           -0.592      0.092     -6.420      0.000
    ITEM6$1           -0.560      0.091     -6.126      0.000
    ITEM7$1           -0.159      0.089     -1.792      0.073
    ITEM8$1            0.225      0.090      2.491      0.013
    ITEM9$1            1.013      0.102      9.945      0.000
    ITEM10$1           1.997      0.152     13.168      0.000

 Variances
    F1                 1.000      0.000      0.000      1.000
    
[...]


The χ² difference test is not significant, so we assume that the item difficulty parameters are equal across groups. The displayed parameters however still differ. This is due to the fact that MPlus centers the means per group to zero for the IRT parametrization. This is in contrast to the output for the standard CFA parametrization (not shown here, but you can look it up in the output file), where you can see that all item thresholds which correspond to the item difficulty parameters are equal. Further, if you look closely at the item difficulty parameters, then you might notice that not only the ranks are the same across groups, but also the differences between items are exactly the same.

We can conclude that the item difficulty parameters are equivalent, but that the mean of the measured latent construct probably differs between groups. If we want to test this assumption, then we could compare the last model to an altered model with fixed mean in both groups. However, this is a story for another post.

Closing Remarks

I hope the given example was illustrative and can help you to carry out your own Rasch model analyses with MPlus.

If you have any questions or comments please post them in the comments section.

3 comments:

  1. The address to download the data and all input and output files for rasch model is not available and can you give our readers a new one? Thank you!

    ReplyDelete
  2. Excellent outline - I've been looking for something like this all day!

    ReplyDelete

Recommended Post

Follow the white robot - Exploring retweets of Austrian politicians with Botometer in R

botometer_publish.utf8.md Hi folks! I guess you are aware that social medi...

Popular Posts