Full Simulation Results for Each Method (nsims = 200)
across different Data Generating Mechanism Settings
Method True ATE Estimated ATE Bias Analytic SE Monte Carlo SE MSE Coverage
Setting 1
Oracle Estimator 1.096 1.101 0.005 0.068 0.069 0.005 0.925
Simple Regression 1.096 1.098 0.002 0.097 0.105 0.011 0.930
Multiple Regression 1.096 1.101 0.005 0.068 0.070 0.005 0.915
IPTW Estimator 1.096 1.101 0.005 0.068 0.070 0.005 0.920
Survey-Weighted Multiple Regression 1.096 1.102 0.005 0.068 0.069 0.005 0.930
IPTW Multiple Regression 1.096 1.101 0.005 0.060 0.070 0.005 0.880
IPTW + Survey-Weighted Multiple Regression 1.096 1.102 0.005 0.060 0.069 0.005 0.875
Weighted IPTW + Survey-Weighted Multiple Regression 1.096 1.102 0.005 0.060 0.069 0.005 0.875
Naively Weighted G-Computation 1.096 0.992 −0.095 0.030 0.102 0.021 0.290
Outcome Modeling and Direct Standardization 1.096 1.090 −0.005 0.067 0.069 0.005 0.920
Inverse Probability Weighting 1 1.096 1.089 −0.006 0.068 0.069 0.005 0.925
Inverse Probability Weighting 2 1.096 1.091 −0.005 0.067 0.069 0.005 0.920
Setting 2
Oracle Estimator 1.090 1.079 −0.010 0.066 0.067 0.005 0.930
Simple Regression 1.090 1.957 0.795 0.083 0.076 0.756 0.000
Multiple Regression 1.090 1.079 −0.010 0.066 0.067 0.005 0.925
IPTW Estimator 1.090 1.071 −0.018 0.094 0.107 0.012 0.930
Survey-Weighted Multiple Regression 1.090 1.080 −0.009 0.066 0.068 0.005 0.925
IPTW Multiple Regression 1.090 1.077 −0.012 0.060 0.069 0.005 0.885
IPTW + Survey-Weighted Multiple Regression 1.090 1.078 −0.011 0.060 0.070 0.005 0.890
Weighted IPTW + Survey-Weighted Multiple Regression 1.090 1.078 −0.011 0.060 0.070 0.005 0.885
Naively Weighted G-Computation 1.090 0.969 −0.111 0.029 0.089 0.023 0.180
Outcome Modeling and Direct Standardization 1.090 1.069 −0.019 0.065 0.066 0.005 0.930
Inverse Probability Weighting 1 1.090 1.060 −0.028 0.093 0.106 0.012 0.925
Inverse Probability Weighting 2 1.090 1.060 −0.027 0.093 0.106 0.012 0.925
Setting 3
Oracle Estimator 1.100 1.096 −0.004 0.050 0.047 0.002 0.960
Simple Regression 1.100 1.099 −0.001 0.073 0.074 0.005 0.945
Multiple Regression 1.100 1.096 −0.004 0.050 0.047 0.002 0.965
IPTW Estimator 1.100 1.096 −0.004 0.050 0.047 0.002 0.965
Survey-Weighted Multiple Regression 1.100 1.096 −0.004 0.056 0.047 0.002 0.990
IPTW Multiple Regression 1.100 1.096 −0.004 0.050 0.047 0.002 0.965
IPTW + Survey-Weighted Multiple Regression 1.100 1.096 −0.004 0.056 0.047 0.002 0.990
Weighted IPTW + Survey-Weighted Multiple Regression 1.100 1.096 −0.004 0.050 0.047 0.002 0.970
Naively Weighted G-Computation 1.100 0.998 −0.093 0.025 0.073 0.016 0.230
Outcome Modeling and Direct Standardization 1.100 1.084 −0.015 0.051 0.047 0.002 0.955
Inverse Probability Weighting 1 1.100 1.098 −0.002 0.051 0.047 0.002 0.965
Inverse Probability Weighting 2 1.100 1.084 −0.015 0.051 0.047 0.002 0.955
Setting 4
Oracle Estimator 1.106 1.102 −0.004 0.055 0.058 0.003 0.905
Simple Regression 1.106 1.972 0.782 0.070 0.069 0.754 0.000
Multiple Regression 1.106 1.121 0.014 0.058 0.062 0.004 0.915
IPTW Estimator 1.106 1.123 0.015 0.103 0.105 0.011 0.940
Survey-Weighted Multiple Regression 1.106 1.123 0.015 0.050 0.062 0.004 0.890
IPTW Multiple Regression 1.106 1.116 0.009 0.044 0.065 0.004 0.830
IPTW + Survey-Weighted Multiple Regression 1.106 1.118 0.010 0.050 0.065 0.004 0.870
Weighted IPTW + Survey-Weighted Multiple Regression 1.106 1.119 0.011 0.045 0.065 0.004 0.820
Naively Weighted G-Computation 1.106 0.997 −0.099 0.022 0.070 0.017 0.175
Outcome Modeling and Direct Standardization 1.106 1.090 −0.014 0.055 0.058 0.004 0.925
Inverse Probability Weighting 1 1.106 1.104 −0.002 0.076 0.081 0.006 0.940
Inverse Probability Weighting 2 1.106 1.089 −0.015 0.081 0.083 0.007 0.945
Setting 5
Oracle Estimator 1.104 1.108 0.004 0.046 0.048 0.002 0.940
Simple Regression 1.104 1.209 0.095 0.048 0.045 0.013 0.365
Multiple Regression 1.104 1.205 0.092 0.034 0.032 0.011 0.110
IPTW Estimator 1.104 1.205 0.092 0.034 0.032 0.011 0.110
Survey-Weighted Multiple Regression 1.104 1.205 0.092 0.046 0.045 0.012 0.355
IPTW Multiple Regression 1.104 1.205 0.092 0.030 0.032 0.011 0.090
IPTW + Survey-Weighted Multiple Regression 1.104 1.205 0.092 0.041 0.045 0.012 0.270
Weighted IPTW + Survey-Weighted Multiple Regression 1.104 1.205 0.092 0.040 0.045 0.012 0.265
Naively Weighted G-Computation 1.104 1.003 −0.091 0.015 0.074 0.016 0.135
Outcome Modeling and Direct Standardization 1.104 1.090 −0.013 0.048 0.047 0.002 0.935
Inverse Probability Weighting 1 1.104 1.100 −0.003 0.057 0.054 0.003 0.945
Inverse Probability Weighting 2 1.104 1.099 −0.004 0.058 0.057 0.003 0.955
Setting 6
Oracle Estimator 1.096 1.096 0.000 0.041 0.042 0.002 0.935
Simple Regression 1.096 2.004 0.828 0.044 0.043 0.826 0.000
Multiple Regression 1.096 1.184 0.080 0.036 0.036 0.009 0.310
IPTW Estimator 1.096 1.222 0.115 0.058 0.054 0.019 0.440
Survey-Weighted Multiple Regression 1.096 1.189 0.085 0.044 0.059 0.012 0.425
IPTW Multiple Regression 1.096 1.185 0.081 0.030 0.037 0.009 0.220
IPTW + Survey-Weighted Multiple Regression 1.096 1.182 0.079 0.040 0.048 0.010 0.440
Weighted IPTW + Survey-Weighted Multiple Regression 1.096 1.182 0.078 0.040 0.047 0.010 0.435
Naively Weighted G-Computation 1.096 0.999 −0.089 0.015 0.065 0.014 0.130
Outcome Modeling and Direct Standardization 1.096 1.079 −0.015 0.044 0.041 0.002 0.950
Inverse Probability Weighting 1 1.096 1.083 −0.012 0.066 0.072 0.005 0.940
Inverse Probability Weighting 2 1.096 1.089 −0.006 0.066 0.070 0.005 0.935
Setting 7
Oracle Estimator 1.102 1.102 0.001 0.035 0.034 0.001 0.975
Simple Regression 1.102 1.058 −0.040 0.036 0.034 0.003 0.805
Multiple Regression 1.102 1.175 0.067 0.026 0.025 0.006 0.190
IPTW Estimator 1.102 1.174 0.066 0.026 0.025 0.006 0.210
Survey-Weighted Multiple Regression 1.102 1.177 0.068 0.038 0.031 0.007 0.510
IPTW Multiple Regression 1.102 1.175 0.067 0.026 0.025 0.006 0.190
IPTW + Survey-Weighted Multiple Regression 1.102 1.176 0.068 0.037 0.032 0.007 0.455
Weighted IPTW + Survey-Weighted Multiple Regression 1.102 1.177 0.068 0.034 0.031 0.007 0.385
Naively Weighted G-Computation 1.102 1.017 −0.077 0.013 0.054 0.010 0.135
Outcome Modeling and Direct Standardization 1.102 1.086 −0.014 0.038 0.033 0.001 0.970
Inverse Probability Weighting 1 1.102 1.100 −0.002 0.043 0.042 0.002 0.970
Inverse Probability Weighting 2 1.102 1.076 −0.023 0.044 0.045 0.003 0.910
Setting 8
Oracle Estimator 1.104 1.101 −0.002 0.034 0.033 0.001 0.970
Simple Regression 1.104 1.919 0.739 0.040 0.037 0.667 0.000
Multiple Regression 1.104 1.210 0.096 0.034 0.032 0.012 0.130
IPTW Estimator 1.104 1.172 0.062 0.070 0.066 0.009 0.835
Survey-Weighted Multiple Regression 1.104 1.200 0.087 0.033 0.063 0.013 0.300
IPTW Multiple Regression 1.104 1.212 0.098 0.022 0.031 0.013 0.015
IPTW + Survey-Weighted Multiple Regression 1.104 1.207 0.094 0.031 0.034 0.012 0.130
Weighted IPTW + Survey-Weighted Multiple Regression 1.104 1.205 0.092 0.030 0.041 0.012 0.160
Naively Weighted G-Computation 1.104 0.986 −0.107 0.011 0.053 0.017 0.040
Outcome Modeling and Direct Standardization 1.104 1.087 −0.015 0.038 0.032 0.001 0.970
Inverse Probability Weighting 1 1.104 1.115 0.010 0.050 0.051 0.003 0.950
Inverse Probability Weighting 2 1.104 1.083 −0.019 0.047 0.047 0.003 0.935