This article gives an overview of D-error and demonstrates how to compute D-error by working through an example. Concepts in this article are covered in more (mathematical) detail here.
Prior parameter assumptions
When computing D-error, a prior assumption about the respondent parameters needs to be made. D0-error assumes that all parameters are zero — i.e., respondents have no preference for any of the attribute levels. DP-error assumes that all respondent parameters are equal to a parameter vector. On the other hand, DB-error assumes that respondent parameters are distributed according to a probability distribution, which is often a multivariate normal distribution with a diagonal covariance matrix.
DP-error example
A small choice experiment design is shown below:
| Version | Task | Question | Alternative | Attribute 1 | Attribute 2 | Attribute 3 |
|---|---|---|---|---|---|---|
| 1 | 1 | 1 | 1 | 1 | 2 | 1 |
| 1 | 1 | 1 | 2 | 2 | 1 | 2 |
| 1 | 2 | 2 | 1 | 1 | 2 | 2 |
| 1 | 2 | 2 | 2 | 2 | 1 | 1 |
| 1 | 3 | 3 | 1 | 2 | 2 | 1 |
| 1 | 3 | 3 | 2 | 1 | 1 | 2 |
| 2 | 4 | 1 | 1 | 2 | 2 | 2 |
| 2 | 4 | 1 | 2 | 1 | 1 | 1 |
| 2 | 5 | 2 | 1 | 2 | 2 | 2 |
| 2 | 5 | 2 | 2 | 1 | 1 | 1 |
| 2 | 6 | 3 | 1 | 1 | 2 | 1 |
| 2 | 6 | 3 | 2 | 2 | 1 | 2 |
The first step is to encode the design, with either dummy coding or effects coding. I use dummy coding in this example, and split the encoded design by its four questions:
![\[\textbf{X}_1=\left[\begin{matrix}0&1&1\\1&0&0\end{matrix}\right],\textbf{X}_2=\left[\begin{matrix}1&0&1\\1&1&0\end{matrix}\right]\]](https://www.displayr.com/wp-content/ql-cache/quicklatex.com-9109173f94ffbfc86235ea6b7be7aa2e_l3.png "Rendered by QuickLaTeX.com")
![\[ \textbf{X}_3=\left[\begin{matrix}1&0&1\\0&1&0\end{matrix}\right],\textbf{X}_4=\left[\begin{matrix}1&1&1\\0&0&0\end{matrix}\right] \]](https://www.displayr.com/wp-content/ql-cache/quicklatex.com-42c37d2cc6d1325bf7383f38a1f3d290_l3.png "Rendered by QuickLaTeX.com")
For DP-error, I assume that the respondent parameters are given by ![\mathbit{\beta}=[0.5, -0.8, 1.0]](https://www.displayr.com/wp-content/ql-cache/quicklatex.com-fd5ad11727c028aa267a403e224ecf1b_l3.png "Rendered by QuickLaTeX.com")
. The next step is to compute the multinomial logit probabilities, using the formula
![\[\mathbittextbf{p}_{q,i}=\frac{\expfuncapply(\textbf{X}_{q,i}\mathbit{\beta})}{\sum_{j=1}^{J}{\expfuncapply(\textbf{X}_{q,j}\mathbit{\beta})}}\]](https://www.displayr.com/wp-content/ql-cache/quicklatex.com-a8fe5e54516121478be7391b3a6f7848_l3.png "Rendered by QuickLaTeX.com")
where 
refers to the probability of selecting alternative i out of J alternatives in question q. The probabilities are
![\[\mathbittextbf{p}_1=[0.43,0.57],\mathbittextbf{p}_2=[0.86,0.14],\mathbittextbf{p}_3=[0.91,0.09],\mathbittextbf{p}_4=[0.67,0.33]\]](https://www.displayr.com/wp-content/ql-cache/quicklatex.com-8415f3cffa741992419e99b494dcea77_l3.png "Rendered by QuickLaTeX.com")
These probabilities are then used to construct the Fisher information matrix, using the formula
![\[\textbf{M}=\sum_{q=1}^{Q}{{\textbf{X}^\prime}_q(\textrm{diag}(\mathbittextbf{p}_q)-p_q{p_q}\prime)}\textbf{X}_q\]](https://www.displayr.com/wp-content/ql-cache/quicklatex.com-8aeb1167b838bece1ca98f08a345e6d4_l3.png "Rendered by QuickLaTeX.com")
Plugging the values for 
and 
into the formula, the information matrix is
![\[\textbf{M}=\left[\begin{matrix}0.55&-0.11&0.06\\-0.11&0.67&0.26\\0.06&0.26&0.67\end{matrix}\right]\]](https://www.displayr.com/wp-content/ql-cache/quicklatex.com-b1f16d31ecd7931d57048f98fcc20440_l3.png "Rendered by QuickLaTeX.com")
The DP-error is 
, where 
is the number of parameters.
D0-error example
Computing D0-error is just a special case of DP-error where 
is assumed to be a vector of zeros. In this case, the probabilities 
(=1/2 in this example) and the information matrix is
![\[\textbf{M}=\left[\begin{matrix}0.75&-0.25&0.25\\-0.25&1&0\\0.25&0&1\end{matrix}\right]\]](https://www.displayr.com/wp-content/ql-cache/quicklatex.com-c3d9a06689e16854ebf4bd4b804dece8_l3.png "Rendered by QuickLaTeX.com")
The D0-error is 
.
DB-error (Bayesian) Example
DB-error is defined as the integral of DP-error over an assumed prior distribution of the respondent parameters. One way to compute this numerically is known as Monte Carlo estimation. It involves calculating the average DP-error of many sets of parameters randomly drawn from the prior distribution. To illustrate this, assume that the parameter distribution is multivariate normal with mean ![\mu=[0.5, -0.8, 1.0]](https://www.displayr.com/wp-content/ql-cache/quicklatex.com-29083a445f678213a1e07b42b67085e9_l3.png "Rendered by QuickLaTeX.com")
and a covariance matrix that is diagonal with standard deviations ![\sigma=[0.4, 0.4, 0.4]](https://www.displayr.com/wp-content/ql-cache/quicklatex.com-dded7536d5b10d7ece9800edda1edcc2_l3.png "Rendered by QuickLaTeX.com")
. I draw 1000 samples from this distribution, as partially shown in the table below:
| Draw | Parameter 1 | Parameter 2 | Parameter 3 | Dp-error |
|---|---|---|---|---|
| 1 | 0.25 | -0.73 | 0.67 | 1.45 |
| 2 | 1.14 | -0.67 | 0.67 | 1.90 |
| 3 | 0.69 | -0.50 | 1.23 | 1.92 |
| ... | ... | ... | ... | ... |
| 1000 | 1.15 | -1.67 | 0.57 | 2.82 |
DB-error is estimated as the mean DP-error, which is 1.90 in this example. Another more computationally efficient but complicated way of computing the integral using quadrature exists1 but is beyond the scope of this article.
Read more handy How To guides, or check out the rest of our blog!
References
1 Christopher M. Gotwalt, Bradley A. Jones & David M. Steinberg, “Fast Computation of Designs Robust to Parameter Uncertainty for Nonlinear Settings,” Technometrics (2012) 51:1, 88-95, DOI: 10.1198/TECH.2009.0009