Attribute MSA Report in Excel

How Do I Create an Attribute MSA Report in Excel Using SigmaXL?

Attribute MSA is also known as Attribute Agreement Analysis. Use the Nominal option if the assessed result is numeric or text nominal (e.g., Defect Type 1, Defect Type 2, Defect Type 3). There must be at least 3 response levels in the assessed result, otherwise it is binary.

1. Open the file Attribute MSA Nominal.xlsx. This is a Nominal MSA example with 30 samples, 3 appraisers and 2 trials. The response is text Type_1, Type_2 and Type_3. The Expert Reference column is the reference standard from an expert appraisal. Note that the worksheet data must be in stacked column format and the reference values must be consistent for each sample.


2. Click SigmaXL > Measurement Systems Analysis > Attribute MSA (Nominal). Ensure that the entire data table is selected. Click Next.


3. Select Sample No., Appraiser, Assessed Result and Expert Reference as shown. Check Report Information and enter Attribute MSA Nominal for Product/Unit Name. Select Percent Confidence Interval Type Wilson Score:

   
   

   
   
   
4. Click OK. The Attribute MSA Nominal Anlaysis Report is produced.

Tip: While this report is quite extensive, a quick assessment of the attribute measurement system can be made by viewing the Kappa color highlights: Green - very good agreement (Kappa >= 0.9); Yellow - marginally acceptable, improvement should be considered (Kappa 0.7 to < 0.9); Red - unacceptable (Kappa < 0.7). See Attribute MSA (Binary) for a detailed discussion of the report graphs and tables. Here we will just look at Kappa.

Attribute Agreement Report:

Attribute Effectiveness Report:

Fleiss Kappa statistic is a measure of agreement that is analogous to a correlation coefficient for discrete data. Kappa ranges from -1 to +1: A Kappa value of +1 indicates perfect agreement. If Kappa = 0, then agreement is the same as would be expected by chance. If Kappa = -1, then there is perfect disagreement. Rule-of-thumb interpretation guidelines: >= 0.9 very good agreement (green); 0.7 to < 0.9 marginally acceptable, improvement should be considered (yellow); < 0.7 unacceptable (red). See Appendix Kappa for further details on the Kappa calculations and rule-of-thumb interpretation guidelines.

Fleiss Kappa P-Value: H0: Kappa = 0. If P-Value < alpha (.05 for specified 95% confidence level), reject H0 and conclude that agreement is not the same as would be expected by chance. Significant P-Values are highlighted in red.

Fleiss' Kappa LC (Lower Confidence) and Fleiss' Kappa UC (Upper Confidence) limits use a kappa normal approximation. Interpretation Guidelines: Kappa lower confidence limit >= 0.9: very good agreement. Kappa upper confidence limit < 0.7: the attribute agreement is unacceptable. Wide confidence intervals indicate that the sample size is inadequate.

Fleiss Kappa Overall is an overall Kappa for all of the response levels.

Fleiss Kappa Individual gives Kappa for each response level (Type_1, Type_2 and Type_3). This is useful to identify if an appraiser has difficulty assessing a particular defect type.

This is a very good attribute measurement system, but Appraiser C is marginal so a refresher would be helpful.