Data Bias : Cliff Stamp


Performance measurement basis

In Measurement Consistency 1 a procedure was outlined for how to compare the performance of two knives in a meaningful way which takes into account the fact that all performance measurements have a degree of inconsistency. In Physical Consistency 2 this discussion was continued by exploring the physical reasons for inconsistencies in measurement and how to minimize their effect. The following article deals with an often mentioned topic on various internal forums when discussing reviews and tests of knives which is bias.

Definition

The common english definition (Merrian Webster / online):

Bias
a personal and sometimes unreasoned judgment

This is often the case when individuals being less than perfect let their emotions effect how the data is presented to support their conclusions. This however comes to the working definition which is used in actual research

Bias
Deviation of the expected value of a statistical estimate from the quantity it estimates

The question is then very simple, is there a systematic inconsistency in the data which will bias it so as to induce a false conclusion? Note the link here between the data and the conclusion, this is very important because that is the only way to judge a bias, what exactly is being concluded?

An example of data bias : CATRA testing of BUCK Ionfusion blades

Buck Knives have an Ionfusion line of cutlery where the blades are titanium nitrided on one side for an effective 80 HRC, much harder and more wear resistant than the blade steel. This coating is designed to enhance edge retention as it forms a part of the edge itself. The Ionfusion knives were tested on a CATRA machine with the following results 3 :

...catra testing for the ion fusion showed initial results of upwards to 200 times edge life.

However Buck realized there was a problem with the CATRA result, specifically there was a systematic bias, as the machine cutting ignored lateral loads and others causes of edge deformation :

We realized that edge retention is lost through wear and also through flattening by impacting the blade edge in use. The 420hc is only 58 RC and will flatten if impacted.

Thus they decided to test the edge by having individuals cut various materials by hand and came up with a more representative data set :

We decided to stick with the 6-8 times edge life as a claim because it was a more accurate reflection of what a user will encounter.

Note that even though the hand gathered data was no where near as precise as the CATRA data it was unbiased and much more accurate. The lack of CATRA testing in general to take these microloads into account during cutting by people has lead to individuals such as Roman Landes to come up with a way to measure this directly4 and determine the edge stability5.

Recap

Bias in terms of data simply means there is a systematic deviation present which makes the sample not representation of the population. It has to be made very clear that a sample can be biased or not depending on what exact population is trying to be represented. In order to judge data as biased or unbiased the conclusion has to be carefully regarded.

Consider for example if a Spyderco Military in S30V and a Buck 110 in 420HC were compared for edge retention. As these two knives are not identical in geometry such a comparison will not be biased in terms of determining if either S30V or 420HC has better edge holding, however it is a perfectly unbiased way to determine if that Military or 110 has better edge holding.

References

1: Measurement Consistency : Cliff Stamp, cutleryscience.com, 2007.

2: Physical Consistency : Cliff Stamp, cutleryscience.com, 2007.

3: CATRA Edge Testing Results : Chuck Buck, bladeforums.com, 1999.

4: Messerklingen und Stahl, 2. Auflage, Wieland Verlag, Bruckmühl, Germany. Dr. R. Landes, 2006

5: Edge Stability : Review : Cliff Stamp, cutleryscience.com, 2007.

Comments

Comments can be emailed to cliffstamp@[REMOVE]cutleryscience.com. Delete the [REMOVE] to respond.


Written: August, 2007 Copyright (c) 2007 : Cliff Stamp
Up