Applying POD to Improve Bolt Hole Eddy

Applying POD to Improve Bolt Hole Eddy

Current Inspection

Holly LEMIRE, P.R. UNDERHILL, T.W. KRAUSE, Royal Military College of Canada, Kingston, Ontario, Canada

Abstract. Probability of Detection (POD) of fatigue cracks and electric discharge machined (EDM) notches in 7075-T6 aluminum was evaluated for a bolt-hole eddy current inspection system. The POD study involved simulated bolt holes in coupons representative of wing areas on CC-130 and CP-140 aircraft. The data were obtained from 24 inspectors who inspected 468 coupons that contained a set of coupons with 45 EDM notches and 72 laboratory-grown fatigue cracks located at the inner surface corner of the bi-layer structures. A single point calibration based upon a 0.76 mm (0.030 inches) deep corner notch set to 20% of screen height (SH) was utilized. Large variability in detection capability between inspectors was observed. A portion of this variability was attributed to significant background noise on the order of 5% screen height (SH). Normalization of each inspector’s data to signal amplitudes from notches at twice the depth (1.5 to 1.6 mm (0.060 inches)), close to 40% SH, reduced the relative variations and resulted in a 10-20% improvement in POD. These results indicate that calibration with signal responses from notches larger than that currently used would improve the POD for this inspection system and demonstrate the potential of using POD data to improve inspection systems.

1. Introduction

Eddy current inspection is a widely used Non Destructive Test (NDT) method for determining the size of cracks in metallic structures. In particular, the technique is extensively used in the aircraft industry to check for cracks in bolt and rivet holes. An important parameter describing such methods is the Probability of Detection (POD) which specifies the probability of detecting a crack of a certain minimum size a certain fraction of the time. Methods for determining POD are described in MIL Handbook 1823 [1]. POD information is used in risk assessments, drives scheduled maintenance and determines

inspection intervals for damage tolerant structures within the aerospace industry. Improving POD by optimizing the inspection system can result in significant economic benefits by reducing the frequency of inspections. In addition to providing data which afford a means by which the inspection system can be evaluated, POD studies potentially allow for methods by which the system can be improved. In particular, it can be used to identify sources of variation and noise that may compromise an NDT system’s effectiveness.

A recent round robin study was performed within the Canadian Forces (CF) NDT community to determine the repeatability of eddy currents in measuring crack sizes in test specimens. These test specimens were representative of bolt holes in specific wing areas of the CC-130 Hercules and CP-140 Aurora aircraft, which undergo frequent bolt-hole eddy current inspection. The inspectors each inspected 468 test coupons using standard DND practices [2]. The coupons consisted of two aluminum alloy plates, bolted together, with a central hole of 4.62 mm (0.182 inches) diameter. The top plate thickness was 7.94 mm

1

www.ndt.net/index.php?id=8327(0.3125 inches) thick, representative of a structural wing member, and the bottom was 2.29 mm (0.090 inches), representative of wing skin. The test coupon set included 72

laboratory-grown cracks of various sizes, 45 EDM notches of different sizes and 351 blank coupons containing no defects. When a defect was present, it was located in the bottom of the top plate and oriented as shown in Figure 1.

Figure 1: Top test coupon and probe configuration with EDM notch orientation and geometry.

The results of this round robin showed substantial spread in the measurements for each crack. The results for the EDM notches are shown in Figure 2 and those for the fatigue cracks are shown in Figure 3. In addition to the broad spread in the data Figure 3 shows that there is a significant difference in the response between the fatigue cracks and the EDM notches. This is particularly troubling because it suggests that POD analyses based on the more easily constructed EDM notches are not directly applicable to fatigue cracks. One possible source of this disparity that could be readily identified was the

difference in the shape and aspect ratio between the EDM notches and the fatigue cracks. The geometry of the EDM notches was triangular with aspect ratio (length-to-depth) of 1.0. The geometry of fatigue cracks was nominally quarter elliptical with an average aspect ratio of 1.7 with the longer side being the “length” as shown in Figure 1. However, this difference alone seemed unlikely to be able to explain the large variation seen within the data sets.

2

Figure 2: Signal response for EDM notches as a function of depth for the 24 inspectors.

Figure 3: Signal response for EDM notches as a function of depth for the 24 inspectors. The grey

line shows the best fit line to the EDM notches.

This study sought to understand the variation within and between the two data sets through a POD analysis and to improve the eddy current technique so as to reduce the

scatter in the data. Laboratory measurements of noise, with the same inspection system and a subset of the same specimens, were made to assess the minimum possible noise levels. Recalibration of the POD data set using signals from EDM notches at higher amplitudes was performed. The results suggest that normalization, incorporating additional calibration measurements on larger notches, with signal amplitudes that remain less than saturation, would improve the current system’s detection capability.

2. Experimental Approach

The EDM notch lengths and depths were measured using a linear microscope and ranged from 0.03-2.13 mm (0.001-0.084 inches) [3]. The fatigue cracks were measured by examining a silicon replica of the crack surface, which was applied to the coupon under tensile stress so that the crack would be opened. The silicon replicas were then measured

3

using a low power stereo microscope. The depths of the cracks ranged from 0.05 -1.27 mm (0.002 - 0.055 inches) and the lengths from 0.15 - 2.69 mm (0.006 - 0.106 inches). The average aspect ratio was 1.7 [3]. The distribution of the number of lengths in a given interval up to a length of 1.5 mm (0.059 inches) was comparable between EDM notches and fatigue cracks. However, the set of fatigue cracks contained more cracks with lengths greater than 1.5 mm.

As in the original study, a Staveley Nortec 2000D Eddy Current (EC) instrument and a Stavely RA 2000 (also called the RA 2000 gun) rotating probe scanner, with a split-D differential coil with 2 mm (0.079 inches) diameter as represented in Figure 1 was used. At the beginning of a series of measurements, the signal was balanced to appear at the centre screen position and lift-off was set to read from right to left on the flat side of the aluminum calibration piece. Then calibration measurements were performed on a general purpose aluminium calibration block VM 308-C1A. The bolt hole used in the calibration block was 4.76 mm (3/16 inch) in diameter. The calibration notch was a corner notch 0.76 mm (0.030 inches) deep by 0.76 mm (0.030 inches) long by 0.13 mm (0.005 inches) wide. Using this block, the gain setting was adjusted to read 2 graticules. The specimens in the test set were then inspected by monitoring the eddy current screen display and recording the maximum signal amplitude, when an indication was obtained. All data were recorded in graticules, where a graticule is 1 screen division or 10% of screen height. Signal amplitudes were measured relative to the centre screen position, limiting maximum recorded amplitude to 5 graticules (50% of screen height).

Background noise data were gathered in the laboratory by a single inspector with 117 separate measurements using the same equipment and a portion of the sample set used in the original inspections. Data were obtained by taking measurements on 45 of the coupons containing an EDM notch and 61 containing a fatigue crack but on the side opposite from the discontinuity location. An additional 11 blank coupons were also included in the measurement set. Measurements were made with the same procedure as used in the bolt hole eddy current study with set-up on the 0.76 mm (0.030 inches)

calibration notch to 2 graticules. By plotting the noise data using a normal probability plot, the noise was shown to be normally distributed with values that ranged from 0.2 to 0.9 graticules. Analysis of the noise data produced a mean and standard deviation of 0.48 and 0.14 graticules, respectively.

3. Results and Discussion

In order to understand why the apparent sensitivity is greater for the fatigue cracks than the EDM notches, it is necessary to consider the physics of the eddy current method. While depth (see Figure 1) is the variable used by structural engineers for condition

assessments, length was found to provide a more consistent signal response between EDM notches and fatigue cracks [4] for this particular configuration. The skin depth, δ, may be obtained from [5];

ρδ=

μπf (1)

where ρ is the resistivity, μ is the permeability and f is the operating frequency of the eddy current instrument. The resistivity for 7075-T6 aluminum is 5.3 μΩ⋅cm (conductivity of 32% IACS) and the operating frequency was 400 kHz. This produces an estimate of the skin depth of 0.18 mm (0.007 inches). Sensitivity to depth is nominally observed up to 3δ or 0. mm (0.021 inches) for the frequency used here. Hence, the probe is unable to distinguish between a crack 0. mm deep and a deeper crack.

4

Similarly, the probe can only respond to cracks whose length (Figure 1) is less than the effective probe diameter. An empirical relationship that takes into account the spread of eddy currents beyond the coil diameter, Dprobe, producing an effective probe diameter, Deffective, can be described as [4]:

Deffective = Dprobe + 4δ (2)

Combining probe diameter of 2 mm (0.079 inches) with the skin depth calculated above, an effective probe diameter of 2.7 mm (0.107 inches) is obtained.

Combining these two observations, it can be seen that the probe output will only be correlated to that portion of the crack that is within a depth of 0. mm and up to a crack length of 2.7 mm. The probe is not sensitive to the depth (Figure 1) of the crack beyond 0. mm. Instead, depth must be inferred from the aspect ratio of the crack. Since it was observed that the aspect ratio of the EDM notches was 1.0 and that of the fatigue cracks was 1.7, it is not surprising that the two do not agree when the signal response is plotted against depth. Separate laboratory measurements of amplitude response as a function of length (performed at lower gain setting to reduce the amount of saturation) are shown in Figure 4. The sensitivity of the instrument to the two types of cracks is now identical within experimental accuracy. This means that the data sets can be used interchangeably when plotting a-hat vs a data.

4. Normalization/POD Analysis

The normalization process standardizes a data set so that the large variation in the signal responses is greatly reduced. In order to reduce the large spread in the data, one-point, two-point, three-point, four-point and trend-line normalizations were investigated. The R2 value from each type of normalization is used to give an indication of the reduction in variability. These values are given in Table 1.

Since the spread of data was wider for larger EDM notches, use of larger notches for the normalization were expected to give the best results. However, if too large an EDM notch was chosen, the signal was saturated for some inspectors. Appropriate EDM notches were chosen to maximize size and avoid saturation levels. The normalization points

corresponding to 1.50 mm (0.059 inches) and 1.60 mm (0.063 inches) were found to satisfy

Figure 4: Signal response up to saturation (horizontal dashed line) as a function of length for fatigue cracks and EDM notches. Vertical dashed line shows effective probe diameter (Deffective).

5

the above criteria to the greatest extent possible. A correction factor was obtained for each inspector based on the normalization points as follows:

CorrectionFactor@0.059\"=

CorrectionFactor@0.063\"=

4.0

Inspector'sMeasurement@calibrationpt 3.8

Inspector'sMeasurement@calibrationpt

Average was taken for two- point normalization

The correction factors above were averaged for each inspector for the two-point normalization. The value for each correction factor calculation (i.e. 4.0 and 3.8) was obtained from the average signal response from the normalization points. Once the correction factor for each inspector was obtained, it was applied to normalize that

inspector’s data set. All saturated data were removed prior to this step in order to avoid a skewed result.

For more than two data points, a linear best fit using the measured EDM notch thickness from well spaced points was obtained and a corresponding linear transformation of each inspector’s data was applied to map the data back to an intercept of zero and a constant slope. Neither the 3 nor the 4 point normalizations resulted in an improvement in R2.

Table 1: R-squared values from a quadratic best fit to combined data set.

Data R Squared Value (Variability) Raw Data 0.61 Adjusted to 1.60 mm Notch 0.71 Adjusted to 1.60 mm Notch 0.75 Two Point Normalization 0.77 Three Point Normalization 0.72 Four Point Normalization 0.74 Trend-line Normalization 0.81

As shown in Table 1 the trend-line normalization, which used a linear best fit to all of the EDM notch signal amplitudes followed by a corresponding linear transformation of each inspector’s data set, resulted in the greatest reduction in variability. However, this technique was not viewed as practical under normal inspection conditions due to the large number of measurements required. The next largest reduction in variability was a two-point normalization utilizing the two largest available EDM notches (1.49 mm (0.059 inches) and 1.60 mm (0.063 inches) long) for which saturation was not encountered. This choice was viewed as practical, due to the potential replacement of the single point calibration that is presently being used by measurements on two larger calibration notches. Calibration at a larger notch size would still require that the maximum size of the EDM notch signal be less than the saturation level of 5.0 graticules. The average amplitude from the 1.6 mm notch was 4.0 graticules, so this condition was met.

In order to determine the effect of the normalization schemes discussed above on the POD, three normalized data sets were constructed, one for each of the one point normalizations and one for the two point normalization. The data points used in the normalization process were removed from each of these data sets. Both the raw and normalized data sets were then analyzed using the Military Handbook 1823 Reliability software [6] to produce POD curves.

In order to perform “a-hat” vs. “a” analysis, noise measurements were required to determine the Probability of a False Positive (PFP) for a given decision threshold. As

6

described above, noise measurements were collected in the laboratory using the original equipment and coupons and these became part of the analysis. The 1823 software produced a decision threshold trade-off graph so that the trade-off between a small threshold and the chance of generating a false positive could be balanced. For comparison purposes, a PFP of 5.4%, which corresponds to a decision threshold of 0.7 graticules, was chosen. Note that this is above the average noise level.

The calculated a50 POD curves for fatigue cracks and EDM notches at an âdecision of 0.7 graticules and a PFP of 5.4% are shown for the raw data set in Figure 5. Note that the fatigue crack data is plotted in terms of crack length, not crack depth. The POD curve for EDM notches corresponds relatively closely with the POD curve for the fatigue cracks as a function of length. These curves cover the length range up to 1.5 mm (0.059 inches)) where the distribution of EDM notches and fatigue cracks are comparable.

Figure 5: As collected (Raw) probability of detection of EDM notches and fatigue cracks (FC)

(âdecision = 0.7 graticules and PFP = 5.4%).

Table 2 summarizes the a50, a90, and a90/95 POD results for the raw, single (1.5 and

1.6 mm) and two point normalizations of the FC and EDM notch data. The most improved POD occurs for the two-point and single-point 1.5 mm normalizations. Less improvement is observed for the single-point 1.6 mm normalization. For the 1.5 mm and two-point normalization the a50 fatigue crack POD shows an improvement of 20%, while the

improvement for the EDM notches is 17%. For the a90/95 point, the largest improvement (23%) is observed for the EDM notch data. The corresponding value for the fatigue crack data is 11%. These results are indicative of the improvements that may be attained by using larger calibration notches.

TABLE 2. Probability of detection results for EDM notches and fatigue cracks for raw data and data

with single and two-point normalization (Normal).

POD Analysis

EDM Notches (Raw) EDM Notches (1.6 mm Normal.) EDM Notches (1.5 mm Normal.) EDM Notches (2 Pt Normal.) Fatigue Cracks (Raw) Fatigue Cracks (1.6 mm Normal.) a50

mm (inches) 0.41 (0.016) 0.37 (0.014) 0.35 (0.014) 0.35 (0.014) 0.41(0.016) 0.34 (0.013) a90 a90/95

mm (inches) mm (inches) 0.67 ( 0.026) 0.79 (0.031) 0.55 (0.022) 0. (0.025) 0.52 (0.020) 0.60 (0.0235) 0.53 (0.021) 0.61 (0.024) 0.78 (0.031) 0.91(0.036) 0.70 (0.028) 0.85 (0.034) 7

Fatigue Cracks (1.5 mm Normal.) Fatigue Cracks (2 Pt Normal.) 0.32 (0.013) 0.33 (0.013) 0.66 (0.026) 0.67 (0.026) 0.81 (0.032) 0.81 (0.032)

The effect of the two-point normalization can be observed by comparing the POD curves of the raw data with the POD curves of the normalized data for both EDM notches and fatigue cracks. Figure 5 and Figure 6 show the effect of two-point normalization on the 50% POD and 95% confidence curves for the EDM notch and fatigue crack data,

respectively. Both the EDM notches and fatigue cracks show an improvement in POD after the two-point normalization has been applied. The EDM notch improvement is mostly concentrated on the top end of the POD curve, where as the fatigue crack improvement shows a more consistent improvement over the entire range of lengths.

Figure 5: Effect of two-point normalization on EDM notch data (âdecision= 0.7 graticules and PFP = 5.4%). Solid curves show the mean curves and dashed curves show the 95% upper confidence bound

Figure 6: Effect of two-point normalization on fatigue crack data (âdecision= 0.7 graticules and PFP = 5.4%). Solid curves show the mean curves and dashed curves show the 95% upper confidence bound

The POD results obtained from data normalization indicate that normalization of

each inspector’s data set to signals from larger notch sizes improved the POD. The bolt

8

hole inspection performed by the CF is currently calibrated to a 0.76 mm (0.030 inches) corner notch. However for this amplitude response, the in-laboratory signal-to-noise level was measured as 4:1. In contrast, the notches used for the single- and two-point

normalization, with twice the signal amplitude, would each produce a signal-to-noise ratio closer to 8:1 (equation 5), reducing the relative effect of noise on the calibration signal amplitude. For POD data, a two-point calibration using both a 1.5 and 1.6 mm deep notch together was statistically equivalent to that obtained using the 1.5 mm notch alone.

However, both were better than that obtained by normalizing to the 1.6 mm notch. There is no reason to believe that the data from the 1.5 mm notch should be better than that for the 1.6 mm notch. It is believed that this discrepancy is due to the natural variation in the data and may well not be reproduced if the process were repeated. On the other hand, there is a statistical reason to believe that the averaging of the response from the two cracks would show less variation. Hence, because of the stability offered by averaging two points compared to one, it is suggested that the two point normalization be adopted.

5. Summary

The effects on probability of detection of renormalizing bolt hole eddy current data

from 24 inspectors using single and paired EDM notches that were more than twice the size of that used in the original calibration were examined. The original calibration was

performed using a 0.76 mm deep corner notch set to 2 graticules (20% screen height). Post in-laboratory noise measurements however indicated that a mean background noise level of 0.5 graticules was present. At least a portion of the large variability in inspection results observed between the 24 inspectors could be attributed to this relatively large background noise variation and its potential effect on calibration measurements. Normalization of each inspector’s data using the average signal amplitude from a pair of notches, 1.5 and 1.6 mm deep, or the signal from a single 1.5 mm deep notch, to an amplitude response of 4

graticules (twice the original calibration amplitude), reduced the relative variations between inspectors and resulted in a 10-20% improvement in POD. The improvement in POD using a single 1.6 mm deep notch was not as good as that obtained using a 1.5 mm notch. This suggests that a higher confidence in improved POD may be obtained with two notches rather than just one. These results suggest that the use of notches at least twice as large as that currently used, with calibration signal amplitudes on the order of 4 graticules (80% SH), would improve POD for this inspection system.

6. Acknowledgements

The authors would like to acknowledge Dr. Abbas Fahr and Muzibur Khan from the

National Research Council of Canada for allowing access to data and coupons from the Generic Bolt Hole Eddy Current Project. This work is supported by the Natural Sciences and Engineering Research Council of Canada and the Academic Research Program at the Royal Military College of Canada.

References

[1] Department of Defense Handbook, Nondestructive Evaluation System Reliability

Assessment, MIL-HDBK 1823, Feb 2007 (Draft) [2] [3]

GEN-74-E Rev 2, Canadian Forces Eddy Current Inspection – General Bolthole Technique for Metallic Structures, 8 Sept 2003.

H. Lemire, “Improving Probability of Detection for Bolt Hole Eddy Current Inspection”, M.Sc. Thesis, Royal Military Colege of Canada, February 2008.

9

[4] H. Lemire, T.W. Krause, M. Bunn and D.J. Butcher, “Variables Affecting Probability

of Detection in Bolt Hole Eddy Current Inspection”, Review of Progress in Quantitative Nondestructive Evaluation, Vol. 28 edited by: D.O. Thompson and D.E. Chimenti, to be published. [5]

V.S. Cecco, G. Van Drunen, G., Sharp, F.L., Eddy Current Testing Manual on Eddy Current Method, Vol. 1, Chalk River Nuclear Laboratories, November 1981, p.11, 13, 14, 48 and 66.

[6]

Charles Annis, P.E. (2008), \"Statistical best-practices for building Probability of Detection (POD) models\" R package mh1823, version 2.5.4, http://StatisticalEngineering.com/mh1823/.

10