Proficiency tests prove the competence of calibration service providers, say Drew Barss and Dr Dimitri Vaissiere.

Instrument calibration is a standard element of most ISO 9001 quality systems for industrial manufacturers. Some industries, like life sciences, food and beverage, and chemicals, place particular emphasis on calibrations due to regulatory compliance and the potential of safety incidents.

However, conformance to the ISO 9001 requirements for calibration does not guarantee that the results are accurate. Certification to ISO 9001 only checks the execution of calibrations at the required interval. Accreditation using ISO 17025 goes further by proving a service provider’s competence to perform calibrations that are accurate and reliable. This standard requires a service provider to participate in a proficiency test program for this very purpose.

Poor calibrations can have significant impacts on an industrial manufacturer. An inaccurate meter could lower plant yields by allowing the carry through of high-value components into low-value product streams. Inaccurate readings could also escalate energy consumption. The accuracy of process information enables the control of product quality. Process measurement errors could lead to off-spec products, with the potential for rework or losses. One way to quantify the costs of poor calibration is to convert product losses to dollar terms. For example, an error of 0.3% on the diesel and gasoline flow meters would cost an average refinery in Europe over €20 million per year. An organisation seeking to optimise diesel production across multiple sites could suffer even greater losses because poor quality information may drive the optimisation algorithms away from the most profitable solution. The availability of high-quality plant information is critical for high-level optimisation programs.

Endress+Hauser runs a proficiency test for our calibration service providers around the world, with more than 30 centres participating in each of the last four years. The test covers flow, pressure, temperature, pH, and conductivity meters. It has been running annually since 2012 and provides a holistic examination of the entire customer experience of our service centres.

Using proficiency testing, Endress+Hauser enables our service centres to qualify for accreditation according to ISO 17025. The program produces statistical results according to ISO 13528 and identifies outliers that require corrective action. The corporate quality organisation of Endress+Hauser manages the proficiency test program and the execution of all corrective actions.

Calibration – What is it and why is it important?

Industries use a wide variety of devices to measure and control process conditions. Each instrument is bought and installed with a claimed accuracy. Over time, however, instrument readings may become less reliable. Several factors contribute to this deterioration, like drift, the environment, process changes or changes to the control loop itself.

Calibration is a vital component of the ISO 9001 quality systems of most industrial and manufacturing plants. Its purpose is to verify the accuracy of a measuring instrument. During calibration, a technician compares the device against a known standard and, if necessary, corrects any errors. A process of calibration should return a reliable and trustworthy instrument to the process, but poor calibration could add errors instead of correcting them.

Calibration is only effective if the competence of the service provider is verified. Proficiency testing is a program designed to achieve this by comparing the results of calibration service providers with each other and with a known standard. Any service provider that participates in a proficiency test program can demonstrate their competence and give their customers confidence in the quality of their results. In addition, it allows them to meet a requirement of ISO 17025 accreditation, which specifies that a calibration service provider must participate in a proficiency test program wherever possible.

The high cost and impact of poor calibration

Calibration is more than just a quality systems issue. There can be significant implications for having inaccurate instruments used to control plant conditions. Some

of these impacts may be immediately visible like safety or environmental incidents, but others may not be immediately apparent.

Instrument accuracy is generally assumed if calibrations are performed on schedule, but this assumption is false if the calibration is of a low standard. Poor calibration does not cause a device to fail, but it introduces errors in process measurement. In other words, the instrument may end up less accurate after calibration than it was before. Calibration errors are small enough to be invisible in daily operations because the instrument does not show any obvious signs of failure. But substantial losses may occur before the instrument error is identified and resolved.

The Life Science Industry and Food and Beverage applications are set up, for example, to maximise production of high-value products. Inaccurate meters may result in the loss of some high-value components to low-value streams. This reduction in yield affects the profitability of the processing plant. Energy consumption is another operating cost that can be elevated by inaccurate meters. Using excess energy is costly from a financial and sustainability perspective. Inaccurate meters have the potential to result in off-spec products. Once again, this may not be visible until product batching, with the unfortunate consequences of downgrade or reprocessing. Advances in the Industrial Internet of Things (IIoT) and optimisation systems also place a demand for high-quality information from industrial facilities. Algorithms depend on the accuracy of process measurements to optimise production plans and process conditions across a single site or multiple sites. Optimisation unlocks great economic value, but the opportunity can be lost due to inaccurate plant measurements caused by poor quality calibration.

The availability of regular and high-quality information from proficiency testing also creates other opportunities. Endress+Hauser can use this information to optimise calibration frequencies, which reduces operating costs without impacting on instrument performance.

Converting meter inaccuracy to dollars

As one example of quantifying the impact of inaccurate process instruments, we can convert product losses to a monetary value. There are 85 operating refineries in Europe with a total capacity of 14.5 Mb/d. This equates to an average refinery capacity of 170 kb/d. European refineries convert approximately 40% of crude oil into diesel, which means that an average refinery produces about 68 kb/d of diesel. Let’s assume that this average European refinery has an error in its diesel flow meter of 0.3%. That error would equate to 32,433 l/d. At an average diesel price of €1.1, the instrument error is costing the refinery €13 million per year. A similar error in the gasoline meter would cost the refinery €7.4 million, bringing the total to more than €20 million per year. Inaccurate instruments have the potential to make substantial impacts on the financial performance of an industrial site, making calibration a critical issue for consideration.

Are compliance and certification enough?

One of the primary measures for quality performance in any industrial facility is compliance to ISO 9001, or derived industry specific standards. Obtaining and maintaining ISO 9001 certification is rightly viewed as a commitment to quality and an indication that the organisation has systems and procedures in place that are followed by employees. An ISO 9001 system stipulates the performance of calibrations, including record retention. However, even a perfect record of compliance with ISO 9001 calibration requirements leaves some questions remaining:

• How do you know if the calibration is accurate?

• How can you be sure that the person or organisation performing the calibration is competent to do so?

Outsourcing calibrations to a third party with ISO 9001 certification does not solve the problem of accuracy. Calibration services is a natural target for industrial plants seeking to lower their overheads and minimise their company headcount. There is a temptation to go with a low-cost service provider, assuming that their ISO 9001 certification provides a level of confidence in their capability. They may have systems and procedures in place and may keep track of training records. However, questions about competence also apply to calibration vendors. Only third-party verification demonstrates that a service provider is performing accurate calibrations.

Low cost calibrations compromise the investment in high-quality instrumentation

An industrial facility is a substantial capital investment in terms of engineering, equipment, human resources and logistics. The highest investment protects the most critical parts of a plant. Redundant equipment keeps a plant running even if the primary equipment fails. Process instrumentation follows a similar pattern. Critical loops involving safety, environment, product quality, product volume and energy consumption attract the highest investment and the best quality instrumentation.

Unfortunately, poor calibration has the potential to compromise this entire investment. A low-cost approach to calibration fails to recognise the importance of competence checks and results in process measurement errors that are unknown. However, proven calibrations that are externally verified keep an instrument performing at the expected and desired level. It doesn’t make sense to invest in high-quality instrumentation but then fail to keep it performing at the necessary level of accuracy.

Low cost calibrations compromise the brand

Global organisations have a vested interest in the consistency of their products all around the world. Customers from Asia to South America should enjoy the same consumer experience even though they are receiving the product from different manufacturing sites. Measurement errors due to poor calibration may introduce a level of variability in product quality, thus impacting on the end-user experience. When the customer is dissatisfied with product quality, it undermines the company’s brand in the market.

It is not only the industrial company brand at risk when instruments are poorly calibrated. The instrument supplier also suffers harm to its brand. Third-party calibration providers with a low-cost approach have no vested interest in the reputation of the instrument. But this is not true of the instrument manufacturer. Their brand reputation is tied to the quality of calibrations and the ongoing performance of their devices.

The Endress+Hauser global proficiency test program

Proficiency tests check whether a calibration service provider can find an error in a specially prepared standard meter. In other words, the instrument has a deliberate bias inserted, but the service provider does not know how big the error is or in which direction.

The service centre performs its calibration and submits its results for comparison with other centres. Statistical analysis of results shows whether any service provider is outside the acceptable uncertainty range. Those that fall within the acceptable range demonstrate their competence to perform calibrations. Results that are outside the acceptable limits allow for investigation and corrective actions to improve the performance.

In the case of Endress+Hauser, it is the corporate quality organisation that develops and runs the proficiency test program. The program covers flow, temperature, pressure, pH, as well as low and high conductivity meters. Meters and buffer standards are prepared and sent to our calibration service providers in 36 countries around the world. Each provider performs its calibration tests and submits its results to the corporate quality organisation.

What makes this program unique is that it tests the entire customer experience of a calibration service provider. They receive a meter just like they would from any other customer. They have a specific time frame to perform the tasks before returning the meter. If there are any corrective actions, they must be closed out with the corporate quality organisation just like any other customer complaint.

What differentiates the Endress+Hauser approach?

Proficiency testing has a clear benefit of verifying the competence of service providers to perform calibrations. But some proficiency tests have more advantages than others. There are specific features that set the Endress+Hauser proficiency test apart:

1. Endress+Hauser has run a proficiency test through its service centres every year since 2012. The program has been refined and improved over its history to maximise its effectiveness. In effect, Endress+Hauser has demonstrated competence to develop and run proficiency tests to international standards.

2. The Endress+Hauser corporate quality organisation encourages every calibration service provider in the Group to obtain ISO 17025 accreditation. In fact, as a whole, the organisation has 45 parameters accredited for laboratory and 41 for on-site calibration across 20 calibration centres around the world.

Participating in a proficiency test is a central requirement for this accreditation.

As such, Endress+Hauser is providing a primary requirement for ISO 17025 to all the calibration service centres in the Group.

3. External proficiency tests can only produce results that show whether a calibration service provider is competent or not. They do not have the expertise or resources to find out what the problems are and how best to rectify them – neither do they have any vested interest to do so.

On the other hand, Endress+Hauser has significant motivation to ensure all their service centres are competent and performing well. All the internal resources of the organisation are available to help resolve corrective actions.

The quality group provides certified training for equipping new technicians and refreshing experienced technicians. They also develop and change SOPs, perform standardised uncertainty calculations, and provide standardised tools. Our product specialists are available to resolve technical issues related to the meters themselves. Engineering resources provide comprehensive root cause analysis and solutions.

4. Endress+Hauser publishes accuracy specifications with each instrument model. This promise of quality instrumentation drives the same brand commitment to quality calibration services. Third-party proficiency tests can only benchmark performance between the participating service centres and against their own claimed uncertainty.

But Endress+Hauser has a global expectation of uncertainty based on the measurement type and test equipment. This global expectation sets a higher standard for calibration performance than can be achieved with a third-party supplier.

These factors differentiate Endress+Hauser from other proficiency test service providers who have an emphasis on the statistical results but no vested interest in the performance of the instruments.

Proficiency testing as a global best practice

The International Laboratory Accreditation Cooperation (ILAC)8 is the worldwide body, associated with ISO standards, responsible for the accreditation of calibration service providers and proficiency test providers. This body uses ISO 17043 to verify that a proficiency program meets the requirements to test the competence of a calibration service provider.

According to ILAC, many calibration laboratories operate in isolation without opportunities to identify errors or bias in their data through comparison with others. This isolation leaves their clients uncertain as to their results and having no external verification of their competence.

ILAC highlights several benefits of a proficiency test program9, such as the one conducted by Endress+Hauser.

Proficiency test programs confirm the competence of technicians and help to identify testing or measurement problems. They compare operator capabilities, satisfy regulators and meet the conditions of accreditation.

Proficiency tests also enable service providers to improve their performance and educate their staff. All these factors help to instill confidence in staff, management and external users of laboratory services.

Conclusion

Proficiency testing is a globally recognised best practice for proving competence in calibration services. ISO 9001 systems ensure that calibration happens on a regular frequency and may satisfy regulatory requirements. But, proficiency testing offers the added benefit of proving that a service provider is competent to perform calibrations.

Failure to prove competence in calibration can lead to inaccurate instruments in an industrial manufacturing facility. While drift in the device measurement may be undetected, the consequences will be felt. Measurement error could result in lower yields, reduced energy efficiency or off-spec products. The consequences of poor calibrations can be quantified in dollar terms, which demonstrate the importance of raising the standard of calibrations in industry.

Besides the financial losses associated with inaccurate instruments, there is also the potential of damage to brand reputation. Product quality may differ slightly from one manufacturing site to another, leading to customer dissatisfaction.

The Endress+Hauser proficiency test program challenges the entire process to deliver calibrations and the associated customer experience from all our service centres worldwide. The program demonstrates their competence to perform calibrations and generates corrective actions where needed. Endress+Hauser’s corporate quality organisation administers the program and evaluates the outcome of corrective actions.

A low-cost approach to calibration in the industrial manufacturing industry may satisfy the certification requirements of ISO 9001, but cannot prove the accuracy of results. Endress+Hauser offers a proficiency test program that tests the competence of our calibration service centres worldwide and provides confidence to our customers.

Drew Barss, Head of Department, Service Industrialisation, Endress+Hauser Group Services.

Dr Dimitri Vaissiere, Expert Data Scientist, Service Innovation, Endress+Hauser Group Services.

Whitepaper link: https://www.in.endress.com/en/endress-hauser-group/Case-studies-application-notes/proficiency-testing

______________________________________________________________________________________________

For a deeper dive into the dynamic world of Industrial Automation and Robotic Process Automation (RPA), explore our comprehensive collection of articles and news covering cutting-edge technologies, robotics, PLC programming, SCADA systems, and the latest advancements in the Industrial Automation realm. Uncover valuable insights and stay abreast of industry trends by delving into the rest of our articles on Industrial Automation and RPA at www.industrialautomationindia.in