Validation Value

Medical device manufacturers that use manufacturing or test instruments that rely on software may be required by federal law to validate that software. This may include software that is embedded as part of the instrument, or external, including off-the-shelf, software that interacts with the instrument. This article does not address software that is embedded in a medical device; it relates only to software or software embedded in instruments used by the medical device manufacturer (MDM) in the design, development, manufacture or control of quality of the device.

 

Often, there is little information available from the instrument manufacturers to assist their users in their need to validate the instruments for their intended use. This creates an opportunity for system and instrumentation manufacturers to stand out in their competitive markets by assisting their users with their validation needs. 

 

Keep in mind validation costs can easily exceed the purchase cost of the instrument or software. Whatever the suppliers of software-controlled instruments can do to reduce their customers’ cost of validation, therefore, reduces the overall cost of use of the system. Today, this can be a marketing advantage. Eventually it will become a necessity to stay competitive in the medical device market.

 

Regulatory Background

 

MDMs are bound by a specific federal regulation that refers to software found in manufacturing and quality systems. This regulation, found in the Federal Code section on medical device production and process controls, states:

 

“(i) Automated processes. When computers or automated data processing systems are used as part of the production or the quality system, the manufacturer shall validate computer software for its intended use according to an established protocol. All software changes shall be validated before approval and issuance. These validation activities and results shall be documented.” 21 CFR 820.70 (i)

 

This is not to be confused with the regulations relating to software embedded in the medical devices themselves. This regulation relates to all other software that is used in the development, manufacture, or control of quality of the device.

 

The FDA has published the General Principles of Software Validation; Final Guidance for Industry and FDA Staff – January 11, 2002 (commonly referred to as the GPSV) to expand on the FDA’s interpretation of all legislated software-validation requirements relating to medical devices. 

 

Does It Fall Under the Regulations?

 

MDMs should inventory all of the software that is used in a production, research, engineering, sales, or service facility. That inventory should include the “intended use” for each software item. If the software item is used to monitor or control part of the manufacturing process, design and development process, or any other part of the quality system, it falls within the regulation. Software to be considered includes that which is:

 

• Used in the design and development of the medical devices – such as CAD systems, compilers, test software, simulators, code generators, etc.Used in the manufacture of the medical devices – such as embedded machine tool software, machine vision quality control, robotics, statistical process control software, etc. 
• Used in the creation, maintenance, and control of quality data about the medical devices, such as complaint-handling systems, lot-tracking systems, training systems, QC systems, patient-tracking systems, etc. 
• Used in the creation, maintenance, reporting, validation and storage of electronic records and electronic signatures.

 

Unregulated software can fall under the requirements of 820.70(i) by its association with other regulated software. A software item that is normally considered unregulated under these guidelines might produce data that is used by regulated software. The validation requirements of 820.70(i) may then apply to that formerly unregulated software item. The validation requirements in this case could be focused on the functionality of the (formerly unregulated) software item related to the creation, storage, manipulation, and security of that data used by the regulated software.

 

Validation Obligations

 

MDMs have certain obligations for validating any software that is “regulated.” Validation involves many activities in addition to testing. Validation as recommended by the FDA’s GPSV includes the following:

 

Determination of the Software Lifecycle: Identify the software lifecycle to facilitate planning of validation activities at each phase of the lifecycle.

 

Document Intended Use: The intended use of the software item is similar to a product-level or user level requirement. Note that the software item need only be validated for its intended use.

 

Risk Analysis and Risk Management: MDMs are required to analyze how failure of a software item would impact the medical device itself. Additionally, the safety of the user, effects on the environment, and regulatory exposure should be considered as risks. Identified unacceptable risks must be managed by designing and implementing risk-control measures. Risk can be managed by reducing the severity of a hazardous situation, by reducing the probability of occurrence of the situation, or both. Sufficient risk control measures should be put in place so that the residual risk is reduced to an acceptable level.

 

Configuration-Management and Version Control: MDMs must have configuration-management plans in place for each software item. These plans should take into consideration who will make the decisions about upgrading the software, who will supply the upgrades, who will install the software, and who will take responsibility for re-validating before it goes online. 

 

Planning: Quality plans and software-verification and-validation plans detail the activities, roles, responsibilities, resources, and deliverables related to an individual software item or the collection of software items that make up an automated process. 

 

Technical Evaluations and Management Reviews: Technical evaluations should be conducted to determine if the software is technically up to the intended use before putting it online. Management reviews reference the output of the technical evaluations and also consider if the residual risk of a failing software item is acceptable prior to deployment of the software. Management also needs to consider whether the users of the software are trained sufficiently to successfully deploy the software. The management review should be considered the final gate to deployment.

 

Testing: Testing by the users of the software should focus on any risk-control measures that were implemented and then on functionality of the software whose failure could lead to severe consequences. There is little value in testing functions of the software that have little effect on the manufactured device. For electronic-record systems, focus testing on functions whose failure could result in record loss, alteration, or loss of security.

 

Traceability: A documented trace is needed to show that all of the functionality detailed in the intended use document was implemented or acquired. Any risk-control measure identified in your risk management report must be traced to where it was implemented and where its ability to mitigate the risk was tested. Finally, functional testing of the software should be traceable back to software requirements or intended use functionality.

 

Some level of interpretation of the validation guidelines is necessary by MDMs. Validation is to be proportional to risk and complexity of the software; so it is clear that risk analysis is needed for all software items. MDMs will need to address each of the components of validation. However, for embedded software that is acquired from an instrument vendor, the MDM has little control or visibility into the software development process. This takes some creativity on the part of the MDM to validate such software.

 

Validation Ideas for Instrument Manufacturers 

 

Validating manufacturing and quality-system software is burdensome for MDMs. The typical MDM user responsible for the validation is not a software engineer and is not familiar with software validation activities. Vendors of software controlled instruments might profit by selling validation packages to their MDM customers.

 

These packages might include partial validation documentation, training and/or consulting with MDM users on validating for their intended use. 

 

Validated or partially validated software for regulated medical-device manufacturer uses will be a marketing advantage.

 

Instrument manufacturers can alleviate many of the validation obligations of their customers if they document that they have followed good software engineering practices by implementing design controls and the components of validation under their control. The details of this are well documented in FDA guidelines.

 

Good software-engineering development practices depend on detailed requirements and functional specifications. Verifying the implementation of the software involves testing each requirement in the specifications for its existence and proper function. This testing should be documented in detailed test protocols. The instrument supplier cannot validate for the MDM user’s intended use, but it can verify that the software correctly implements the requirements specified.

 

Detailed requirements specifications, test protocols, test results, evaluation and review minutes dictated by a controlled design environment are likely to be considered proprietary information by instrument manufacturers. Encourage your MDM customers to audit your design process, making the validation and design control documentation available to them for viewing. This will satisfy some of your MDM customer’s obligations for validation.

 

A well-tested instrument or software item is likely to have a list of residual defects at release whose resolution was deferred for future releases. After release to the field, the users of the software might discover additional defects. These defect lists should be made available to the users and prospective users of the software or software driven instrumentation so that they can assess the risk of the defect in their intended-use environment.

 

Instrument suppliers cannot take full responsibility for the validation of their embedded software used by MDMs. Only the user of the instrument can define his intended use of the instrument. The risk associated with the use of the instrument will depend on the intended use, so that too must be the responsibility of the MDM user. 

 

On the other hand, the instrument supplier has experience with the instrument and possibly experience with how MDMs are using the instrument. Instrument suppliers can coach their MDM users in identifying and documenting intended uses. Further, instrument suppliers can leverage their knowledge of instrument failure modes to assist their MDM users in risk analysis and in identifying risk-control measures to manage the risk to acceptable levels.

 

MDM users of software-driven instruments often are not software professionals and often have no experience with software validation. Instrumentation suppliers may not have knowledge of the regulated medical device industry and may not have enough business in the medical device industry to develop in-house expertise in dealing with users’ validation needs. 

 

An outsourced validation expert can consult with the instrument supplier on improving internal software development processes to survive customer audits. Additionally the validation expert can work with customers to assist them in final validation for their intended use. The instrument supplier benefits because validation no longer seems like an obstacle in the acquisition of the instrument.

 

Case in Point

 

Take for example, an intelligent filling system that controls fill volumes, fill contents, fill rates, temperature, delivery of cleaning agents, and labeling of the containers. The system is controlled by a PC and can store multiple filling profiles for different formulations and container specifications.

 

A manufacturer of a clinical laboratory blood analyzer wants to purchase a filling system to fill bottles of reagents sold for use with the device. The machine will be used to bottle a number of different reagents for several different versions of the blood analyzer.

 

As part of good engineering practice and design control, document product requirements, performance requirements and limitations, the filling system manufacturer should specify any communications protocols between the filling system and external controllers or monitors, as well as specify which configurations of hardware and software (e.g. operating system) the filling system software was designed to work on.

 

The manufacturer should design verification test protocols for the system, specifically testing each requirement for the software. Tests should be executed and results formally documented. Additionally, a known defect list should be prepared from the results of the verification testing for any defects in the released product. Finally, make all of the above results available to the customer. Proprietary information can be made available in audits.

 

On the other hand, the medical device manufacturer is responsible for identifying the intended use(s) of the filling system. The intended use should be specific enough to know what performance requirements are expected of the system.

 

The medical device maker must consider how changes to the software embedded in the filing system’s PC will be controlled. For example, what happens if the PC running Version 1.0 software on Windows XP SP1 suffers a hard disk failure and is replaced with a new PC with Version 2.0 software running on Windows XP SP2? Is there a way to know? Who will be responsible for revalidating the new PC with the new software and new operating system (and potentially a different PC running at a different speed) in the manufacturing system? If there is a way to update software with the system in place, how will unauthorized updates be prevented? Who will control what other software can be installed on the PC?

 

They are responsible for conducting a risk analysis for the intended use of the filling system. Consider how the filling system might fail in a way that the end product will be affected. In this case, the end product is not simply the filled bottle, but is the blood analyzer, which uses the defective bottle of reagent. Assign severity levels to the anticipated medical device defects relative to the ultimate safety impact to the patient. For example, a failure of the system that would allow a bottle to pass through the system under-filled might be a lower severity than if it mislabeled the reagent or allowed the contents to be overheated. This includes, implementing risk management for the more severe and/or probable risks. Determine how the failure modes could be detected or corrected as part of the overall manufacturing system.

 

They should conduct validation testing in order to provide a level of assurance that risky failure modes do not exist, that critical performance specifications can be met, and that any risk control measures that are identified are in place and do reduce the risk to an acceptable level.

 

The above is an abbreviation of the tasks for the supplier of a software-controlled instrument (in this case a filling system), and the MDM user of the instrument. A competitive filling-system supplier that does not perform the validation tasks suggested above puts additional validation burden on the MDM customer.

 

Conversely, a competitive filling system supplier that does perform the validation tasks suggested above and that assists with some of the user’s responsibilities by coaching, training, or consulting creates added value and a marketing advantage for its filling system.

 

In some cases, the MDM may be using an instrument for a non-critical purpose in the manufacturing process or may be using it in a way in the process that 100% of all product is inspected and verified downstream in the process. In these situations a case can be made for reduced validation rigor. A supplier that can assist its user in this type of analysis and documentation can save the user considerable costs in unnecessary analysis and testing.

 

The ideas discussed in this article are specific to software used in the design or manufacture of medical devices. With little change, the same concepts could apply to software-controlled subsystems that are embedded in the medical device itself or to software modules that are included as part of the medical device embedded software. The main point is that MDMs are legally obligated to validate any software used for regulated purposes. Anything suppliers can do to assist the MDMs will become a marketing advantage for the supplier.

 

Outside the medical device industry, the needs are the same, although they might not be federally regulated. Instrument suppliers can turn their good engineering processes into a marketing advantage by packaging validation as an identifiable benefit to the customer.