How to calculate risk effectively

Identify Data

Identify all the data you have access that may help evaluate risk

e.g:

• Usability studies
• Bench testing
• Clinical studies
• Experience with similar devices
• Experience of other manufacturers with these devices (academic published papers, recall reports, instructions for use)
• Complaints and CAPA system
• Post market clinical follow up studies
• Sales
• Feedback surveys, interviews, etc.

Define Probability

• Select a method of defining probability based on the data sources available to you
• Identify different probability methods for different risks depending on what is appropriate
• Find the most frequently occurring risks and set a value around there as the maximum probability range

TIPS:

• Data based on reported customer complaints is only useful for risks associated with more serious harm. If the harm is not serious enough, it is unlikely that customers will report it to you and therefore your risk will be underestimated
• The period of time in which the data is collected and used for this calculation should be long enough to mitigate against temporary effects and short enough to avoid including data that no longer reflects the current situation for the best estimation possible

TIPS:

• Set up bench testing to determine how often a hazardous situation may occur

Example: how often a needle tip breaks using normal use forces

• Use usability studies to simulate a hazardous situation and observe how often these end in simulated harm – but make sure these tests are safe for participants!!

Example: check how often the broken needle tip is ‘used’ by participants that don’t realize it is broken

TIPS:

• You can use different probability ranges for different risks, if necessary, just document your reasoning
• Try to make sure your ranges are reasonable so that effectively calculate acceptability (all risks shouldn’t fall under one probability range!)

Define Severity

• Find the worse severity of harm to either patient or operator and set that as your maximum
• Consider including intangible harms such as lost time or frustration in your severity system
• Severity should be as objective as possible!

Define Acceptability

• Make sure a clinician with an appropriate background supports the development of acceptability criteria
• Evaluate what frequency and probability combinations would be acceptable
• Always apply further risk control measures to risks that are not acceptable until it is no longer possible to do so

Tips

• If data is not yet well established and risk calculation is based on estimations, consider adding a strategy to acquire the missing data to the Post-Market Clinical Follow-up plan
• Conduct risk reviews periodically and update based on new data
• Conduct risk meetings with a range of stakeholders including potential users, engineers, clinical specialists, risk specialists, requirements engineers, etc.
• Document methodology for arriving at calculated probabilities and severity as well as the probability and severity definitions