Insurance: Part 2 - Adequate, fair and competitive insurance pricing




05 Jul 2018


Data Science


By: Natasha Mashanovich, Senior Data Scientist at World Programming, UK

Part 2: Adequate, fair and competitive insurance pricing

"Which Insurance?" – is the first question a prospect asks themselves when looking for an attractive policy quotation. In the very competitive UK insurance market (Figure 1), insurers need to develop a bespoke pricing methodology to ensure the policy premiums are, firstly, adequate so they would cover expected losses and incurred expenses. Secondly, the policies must be set up in a fair manner so the premiums are closely associated with expected losses and expenses. Thirdly, premiums must be competitive for attracting new customers and retaining existing customers.

Blog2 insurance images1

Figure 1. Competitive UK Insurance Market

Ratemaking or risk-based pricing is an essential step and the key element of insurance pricing. Actuarial pricing techniques and methodologies are heavily dependent on insurance type, data availability and extensive regulatory, marketing and operational constraints. Furthermore, selected modelling techniques are constantly changing in line with advances in technology and data science.

Techniques range from different statistical methods such as linear, additive and mixture models, to an extensive set of machine-learning models such as random forest, gradient boosting, neural networks or support vector machines. Statistical models typically work on a number of assumptions focusing on data fitting in the form of equations. In contrast, machine-learning methods tend to have less assumption about data and focus on learning through algorithm construction. Recognising the most appropriate technique in the rich analytics landscape is often challenging and requires consideration of multiple facets such as model accuracy, model stability over time, business constraints, required DevOps resources for model deployment and implementation, model response time, and so on.

Despite such model diversity, the Generalised Linear Model (GLM) remains the de facto standard in the insurance industry. Although machine-learning methods can often achieve better prediction performance, GLM has gained popularity over the other modelling techniques as it is easy to interpret and understand the results; the model assumes a linear relationship between predictors and outcome; and provides better control when selecting the rating factors. In addition, a GLM-based rating algorithm (Table 1) is easy to implement, fast to execute and offers more flexibility to integrate actuarial experience and implement various constraints including regulatory, operational and marketing constraints.

Base Rate£500Rating Algorithm
Rating factorLevelRelativityPure premium = Base rate
Territory zone13.81* Territory zone
Vehicle age12.38* Vehicle age
Engine power10.37* Engine power
Bonus10.58* Bonus

Table 1. GLM-based rating relativities and rating algorithm (illustration)

The result of the ratemaking process is a predicted technical price (i.e. pure premium) that matches the likelihood of making a claim. If modelled accurately it should provide the adequate price to cover expected losses. The technical price is then adjusted to include other underwriting expenses including acquisition cost, commissions and tax fees as well as underwriting profit.

The traditional approach to insurance pricing has focused solely on the risk-based pricing, neglecting competitor prices. In the current climate, this approach is not sustainable and many insurers are transitioning towards more sophisticated pricing methodologies. World Programming may advocate a holistic approach to insurance pricing illustrated in Figure 2. With a symbolic reference to ocean diving, the deeper you dive into a (data) ocean in a quest for discovering new (data) species, the more successful you are in seizing new and valuable (business) gems.

Blog2 insurance images2

Figure 2. Insurance Pricing Process

In this view, premiums based solely on risk-based pricing would be like searching in a shallow water and not maximising the full potential of the available data. Ratemaking pricing ensures an adequate price but not necessarily the competitive one. Hence, the pricing process should comprise the additional elements including customer segmentation, consideration of competitors' prices and price optimisation.

Customer segmentation is an important step to identify if a customer should be targeted for price increase or decrease. Additionally, it serves as a reassurance that premium discounts are only being offered to the least-risky customers. Segmentation can be very simple based on a few business rules, or more sophisticated based on a propensity model such as probability of making a claim or a clustering model that creates segments by eligibility levels.

Competitive pricing is about adjusting the risk-based premiums to competitors' rates. Depending on marketing initiatives, demand models may be required to capture the competitive market – a conversion model for customer acquisition or a churn model for customer retention. The decision as to which binary classifier to utilise for building these propensity models will depend on many factors including the required model performance and ease of model deployment and implementation. In addition, conversion models usually run in real-time - hence the speed of model scoring is an important factor to consider when deciding on modelling technique.

The final step of the pricing process is price optimisation, usually referred to as price elasticity and is about assessing price tolerance at an individual level. This step provides extra benefit ensuring the relevant competitive model (the yellow line in Figure 3) sets up the price, which maximises the profit (the blue line in Figure 3). An optimisation model usually requires price simulations for different what-if scenarios so the maximum benefit can be extracted given the optimisation constraints.

Insurance blog image price optimisation

Figure 3. Price Optimisation Problem (source:

Figure 4 is an illustration of an optimisation model that offers premium discounts of up to 20% on the risk-based price. Depending on the underlying demand model, the premium discount distribution ranges from no-discount to a maximum 20% discount.

Blog2 insurance images4

Figure 4. Price Optimisation Results (illustration)

Once the optimal scenario is selected, the complete pricing solution is ready for deployment, implementation and testing. This includes the risk-based model, the demand model and the optimisation model. The complete model suite can be implemented on a single rating engine or hosted across multiple engines. After a rigorous testing process, the pricing solution is ready for a racing competition.