1. Predictive Models
2. Decision Optimization Models
3. Segmentation Models
4. Forecasting Models

Keys to a Successful Model Development

Objective

Nothing is more important to the success of a model development than definition of objectives.  Most modeling efforts should be ultimately focused on creating value for the organization, but the objectives should show how such value will be created.

If your organization is developing a new risk-prediction underwriting model, a developer might suggest that the model will improve prediction over the old model.  But how will the improved prediction result in increased profits and shareholder value?  For value to improve, the new model will have to result in a reduction in value-destroying accounts, or an increase in value-creating accounts.  But how exactly will the new model accomplish this?  If a K-S Test is being used to measure predictive power, how does an increase in K-S translate into economic or shareholder value?

Near-term objectives for predictive models are quite different from those of decision optimization models, which are quite different from those of forecasting models.  In all cases, making sure that all interested parties understand and agree with the objectives is imperative to the success of the effort.

Discovery

Too often modelers go straight from gathering the sample data into the modeling effort.  This leaves out one of the most important components of the process - discovery.  Discovery is the research effort wherein the data is examined in great detail for clues about

• where and how potential predictive relationships may reside
• interactions and dependencies among characteristics
• values found in data fields
• truncation of populations related to existing policy
• which population segments are represented with various product and pricing combinations
• key filters through which population segments have been run

Ideally, many of these issues would be visited in the advance preparation for the development.  In reality, no matter how diligently the project is discussed in advance, it is impossible that every imaginable issue will be raised before examination of the data begins.

It is here in discovery where the skill and knowledge of the developers add value beyond simply understanding statistical and financial modeling techniques.

Validation/ Testing

Most model development efforts include validation on a holdout sample.  This is a good process to follow, but it is not sufficient.  Validation and proof-testing should be conducted on every population subset that may behave in a unique manner.

If multiple products are included in the development, performance for each should be examined.  The same is true for price structures and price points, demographic segments, marketing channels, and key filters (e.g. which credit bureau was used).

This rigorous validation process will result in important knowledge of the performance of the system on all key customer and product dimensions.

Implementation/ Execution

Many a good model has ended up as a dust-collecting binder because insufficient consideration was given to the ability to implement the model in live production.

Consideration of implementation issues should begin prior to any data being collected.  Application processing software, accounting and billing systems, data and computing resources required for model execution, and data storage must all be considered.

VALANTEX experience in model development spans all of these areas. We can develop models for you, or work with your professional staff to have them developed internally.

Contact VALANTEX for more information, or to request a proposal