Scoring / rating engines - advice and examples?

Question

I need to create a flexible (and preferably dynamic) scoring engine, much like a credit scoring or premium calculating system. Does anyone with practical experience of creating

Answer 1

A scoring engine has to actually score something - with your FICO score, it's the risk that you'll default on new credit. Essentially, there are a handful of steps:

1. Collect mountains of data
2. Decide what you want to know about the data - what do you want to predict?
3. Mine the data for a relevant formula that predicts accurately
4. Implement the formula in code in a flexible way

Assuming you're asking about step 4, and not about an earlier step in the data mining process, here are some ideas:

• If your formula is simple, you can just code it and allow access to edit the co-efficients (Ax + By + C, as an example, where A, B and C are stored in a database somewhere and are easily updated, and where x and y are some data from your user/customer).
• If you want something highly dynamic, in that you might dramatically change the formula later, a rules engine is one choice, though I'd still lean towards a custom-coded solution that can be easily switched out. Maybe even a DLL with a single function in it that does your calculation, and can be easily replaced if the formula is changed significantly.

If you add some more details, or if you're not quite at step 4 in your mining process, I can suggest some steps for you to take or some reading that might help.

Answer 2

Generally speaking, people who go about developing a scoring system have a good idea about how they might do the scoring by hand, and the start of the process is to code up the naive algorithm suggested by their experience. Sometimes, people then get fancy, using concepts such as genetic algorithms. Thinking about things like FICO scoring, another part of the work tends to be the collection of data, both transactional data (purchase patterns, earning patterns) and 'result' data (defaults or other issues with credit). By analyzing patterns and results, the scoring tools develop a score for users, which is then used as a predictor of future behaviour (is this person with a FICO score of x likely to cause me problems if I extend them credit?).

I don't think there is a general solution for these analyses. Someone has to have an insight into how to do the analysis and correlation, and can then use computers to define and refine the insight.