I worked as a consultant for a leading rule engine in the industry for close to three years. You will pay a LOT of money for a commercial rule engine and need to decide if the features and support you potentially receive are worth the price. All rule engine vendors will claim theirs is the best hands down; the best advice I can give you is to do a prototype with each rule engine you're considering that is limited in scope, but representative of the features you need. Do NOT let sales people just show you a fancy demo; YOU should actively build the prototype. I have seen too many people buy a rule engine that really have no idea what it is they're buying. Personally, I would opt for one of the open source rule engines (someone has already mentioned Drools) if you have your heart set on a rule engine. However, I would seriously consider alongside rule engines the possibility of not using a rule engine. You may save time initially with a rule engine (ignoring the learning curve), but a custom solution is likely to be superior in the long term to meet you specific needs.
Here are a couple links from Martin Fowler's website that are relevant to your question:
- http://www.martinfowler.com/bliki/RulesEngine.html
- http://www.martinfowler.com/bliki/DomainSpecificLanguage.html