Pybrain cross-validation method

Question

I'm trying to use the cross-validator on my data, but I'm getting 0.0 success rate, which doesn't make sense.

My data is comprised of samples with 5 continuous attributes and two possible classes: "y" and "n".

My code:

net = pybrain.tools.shortcuts.buildNetwork(5, 8, 1)
trainer = BackpropTrainer(net, ds)
evaluation = ModuleValidator.classificationPerformance(trainer.module, ds)
validator = CrossValidator(trainer=trainer, dataset=trainer.ds, n_folds=5, valfunc=evaluation)
print(validator.validate())

When I'm doing a regular training like so

print(trainer.train())

I'm getting a reasonable error rate, so I'm guessing this means the dataset and network are okay, and the problem is in the cross validator.

Any ideas?

Update:

I looked in the cross-validation code and noticed that my network outputs continuous values and not 0/1 as required. I'm guessing these are the probabilities for each class. When the model is used inside the cross-validation methods it does not account for that, and this means all answers are regarded as flase, si I'm getting 0 correct answers. How can I add a layer that looks at the continuous values and returns 0 or 1 according to whichever is bigger? The documentation is unclear.

Answer 1

I am working on neural networks too, i recommend your to check the FANN library with python bindings, its better and easier to use than pybrain