Arrayfire error with conditionals

问题

I am trying to saturate my arrays in ArrayFire. I want all values greater than 0.75 to saturate to 1.0 and all less than 0.25 to saturate to 0.0. I am using the following expressions.

a(a > 0.75) = 1.0;
a(a < 0.25) = 0.0;

Here is is an af::array type. It works for a while but as soon as I get an array where there are no values greater than 0.75 I get the following exception.

terminate called after throwing an instance of 'af::exception'
  what():  ArrayFire Exception (Invalid input size:203):
In function verifyDims
In file src/api/c/data.cpp:36
Invalid dimension for argument 1
Expected: ndims >= 1

In function af::array af::constant(T, const af::dim4&, af::dtype) [with T = double; af::dtype = af_dtype]
In file src/api/cpp/data.cpp:28

If I call af::print("", a > 0.75); I get the following output right before it crashes.

Is it somehow seeing that this array is all zeros (which it should be since non are greater than 0.75) and then saying the dimension is zero? It it something I am doing wrong or is it a bug in their code?

The following code seems to fix it but I feel this solution is somewhat inefficient.

af::array bellow = a[levels - 1] < 0.25f;
af::array above = a[levels - 1] > 0.75f;

if(af::anyTrue<bool>(above))
    a[levels - 1](above) = 0.75f;

if(af::anyTrue<bool>(bellow))
    a[levels - 1](bellow) = 0.25f;

For those of you who want to see the entire function I am doing gradient decent in a neural network. a is actually and array of type af::array. I left that out to simplify the question.

void train(const float* in, const float* expected_out, float learning_rate)
{
    std::unique_ptr<af::array[]> a(new af::array[levels]),
            z(new af::array[levels]), d(new af::array[levels]);

    af::array in_array(inputs, in);
    af::array y(dims[levels - 1], expected_out);

    z[0] = af::matmul(weights[0], in_array) + biases[0];
    a[0] = sigma(z[0]);


    for(size_t i = 1; i < levels; i++)
    {
        z[i] = af::matmul(weights[i], a[i - 1]) + biases[i];
        a[i] = sigma(z[i]);
    }


    a[levels - 1](a[levels - 1] < 0.25f) = 0.0f;
    a[levels - 1](a[levels - 1] > 0.75f) = 1.0f;

    d[levels - 1] = (y - a[levels - 1]) * sigma_prime(z[levels - 1]);
    for(size_t i = levels - 1; i-- > 0;)
        d[i] = af::matmul(weights[i + 1].T(), d[i + 1]) * sigma_prime(z[i]);

    for(size_t i = 0; i < levels; i++)
    {
        biases[i] += learning_rate * d[i];
        weights[i] += learning_rate * af::matmul(d[i], (i ? a[i - 1] : in_array).T());
    }
}

回答1:

The error you are seeing is because of this open bug about zero length arrays (EDIT: FIxed as of v3.4.0). This is a pervasive issue that we are trying to fix properly for some time now.

Here is the work around for your case. You don't even need indexing to achieve what you are trying to do.

a[levels - 1] = af::min(0.75, af::max(0.25, a[levels - 1]));

EDIT: As of 3.4, you can do the following to achieve the same functionality in arrayfire:

a[levels - 1] = af::clamp(a[levels - 1], 0.25, 0.75);

This method is much much faster than indexing for your case.

That said, there are some cases when you can not use af::min and af::max to replace indexing. In those cases, you could do something like this as a work around:

af::array cond = arr < some_val;
arr = arr * (1 - cond) + cond * other_val;

This should also be faster than indexing. However the arithmetic will not work if the arrays have NAN in them and you are trying to replace them. In which case you could fall back to one of the following functions.

Using select (uses additional memory):

arr = af::select(af::isNaN(arr), arr, other_val));

Using replace (replaces in place, no additional memory used):

af::replace(arr, af::isNaN(arr) other_val));

However some benchmarking showed us that select and replace can be slower than indexing in certain cases (which we are trying to fix). So you could try using the following work around for indexing if select / replace are slow in your algorithm.

af::array idx = af::where(af::isNaN(arr));
if (idx.elements()) arr(idx) = replace_val;

Note that indexing on a boolean af::array calls af::where internally. So this is as efficient as the following

arr(arr < some_val) = other_val;

with the added benefit of not failing for zero sized arrays.

EDIT: Added additional workarounds for posterity.

来源：https://stackoverflow.com/questions/36263133/arrayfire-error-with-conditionals

标签

c++

arrayfire