Detecting the parameter types in a Spirit semantic action

Question

General case: I can\'t figure out why my Spirit grammar/semantics actions aren\'t compiling.

Sometimes, the compiler will complain about assignment or type incompat

Answer 1

For clarity - the error here is that base_ >> int_ >> int_ was used as the expression for a rule that creates a myderived, and since base_ is fixed to type mybase, we'd have to create a myderrived from a mybase and two ints, but there's nothing to tell Spirit how to do that.

You can get boost to print out the type of the value that boost creates from parsing base_ >> int_ >> int_ by defining a functor that will take any parameters, and tell you what they are (the following code is adapted from some code sehe put on SO chat):

struct what_is_the_attr
{
    template <typename> struct result { typedef bool type; };

    template <typename T>
    static void print_the_type()
    {
        std::cout << "    ";
        std::cout << typeid(T).name();
        if(std::is_const<typename std::remove_reference<T>::type>::value)
            std::cout << " const";
        if(std::is_rvalue_reference<T>::value)
            std::cout << " &&";
        else if(std::is_lvalue_reference<T>::value)
            std::cout << " &";
    }

    template <typename Th, typename Th2, typename... Tt>
    static void print_the_type()
    {
        print_the_type<Th>();
        std::cout << ",\n";
        print_the_type<Th2, Tt...>();
    }

    template <typename... Ts>
    void operator()(Ts&&...) const
    {
        std::cout << "what_is_the_attr(\n";
        print_the_type<Ts...>();
        std::cout << ")" << std::endl;
    }
};

Then to use it, use the above actor in a semantic action on initializer for your faulty rule:

std::string input = "1 2 3 4";
auto f(std::begin(input)), l(std::end(input));

rule<decltype(f), mybase()   , space_type> base_    = int_ >> int_;
rule<decltype(f), myderived(), space_type> derived_ = (base_ >> int_ >> int_)[what_is_the_attr()];

myderived data;
bool ok = phrase_parse(f,l,derived_,space,data);

Note, you cannot use automatic attribute propagation with %= (unless you remove the exposed attribute type from the rule's declared type).

Running this should then yield an encoded type, which can be decoded with c++filt -t: Live On Coliru

$ g++ 9404189.cpp -std=c++0x
$ ./a.out |c++filt -t
what_is_the_attr(
    boost::fusion::vector3<mybase, int, int> &,
    boost::spirit::context<boost::fusion::cons<boost::spirit::unused_type&, boost::fusion::nil>, boost::fusion::vector0<void> > &,
    bool &)

The first line, boost::fusion::vector3<mybase, int, int>, least tells you that boost is trying to create your return type from 3 objects of types mybase, int and int.

Answer 2

I could solve the problem for this particular case (in fact we discussed options on the list), but really, this kind of 'enigmatic' error creeps up more often with Boost Spirit and it would be nice to get a handle on the general class of problems.

Your first resource should be the excellent spirit documentation, which details exactly what the synthesized attribute will be for a given parser primitive, operator or directive. See the Reference section to Spirit Qi Docs.

In some cases, I have taken to shifting the focus from 'trying to pry the information from the compiler error list' to 'actively querying Spirit for the types it passes'. The technique I use for this is the Polymorphic Callable Type (see Spirit/Fusion docs).

Here is one that uses GCC specific APIs to pretty [sic] print the types it detects:

Functor what_is_the_attr

#include <cxxabi.h>
#include <stdlib.h>
#include <string>
#include <iostream>

template <typename T> std::string nameofType(const T& v) {
    int     status;
    char   *realname = abi::__cxa_demangle(typeid(v).name(), 0, 0, &status);
    std::string name(realname? realname : "????");
    free(realname);
    return name;
}

struct what_is_the_attr {
    template <typename> struct result { typedef bool type; };

    template <typename T> bool operator()(T& attr) const {
        std::cerr << "what_is_the_attr: " << nameofType(attr) << std::endl;
        return true;
    }
};

Sample use: detect synthesized attribute type

You can use it to detect exactly what type the synthesized attribute type of a parser expression actually ends up being:

template <typename Exp>
    void detect_attr_type(const Exp& exp)
{
    using namespace boost::spirit::qi;

    const char input[] = "1 2 3 4";
    auto f(std::begin(input)), l(std::end(input)-1);

    bool dummy = phrase_parse(
            f, l, 
            exp [ what_is_the_attr() ],
            space);
}

(Note: this shows a limitation of the approach - the technique assumes you have an 'otherwise' working grammar and you know how to pass an input that satisfies the expression enough to trigger the semantic action. In most cases, this will be true when you are hacking on your Spirit parser, though)

Let's test it. E.g. let's see what the difference is between and expression of medium complexity, and the same wrapped inside a qi::raw[] directive:

int main()
{
    detect_attr_type(       -(int_ >> *int_)    );
    detect_attr_type( raw [ -(int_ >> *int_) ] );
}

Output:

what_is_the_attr: boost::optional<boost::fusion::vector2<int, std::vector<int, std::allocator<int> > > >
what_is_the_attr: boost::iterator_range<char const*>

At the bottom, we will apply this to the question in the OP.

Sample use: detecting types passed into Semantic Actions

We could use the same unary function object (what_is_the_attr) to detect these, however, semantic actions can take any number of arguments, so we need to generalize. This would be tedious work, if it weren't for variadic template (woot! for c++0x):

struct what_are_the_arguments {
    template <typename...> struct result { typedef bool type; };

    template <typename... T> bool operator()(const T&... attr) const {
        std::vector<std::string> names { nameofType(attr)... };
        std::cerr << "what_are_the_arguments:\n\t";
        std::copy(names.begin(), names.end(), std::ostream_iterator<std::string>(std::cerr, "\n\t"));
        std::cerr << '\n';
        return true;
    }
};

Repeated for the above test cases reveals that Spirit actually tries to call the semantic action with three arguments if possible (as documented):

what_are_the_arguments:
    boost::optional<boost::fusion::vector2<int, std::vector<int, std::allocator<int> > > >
    boost::spirit::unused_type
    bool

what_are_the_arguments:
    boost::iterator_range<char const*>
    boost::spirit::unused_type
    bool

But the nice thing is, that you can now apply this to any semantic action:

template <typename ExpWSA> void test(const ExpWSA& exp)
{
    const char input[] = "1 2 3 4";
    auto f(std::begin(input)), l(std::end(input)-1);

    qi::phrase_parse(f, l, exp, qi::space);
}

int main()
{
    test(-(-double_ >> *int_) [ phx::bind(what_are_the_arguments(), _1, _2, _0, phx::ref(std::cout), 42) ]);
}

Printing, for this (sorry) very contrived example:

what_are_the_arguments:
    boost::optional<double>
    std::vector<int, std::allocator<int> >
    boost::fusion::vector2<boost::optional<double>, std::vector<int, std::allocator<int> > >
    std::ostream
    int

Applied to the OP

The synthesized attribute of the derived rule is not the same as for int_>>int_>>int_>>int_:

auto base_expr = int_ >> int_; // avoids assigning to struct attribute

rule<const char*, mybase(), space_type> base_       = base_expr;

test(base_     >> int_ >> int_ [ what_is_the_attr() ] );
test(base_expr >> int_ >> int_ [ what_is_the_attr() ] );

Will print

what_is_the_attr: boost::fusion::vector3<mybase, int, int>
what_is_the_attr: boost::fusion::vector4<int, int, int, int>

There is your problem. We discussed some workarounds based on this diagnostic in the original thread (and see the other answers here). But this post should help answering the general case question.

Full code listing

In integrated form, compiled with gcc 4.6.1 --std=c++0x and boost 1_48:

#include <cxxabi.h>
#include <iostream>
#include <iterator>
#include <stdlib.h>
#include <string>
#include <vector>

template <typename T> std::string nameofType(const T& v)
{
    int     status;
    char   *realname = abi::__cxa_demangle(typeid(v).name(), 0, 0, &status);
    std::string name(realname? realname : "????");
    free(realname);

    return name;
}

struct what_is_the_attr {
    template <typename> struct result { typedef bool type; };

    template <typename T> bool operator()(T& attr) const {
        std::cerr << "what_is_the_attr: " << nameofType(attr) << std::endl;
        return true;
    }
};

struct what_are_the_arguments {
    template <typename...> struct result { typedef bool type; };

    template <typename... T> bool operator()(const T&... attr) const {
        std::vector<std::string> names { nameofType(attr)... };
        std::cerr << "what_are_the_arguments:\n\t";
        std::copy(names.begin(), names.end(), std::ostream_iterator<std::string>(std::cerr, "\n\t"));
        std::cerr << '\n';
        return true;
    }
};

#include <boost/fusion/adapted.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/include/qi.hpp>

struct mybase             { int a,b; };
struct myderived : mybase { int c,d; };

BOOST_FUSION_ADAPT_STRUCT(mybase,    (int,a)(int,b));
BOOST_FUSION_ADAPT_STRUCT(myderived, (int,a)(int,b)(int,c)(int,d));

template <typename ExpWSA>
void test(const ExpWSA& exp)
{
    using namespace boost::spirit::qi;

    const char input[] = "1 2 3 4";
    auto f(std::begin(input)), l(std::end(input)-1);

    bool dummy = phrase_parse(f, l, exp, space);
}

int main()
{
    using namespace boost::spirit::qi;

    // Diagnostics for the OP case
    auto base_expr = int_ >> int_;                                   // avoids assigning to struct attribute
    rule<const char*, mybase(), space_type> base_       = base_expr;

    // Derived rule, different formulations
    test((base_     >> int_ >> int_) [ what_is_the_attr() ] );
    test((base_expr >> int_ >> int_) [ what_is_the_attr() ] );

    // Applied to attribute types
    test(raw [ -(int_ >> *int_) ]  [ what_is_the_attr() ] );
    test(-(int_ >> *int_)          [ what_is_the_attr() ] );

    // applied to semantic actions - contrived example
    namespace phx = boost::phoenix;
    test(-(-double_ >> *int_) [ phx::bind(what_are_the_arguments(), _1, _2, _0, phx::ref(std::cout), 42) ]);

    return 0;
}