Can spirit X3 work with BOOST_FUSION_ADAPT_ADT?

Question

Change my codes from QI to X3, and get some compile error with BOOST_FUSION_ADAPT_ADT. I tried boost 1.64 and 1.67, neither of them work. I modified the spirit X3 example rexpr_min, adding getter and setter to struct rexpr, changing the BOOST_FUSION_ADAPT_STRUCT to BOOST_FUSION_ADAPT_ADT, and compile it fail, too.

Enviroment:

• ubuntu 16.04

• G++ 5.4, with -std=c++17 flag

• boost 1.67

Error message:

boost/spirit/home/x3/core/detail/parse_into_container.hpp:142:35: error: invalid initialization of non-const reference of type ‘boost::fusion::extension::adt_attribute_proxy<client::ast::rexpr, 0, false>&’ from an rvalue of type ‘boost::fusion::extension::deref_impl<boost::fusion::struct_iterator_tag>::apply<boost::fusion::basic_iterator<boost::fusion::struct_iterator_tag, boost::fusion::random_access_traversal_tag, client::ast::rexpr, 0> >::type {aka boost::fusion::extension::adt_attribute_proxy<client::ast::rexpr, 0, false>}’
             return call_synthesize(parser, first, last, context, rcontext,

I guess the fusion::front(attr) return a const reference, and the call_synthesize want a non-const reference (at boost_1_64_0/boost/spirit/home/x3/core/detail/parse_into_container.hpp, line 146). But I don't know what to do.

I googled and find some regression of QI and they are patched in newest version. But there is no information with X3.

The original code spirit X3 example rexpr_min, And my modification:

/*=============================================================================
    Copyright (c) 2001-2015 Joel de Guzman

    Distributed under the Boost Software License, Version 1.0. (See accompanying
    file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
=============================================================================*/
///////////////////////////////////////////////////////////////////////////////
//
//  A simple parser for X3 intended as a minimal starting point.
//  'rexpr' is a parser for a language resembling a minimal subset
//  of json, but limited to a dictionary (composed of key=value pairs)
//  where the value can itself be a string or a recursive dictionary.
//
//  Example:
//
//  {
//      "color" = "blue"
//      "size" = "29 cm."
//      "position" = {
//          "x" = "123"
//          "y" = "456"
//      }
//  }
//
///////////////////////////////////////////////////////////////////////////////

#include <boost/config/warning_disable.hpp>
#include <boost/spirit/home/x3.hpp>
#include <boost/spirit/home/x3/support/ast/variant.hpp>
#include <boost/fusion/include/adapt_adt.hpp>
#include <boost/fusion/include/std_pair.hpp>
#include <boost/fusion/include/io.hpp>

#include <iostream>
#include <fstream>
#include <string>
#include <map>

///////////////////////////////////////////////////////////////////////////////
//  Our AST
///////////////////////////////////////////////////////////////////////////////
namespace client { namespace ast
{
    namespace fusion = boost::fusion;
    namespace x3 = boost::spirit::x3;

    struct rexpr;

    struct rexpr_value : x3::variant<
            std::string
          , x3::forward_ast<rexpr>
        >
    {
        using base_type::base_type;
        using base_type::operator=;
    };

    typedef std::map<std::string, rexpr_value> rexpr_map;
    typedef std::pair<std::string, rexpr_value> rexpr_key_value;

    struct rexpr
    {
        rexpr_map i_entries;

        const rexpr_map& entries() const { return i_entries; }
        void entries(const rexpr_map& ent) { i_entries = ent; }
    };
}}

// We need to tell fusion about our rexpr struct
// to make it a first-class fusion citizen
BOOST_FUSION_ADAPT_ADT(client::ast::rexpr,
    (obj.entries(), obj.entries(val))
)

///////////////////////////////////////////////////////////////////////////////
//  AST processing
///////////////////////////////////////////////////////////////////////////////
namespace client { namespace ast
{
    ///////////////////////////////////////////////////////////////////////////
    //  Print out the rexpr tree
    ///////////////////////////////////////////////////////////////////////////
    int const tabsize = 4;

    struct rexpr_printer
    {
        typedef void result_type;

        rexpr_printer(int indent = 0)
          : indent(indent) {}

        void operator()(rexpr const& ast) const
        {
            std::cout << '{' << std::endl;
            for (auto const& entry : ast.entries())
            {
                tab(indent+tabsize);
                std::cout << '"' << entry.first << "\" = ";
                boost::apply_visitor(rexpr_printer(indent+tabsize), entry.second);
            }
            tab(indent);
            std::cout << '}' << std::endl;
        }

        void operator()(std::string const& text) const
        {
            std::cout << '"' << text << '"' << std::endl;
        }

        void tab(int spaces) const
        {
            for (int i = 0; i < spaces; ++i)
                std::cout << ' ';
        }

        int indent;
    };
}}

///////////////////////////////////////////////////////////////////////////////
//  Our rexpr grammar
///////////////////////////////////////////////////////////////////////////////
namespace client { namespace parser
{
    namespace x3 = boost::spirit::x3;
    namespace ascii = boost::spirit::x3::ascii;

    using x3::lit;
    using x3::lexeme;

    using ascii::char_;
    using ascii::string;

    x3::rule<class rexpr_value, ast::rexpr_value>
        rexpr_value = "rexpr_value";

    x3::rule<class rexpr, ast::rexpr>
        rexpr = "rexpr";

    x3::rule<class rexpr_key_value, ast::rexpr_key_value>
        rexpr_key_value = "rexpr_key_value";

    auto const quoted_string =
        lexeme['"' >> *(char_ - '"') >> '"'];

    auto const rexpr_value_def =
        quoted_string | rexpr;

    auto const rexpr_key_value_def =
        quoted_string >> '=' >> rexpr_value;

    auto const rexpr_def =
        '{' >> *rexpr_key_value >> '}';

    BOOST_SPIRIT_DEFINE(rexpr_value, rexpr, rexpr_key_value);
}}

///////////////////////////////////////////////////////////////////////////////
//  Main program
///////////////////////////////////////////////////////////////////////////////
int main(int argc, char **argv)
{
    char const* filename;
    if (argc > 1)
    {
        filename = argv[1];
    }
    else
    {
        std::cerr << "Error: No input file provided." << std::endl;
        return 1;
    }

    std::ifstream in(filename, std::ios_base::in);

    if (!in)
    {
        std::cerr << "Error: Could not open input file: "
            << filename << std::endl;
        return 1;
    }

    std::string storage; // We will read the contents here.
    in.unsetf(std::ios::skipws); // No white space skipping!
    std::copy(
        std::istream_iterator<char>(in),
        std::istream_iterator<char>(),
        std::back_inserter(storage));

    using client::parser::rexpr; // Our grammar
    client::ast::rexpr ast; // Our tree

    using boost::spirit::x3::ascii::space;
    std::string::const_iterator iter = storage.begin();
    std::string::const_iterator end = storage.end();
    bool r = phrase_parse(iter, end, rexpr, space, ast);

    if (r && iter == end)
    {
        std::cout << "-------------------------\n";
        std::cout << "Parsing succeeded\n";
        std::cout << "-------------------------\n";
        client::ast::rexpr_printer printer;
        printer(ast);
        return 0;
    }
    else
    {
        std::string::const_iterator some = iter+30;
        std::string context(iter, (some>end)?end:some);
        std::cout << "-------------------------\n";
        std::cout << "Parsing failed\n";
        std::cout << "stopped at: \": " << context << "...\"\n";
        std::cout << "-------------------------\n";
        return 1;
    }
}

Answer 1

As I've been warning people before¹ you're pushing limits right on the intersection of things that frequently break Spirit's gears:

1. single-element fusion sequences
2. ADT adaptation in general
3. Persistent bugs with ADT fixed in develop (after 1.67.0 release)

1. The Single-Element Conundrum

I won't spend much time on this because it's a rather old, boring, well-documented² and not essential to your question.

Let's side-step it by adding a dummy field:

struct rexpr
{
    rexpr_map i_entries;

    const rexpr_map& entries() const { return i_entries; }
    rexpr_map& entries() { return i_entries; }

    void entries(const rexpr_map& ent) { i_entries = ent; }

    int i_dummy;
    int dummy() const { return i_dummy; }
    void dummy(int i) { i_dummy = i; }
};

// ... later:
BOOST_FUSION_ADAPT_ADT(client::ast::rexpr,
    (obj.entries(), obj.entries(val))
    (obj.dummy(), obj.dummy(val))
)

// ... even later:
auto const rexpr_def =
    '{' >> *rexpr_key_value >> '}' >> x3::attr(42);

2. The ADT Proxy

The attribute-category machinery of Spirit detects the entries property as a container attribute (is_container<...>{} evaluates to true).

However the requisite container traits are not in place.

What's more, because of the restrictive interface that ADT proxies grant, the property values can only replaced whole-sale, meaning that we can only implement a very suboptimal version of it:

namespace boost { namespace spirit { namespace x3 { namespace traits {

    template <typename T, auto... Other>
    struct container_value<boost::fusion::extension::adt_attribute_proxy<T, Other...> > 
        : container_value<typename boost::fusion::extension::adt_attribute_proxy<T, Other...>::type>
    { };

    template <typename T, auto... Other>
    struct push_back_container<boost::fusion::extension::adt_attribute_proxy<T, Other...> > 
    {
        using underlying_type = typename boost::fusion::extension::adt_attribute_proxy<T, Other...>::type;

        template <typename X, typename Y>
            static bool call(X& proxy, Y&& v) {
                auto u = proxy.get();
                bool b = push_back_container<underlying_type>::call(u, std::forward<Y>(v));
                proxy = u;
                return b;
            }
    };

} } } }

3. Surprise: Old bugs fixed after 1.67.0

You require the commits:

commit ae78e1ec2431517a8b0580099aeba8f9122d8abb
Author: Nikita Kniazev <nok.raven@gmail.com>
Date:   Thu Mar 15 17:33:36 2018 +0300

    X3: sequence: Fixed reference to temporary bug

commit e7f31017ec7c0b5584d12ec1b718d8c415b26fa1
Author: Nikita Kniazev <nok.raven@gmail.com>
Date:   Wed Mar 14 18:54:35 2018 +0300

    Qi: Fixed ADT support by permutation and sequence_or operator

    This is follow-up to 0f2b3c49ce55a41a7d22cc5533e0f4ba59e491ae

These are more recent than 1.67.0 and currently in the develop branch. They (in part) fix an old issue: https://github.com/boostorg/spirit/pull/153#issuecomment-152879056. The current behaviour may also be impacted by commit

commit a0df3c098ff4e42c0958796c4f47d4d72a20c164
Merge: f73b121 fac9dfa
Author: Nikita Kniazev <nok.raven@gmail.com>
Date:   Thu Mar 1 13:44:27 2018 +0300

    Merge pull request #370 from Kojoley/x3-pass-container-attribute-through-sequence

    X3: Pass container attribute through sequence

It's hard to gauge whether the impact is positive or negative in this ... turbulent landscape.

Demo

Suffice it to say that iff you

1. compile against ae78e1ec243151 or later (develop)
2. Apply both the workarounds described above

then I see the expected output:

-------------------------
Parsing succeeded
-------------------------
{
    "color" = "blue"
    "position" = {
        "x" = "123"
        "y" = "456"
    }
    "size" = "29 cm."
}

(based on libs/spirit/example/x3/rexpr/rexpr_examples/a.rexpr input).

Summarizing

I hope you don't think this is "fine". Please consider filing an issue at the mailing list/github. Also take these into account:

• Is Boost Spirit X3 production ready?
• How future-safe is it to write a parser with Boost Spirit X3?

• In the light of recent recurring issues revolving arround container-attributes, I think a change around boost 1.65.1 caused container attributes to regress across the board:

  • How to make a recursive rule in boost spirit x3 in VS2017
  • Boost.Spirit X3 parser "no type named type in(...)"

---

¹ not to mention my dislike for it in most cases: Using spirit to parse into classes?

² Spirit Qi attribute propagation issue with single-member struct, X3, what is attr_gen?, boost::spirit::x3 attribute compatibility rules, intuition or code?