Question

我尝试用精彩的 Eric Niebler 中的优秀Boost.XPressive编写小胡子解析器。但由于这是我的第一个解析器，我不熟悉＆＃34;正常＆＃34;编译器编写者的方法和术语，经过几天的试用和错误后感觉有点丢失。所以我来到这里，希望有人可以告诉我我的愚蠢行为的愚蠢;）

这是包含我要提取的小胡子模板的HTML代码（http://mustache.github.io/）： Now <bold>is the {{#time}}gugus {{zeit}} oder nicht{{/time}} for all good men to come to the {007} aid of their</bold> {{country}}. Result: {{#Res1}}Nullum est mundi{{/Res1}}

我有以下问题，我无法单独解决：

我写的解析器不打印任何内容，但也没有在编译时发出警告。我之前设法让它打印出胡子代码的一部分，但从来都没有正确。
我不知道如何遍历所有代码以查找所有匹配项，但之后也可以像使用smatch what;变量一样访问它们。该文档仅显示如何使用＆＃34; what＆＃34;或者如何使用＆＃34; iterator＆＃34;输出所有出现的内容。
- 其实我需要两者兼而有之。因为一旦发现某些东西，我需要质疑标签名称和标签之间的内容（＆＃34;什么＆＃34;会提供但＃34;迭代器＆＃34;赢得不允许） - 并采取行动因此。我想我可以使用＆＃34;动作＆＃34;但是如何？
- 我认为应该可以在标签和标签之间进行标签查找和＃34;一举，对吗？或者我需要解析2次 - 如果是这样的话？
是否可以像我一样解析开括号和右括号，因为总是 2个括号？或者我应该按顺序执行还是使用repeat<2,2>('{')？
我仍然对keep()和by_ref()是必要的情况以及最好不使用它们的情况有点不确定。
我无法找到迭代器sregex_token_iterator cur( str.begin(), str.end(), html, -1 );的第4个参数的其他选项 -1 ，除了匹配的标签外，它们都会输出。
我的解析器字符串是否正确查找嵌套的胡须标记？

#include <boost/xpressive/xpressive_static.hpp>
#include <boost/xpressive/match_results.hpp>
typedef std::string::const_iterator It;
using namespace boost::xpressive;

std::string str = "Now <bold>is the {{#time}}gugus {{zeit}} oder nicht{{/time}} <i>for all good men</i> to come to the {007} aid of their</bold> {{country}}. Result: {{#Res1}}Nullum <b>est</b> mundi{{/Res1}}";
// Parser setup --------------------------------------------------------
mark_tag mtag (1), cond_mtag (2), user_str (3);
sregex brackets = "{{"
                  >> keep ( mtag = repeat<1, 20> (_w) )
                  >> "}}"
                  ;

sregex cond_brackets = "{{#"
                   >> keep (cond_mtag = repeat<1, 20> (_w) )
                   >> "}}"
                   >> * (
                       keep (user_str = + (*_s >> +alnum >> *_s) ) |
                       by_ref (brackets) |
                       by_ref (cond_brackets)
                   )
                   >> "{{/"
                   >> cond_mtag
                   >> "}}"
                   ;
sregex mexpression = *( by_ref (cond_brackets) | by_ref (brackets) );

// Looping + catching the results --------------------------------------
smatch what2;
std::cout << "\nregex_search:\n" << str << '\n';
It strBegin = str.begin(), strEnd = str.end();
int ic = 0;

do
{
    if ( !regex_search ( strBegin, strEnd, what2, mexpression ) )
    {
        std::cout << "\t>> Breakout of this life...! Exit after " << ic << " loop(s)." << std::endl;
        break;
    }
    else
    {
        std::cout << "**Loop Nr: " << ic << '\n';
        std::cout << "\twhat2[0] "         << what2[0]      << '\n'; // whole match
        std::cout << "\twhat2[mtag] "      << what2[mtag]   << '\n';
        std::cout << "\twhat2[cond_mtag] " << what2[cond_mtag]   << '\n';
        std::cout << "\twhat2[user_str] "  << what2[user_str]    << '\n';
        // display the nested results
        std::for_each (
            what2.nested_results().begin(),
            what2.nested_results().end(),
            output_nested_results() // <--identical function from E.Nieblers documentation
        );

        strBegin = what2[0].second;
    }
    ++ic;
}
while (ic < 6 || strBegin != str.end() );

Answer 1

Boost Spirit建立在Proto上（由同一个英雄，Eric Niebler！），所以我希望你不要介意我是否坚持我的个人传统并在Boost Spirit中提出一个实现。

我发现从所显示的代码中看到你想要实现的目标非常棘手。因此，我直接进入 mustache docs 并为以下AST实现了解析器：

namespace mustache {

    // any atom refers directly to source iterators for efficiency
    using boost::string_ref;
    template <typename Kind> struct atom {
        string_ref value;

        atom() { }
        atom(string_ref const& value) : value(value) { }
    };

    // the atoms
    using verbatim = atom<struct verbatim_tag>;
    using variable = atom<struct variable_tag>;
    using partial  = atom<struct partial_tag>;

    // the template elements (any atom or a section)
    struct section;

    using melement = boost::variant<
            verbatim,
            variable,
            partial, 
            boost::recursive_wrapper<section>
            // TODO comments and set-separators
        >;

    // the template: sequences of elements
    using sequence = std::vector<melement>;

    // section: recursively define to contain a template sequence
    struct section {
        bool       sense; // positive or negative
        string_ref control;
        sequence   content;
    };
}

正如您所看到的，我已经添加了对否定部分以及部分模板（即扩展到模板以动态扩展的变量）的支持。

以下是作品：

sequence     = *element;
element      = 
            !(lit("{{") >> '/') >> // section-end ends the current sequence
            (partial | section | variable | verbatim);

reference    = +(graph - "}}");

partial      = qi::lit("{{") >> "> " >> reference >> "}}";

sense        = ('#' > attr(true))
             | ('^' > attr(false));

section     %= "{{" >> sense >> reference [ section_id = phx::bind(&boost::string_ref::to_string, _1) ] >> "}}"
            >> sequence // contents
            > ("{{" >> ('/' >> lit(section_id)) >> "}}");

variable     = "{{" >> reference >> "}}";

verbatim     = +(char_ - "{{");

唯一的好处是使用名为qi::local<>的{{1}}来检查某个部分的结束标记是否与当前部分的开始标记匹配。

section_id

我根据输入数据保持不变的假设来优化事物，因此我们不需要复制实际数据。这应该避免99％的分配需求。我在这里使用qi::rule<Iterator, mustache::sequence()> sequence; qi::rule<Iterator, mustache::melement()> element; qi::rule<Iterator, mustache::partial()> partial; qi::rule<Iterator, mustache::section(), qi::locals<std::string> > section; qi::rule<Iterator, bool()> sense; // postive or negative qi::rule<Iterator, mustache::variable()> variable; qi::rule<Iterator, mustache::verbatim()> verbatim;来实现这一点，我认为可以说这引入了复杂性（见下面的完整代码）。

boost::string_ref

现在我们已准备好让我们的解析器进行旋转查看Live On Coliru

qi::rule<Iterator, boost::string_ref()>   reference;

int main() { std::cout << std::unitbuf; std::string input = "<ul>{{#time}}\n\t<li>{{> partial}}</li>{{/time}}</ul>\n " "for all good men to come to the {007} aid of " "their</bold> {{country}}. Result: {{^Res2}}(absent){{/Res2}}{{#Res2}}{{Res2}}{{/Res2}}" ; // Parser setup -------------------------------------------------------- typedef std::string::const_iterator It; static const mustache_grammar<It> p; It first = input.begin(), last = input.end(); try { mustache::sequence parsed_template; if (qi::parse(first, last, p, parsed_template)) std::cout << "Parse success\n"; else std::cout << "Parse failed\n"; if (first != last) std::cout << "Remaing unparsed input: '" << std::string(first, last) << "'\n"; std::cout << "Input: " << input << "\n"; std::cout << "Dump: "; Dumping::dumper()(std::cout, parsed_template) << "\n"; } catch(qi::expectation_failure<It> const& e) { std::cout << "Unexpected: '" << std::string(e.first, e.last) << "'\n"; } }只是从解析的AST中打印出胡须模板。您可能想知道Dumping::dumper是如何实现的：

dumper

没有什么比这更复杂了。 Boost Variant确实提供了一种声明式编程风格。为了更加彻底地说明这一点，让我们根据上下文对象添加扩展！

我没有为此实现JSON，所以让我们假设一个上下文值模型，如：

struct dumper : boost::static_visitor<std::ostream&>
{
    std::ostream& operator()(std::ostream& os, mustache::sequence const& v) const {
        for(auto& element : v)
            boost::apply_visitor(std::bind(dumper(), std::ref(os), std::placeholders::_1), element);
        return os;
    }
    std::ostream& operator()(std::ostream& os, mustache::verbatim const& v) const {
        return os << v.value;
    }
    std::ostream& operator()(std::ostream& os, mustache::variable const& v) const {
        return os << "{{" << v.value << "}}";
    }
    std::ostream& operator()(std::ostream& os, mustache::partial const& v) const {
        return os << "{{> " << v.value << "}}";
    }
    std::ostream& operator()(std::ostream& os, mustache::section const& v) const {
        os << "{{" << (v.sense?'#':'^') << v.control << "}}";
        (*this)(os, v.content);
        return os << "{{/" << v.control << "}}";
    }
};

现在我们对struct Nil { }; using Value = boost::make_recursive_variant< Nil, double, std::string, std::map<std::string, boost::recursive_variant_>, std::vector<boost::recursive_variant_> >::type; using Dict = std::map<std::string, Value>; using Array = std::vector<Value>;和此上下文mustache::melement变体使用二进制访问。这比转储更多的代码，但让我们先看一下使用网站：

Value

打印（再次查看Live On Coliru ）：

using namespace ContextExpander;
expander engine;

Value const ctx = Dict { 
    { "time", Array {
        Dict { { "partial", "gugus {{zeit}} (a.k.a. <u>{{title}}</u>)"},             { "title", "noon" },    { "zeit", "12:00" } },
        Dict { { "partial", "gugus {{zeit}} (a.k.a. <u>{{title}}</u>)"},             { "title", "evening" }, { "zeit", "19:30" } },
        Dict { { "partial", "gugus <u>{{title}}</u> (expected at around {{zeit}})"}, { "title", "dawn" },    { "zeit", "06:00" } },
    } },
    { "country", "ESP" },
    { "Res3", "unused" }
};

engine(std::cout, ctx, parsed_template);

完整代码清单

供参考：

Evaluation: <ul>
    <li>gugus 12:00 (a.k.a. <u>noon</u>)</li>
    <li>gugus 19:30 (a.k.a. <u>evening</u>)</li>
    <li>gugus <u>dawn</u> (expected at around 06:00)</li></ul>
 <i>for all good men</i> to come to the {007} aid of their</bold> ESP. Result: (absent)

Answer 2

您需要的是递归下降解析器。这里讨论一个http://www.drdobbs.com/cpp/recursive-descent-peg-parsers-using-c-te/212700432

Answer 3

以下是来自@sehe的正确完整代码，现在可以在GCC＆gt; 4.8和CLANG下在Linux和Windows下运行。再次非常感谢这个令人敬畏的帮助，即使这意味着我可以埋葬XPressive：D

以下行已更改或已添加：

// --
#define BOOST_RESULT_OF_USE_DECLTYPE
// --
struct to_string_f {
template <typename T>
std::string operator()(T const& v) const { return v.to_string(); }};
// --
section     %= "{{" >> sense >> reference [ section_id = to_string(_1) ] >> "}}"
                >> sequence // contents
                > ("{{" >> ('/' >> lexeme [ lit(section_id) ]) >> "}}");
// --
phx::function<to_string_f> to_string;

//#define BOOST_SPIRIT_DEBUG
#define BOOST_RESULT_OF_USE_DECLTYPE
#define BOOST_SPIRIT_USE_PHOENIX_V3
#include <boost/fusion/adapted/struct.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/utility/string_ref.hpp>
#include <functional>
#include <map>

namespace mustache {

    // any atom refers directly to source iterators for efficiency
    using boost::string_ref;
    template <typename Kind> struct atom {
        string_ref value;

        atom() { }
        atom(string_ref const& value) : value(value) { }

        friend std::ostream& operator<<(std::ostream& os, atom const& v) { return os << typeid(v).name() << "[" << v.value << "]"; }
    };

    // the atoms
    using verbatim = atom<struct verbatim_tag>;
    using variable = atom<struct variable_tag>;
    using partial  = atom<struct partial_tag>;

    // the template elements (any atom or a section)
    struct section;

    using melement = boost::variant<
            verbatim,
            variable,
            partial, // TODO comments and set-separators
            boost::recursive_wrapper<section>
        >;

    // the template: sequences of elements
    using sequence = std::vector<melement>;

    // section: recursively define to contain a template sequence
    struct section {
        bool       sense; // positive or negative
        string_ref control;
        sequence   content;
    };
}

BOOST_FUSION_ADAPT_STRUCT(mustache::section, (bool, sense)(boost::string_ref, control)(mustache::sequence, content))

namespace qi = boost::spirit::qi;
namespace phx= boost::phoenix;

struct to_string_f {
    template <typename T>
    std::string operator()(T const& v) const { return v.to_string(); }
};

template <typename Iterator>
    struct mustache_grammar : qi::grammar<Iterator, mustache::sequence()>
{
    mustache_grammar() : mustache_grammar::base_type(sequence)
    {
        using namespace qi;
        static const _a_type section_id = {}; // local
        using boost::phoenix::construct;
        using boost::phoenix::begin;
        using boost::phoenix::size;

        sequence     = *element;
        element      = 
                    !(lit("{{") >> '/') >> // section-end ends the current sequence
                    (partial | section | variable | verbatim);

        reference    = raw [ lexeme [ +(graph - "}}") ] ]
                        [ _val = construct<boost::string_ref>(&*begin(_1), size(_1)) ];

        partial      = qi::lit("{{") >> "> " >> reference >> "}}";

        sense        = ('#' > attr(true))
                     | ('^' > attr(false));

        section     %= "{{" >> sense >> reference [ section_id = to_string(_1) ] >> "}}"
                    >> sequence // contents
                    > ("{{" >> ('/' >> lexeme [ lit(section_id) ]) >> "}}");

        variable     = "{{" >> reference >> "}}";

        verbatim     = raw [ lexeme [ +(char_ - "{{") ] ]
                        [ _val = construct<boost::string_ref>(&*begin(_1), size(_1)) ];

        BOOST_SPIRIT_DEBUG_NODES(
                (sequence)(element)(partial)(variable)(section)(verbatim)
                (reference)(sense)
            )
    }
  private:
    phx::function<to_string_f> to_string;
    qi::rule<Iterator, mustache::sequence()> sequence;
    qi::rule<Iterator, mustache::melement()> element;
    qi::rule<Iterator, mustache::partial()>  partial;
    qi::rule<Iterator, mustache::section(), qi::locals<std::string> >  section;
    qi::rule<Iterator, bool()>                sense;                  // postive  or negative
    qi::rule<Iterator, mustache::variable()> variable;
    qi::rule<Iterator, mustache::verbatim()> verbatim;
    qi::rule<Iterator, boost::string_ref()>   reference;
};

namespace Dumping {
    struct dumper : boost::static_visitor<std::ostream&>
    {
        std::ostream& operator()(std::ostream& os, mustache::sequence const& v) const {
            for(auto& element : v)
                boost::apply_visitor(std::bind(dumper(), std::ref(os), std::placeholders::_1), element);
            return os;
        }
        std::ostream& operator()(std::ostream& os, mustache::verbatim const& v) const {
            return os << v.value;
        }
        std::ostream& operator()(std::ostream& os, mustache::variable const& v) const {
            return os << "{{" << v.value << "}}";
        }
        std::ostream& operator()(std::ostream& os, mustache::partial const& v) const {
            return os << "{{> " << v.value << "}}";
        }
        std::ostream& operator()(std::ostream& os, mustache::section const& v) const {
            os << "{{" << (v.sense?'#':'^') << v.control << "}}";
            (*this)(os, v.content);
            return os << "{{/" << v.control << "}}";
        }
    };
}

namespace ContextExpander {

    struct Nil { };

    using Value = boost::make_recursive_variant<
        Nil,
        double,
        std::string,
        std::map<std::string, boost::recursive_variant_>,
        std::vector<boost::recursive_variant_>
    >::type;

    using Dict  = std::map<std::string, Value>;
    using Array = std::vector<Value>;

    static inline std::ostream& operator<<(std::ostream& os, Nil   const&)   { return os << "#NIL#"; }
    static inline std::ostream& operator<<(std::ostream& os, Dict  const& v) { return os << "#DICT("  << v.size() << ")#"; }
    static inline std::ostream& operator<<(std::ostream& os, Array const& v) { return os << "#ARRAY(" << v.size() << ")#"; }

    struct expander : boost::static_visitor<std::ostream&>
    {
        std::ostream& operator()(std::ostream& os, Value const& ctx, mustache::sequence const& v) const {
            for(auto& element : v)
                boost::apply_visitor(std::bind(expander(), std::ref(os), std::placeholders::_1, std::placeholders::_2), ctx, element);
            return os;
        }

        template <typename Ctx>
        std::ostream& operator()(std::ostream& os, Ctx const&/*ignored*/, mustache::verbatim const& v) const {
            return os << v.value;
        }

        std::ostream& operator()(std::ostream& os, Dict const& ctx, mustache::variable const& v) const {
            auto it = ctx.find(v.value.to_string());
            if (it != ctx.end())
                os << it->second;
            return os;
        }

        template <typename Ctx>
        std::ostream& operator()(std::ostream& os, Ctx const&, mustache::variable const&) const {
            return os;
        }

        std::ostream& operator()(std::ostream& os, Dict const& ctx, mustache::partial const& v) const {
            auto it = ctx.find(v.value.to_string());
            if (it != ctx.end())
            {
                static const mustache_grammar<std::string::const_iterator> p;

                auto const& subtemplate = boost::get<std::string>(it->second);
                std::string::const_iterator first = subtemplate.begin(), last = subtemplate.end();

                mustache::sequence dynamic_template;
                if (qi::parse(first, last, p, dynamic_template))
                    return (*this)(os, Value{ctx}, dynamic_template);
            }
            return os << "#ERROR#";
        }

        std::ostream& operator()(std::ostream& os, Dict const& ctx, mustache::section const& v) const {
            auto it = ctx.find(v.control.to_string());
            if (it != ctx.end())
                boost::apply_visitor(std::bind(do_section(), std::ref(os), std::placeholders::_1, std::cref(v)), it->second);
            else if (!v.sense)
                (*this)(os, Value{/*Nil*/}, v.content);

            return os;
        }

        template <typename Ctx, typename T>
        std::ostream& operator()(std::ostream& os, Ctx const&/* ctx*/, T const&/* element*/) const {
            return os << "[TBI:" << __PRETTY_FUNCTION__ << "]";
        }

      private:
        struct do_section : boost::static_visitor<> {
            void operator()(std::ostream& os, Array const& ctx, mustache::section const& v) const {
                for(auto& item : ctx)
                    expander()(os, item, v.content);
            }
            template <typename Ctx>
            void operator()(std::ostream& os, Ctx const& ctx, mustache::section const& v) const {
                if (v.sense == truthiness(ctx))
                    expander()(os, Value(ctx), v.content);
            }
          private:
            static bool truthiness(Nil)                              { return false; }
            static bool truthiness(double d)                         { return 0. == d; }
            template <typename T> static bool truthiness(T const& v) { return !v.empty(); }
        };
    };

}

int myMain()
{
    std::cout << std::unitbuf;
    std::string input = "<ul>{{#time}}\n\t<li>{{> partial}}</li>{{/time}}</ul>\n "
        "<i>for all good men</i> to come to the {007} aid of "
        "their</bold> {{country}}. Result: {{^Res2}}(absent){{/Res2}}{{#Res2}}{{Res2}}{{/Res2}}"
        ;
    // Parser setup --------------------------------------------------------
    typedef std::string::const_iterator It;
    static const mustache_grammar<It> p;

    It first = input.begin(), last = input.end();

    try {
        mustache::sequence parsed_template;
        if (qi::parse(first, last, p, parsed_template))
        {
            std::cout << "Parse success\n";
        } else
        {
            std::cout << "Parse failed\n";
        }

        if (first != last)
        {
            std::cout << "Remaing unparsed input: '" << std::string(first, last) << "'\n";
        }

        std::cout << "Input:      " << input << "\n";
        std::cout << "Dump:       ";
        Dumping::dumper()(std::cout, parsed_template) << "\n";

        std::cout << "Evaluation: ";

        {
            using namespace ContextExpander;
            expander engine;

            Value const ctx = Dict { 
                { "time", Array {
                    Dict { { "partial", "gugus {{zeit}} (a.k.a. <u>{{title}}</u>)"},             { "title", "noon" },    { "zeit", "12:00" } },
                    Dict { { "partial", "gugus {{zeit}} (a.k.a. <u>{{title}}</u>)"},             { "title", "evening" }, { "zeit", "19:30" } },
                    Dict { { "partial", "gugus <u>{{title}}</u> (expected at around {{zeit}})"}, { "title", "dawn" },    { "zeit", "06:00" } },
                } },
                { "country", "ESP" },
                { "Res3", "unused" }
            };

            engine(std::cout, ctx, parsed_template);
        }
    } catch(qi::expectation_failure<It> const& e)
    {
        std::cout << "Unexpected: '" << std::string(e.first, e.last) << "'\n";
    }
}

如何正确解析Boost.Xpressive的胡子？

我有以下问题，我无法单独解决：

3 个答案:

完整代码清单