Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor mbe a tiny bit #64759

Merged
merged 3 commits into from
Sep 25, 2019
Merged

Refactor mbe a tiny bit #64759

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9a02018
remove unused parameters
matklad Sep 25, 2019
538437e
move function closer to its usage
matklad Sep 25, 2019
f60a873
remove unused peekable
matklad Sep 25, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 0 additions & 12 deletions src/libsyntax/ext/mbe/macro_parser.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,18 +413,6 @@ fn nameize<I: Iterator<Item = NamedMatch>>(
Success(ret_val)
}

/// Generates an appropriate parsing failure message. For EOF, this is "unexpected end...". For
/// other tokens, this is "unexpected token...".
crate fn parse_failure_msg(tok: &Token) -> String {
match tok.kind {
token::Eof => "unexpected end of macro invocation".to_string(),
_ => format!(
"no rules expected the token `{}`",
pprust::token_to_string(tok)
),
}
}

/// Performs a token equality check, ignoring syntax context (that is, an unhygienic comparison)
fn token_name_eq(t1: &Token, t2: &Token) -> bool {
if let (Some((ident1, is_raw1)), Some((ident2, is_raw2))) = (t1.ident(), t2.ident()) {
Expand Down
23 changes: 14 additions & 9 deletions src/libsyntax/ext/mbe/macro_rules.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ use crate::ext::base::{SyntaxExtension, SyntaxExtensionKind};
use crate::ext::expand::{AstFragment, AstFragmentKind};
use crate::ext::mbe;
use crate::ext::mbe::macro_check;
use crate::ext::mbe::macro_parser::{parse, parse_failure_msg};
use crate::ext::mbe::macro_parser::parse;
use crate::ext::mbe::macro_parser::{Error, Failure, Success};
use crate::ext::mbe::macro_parser::{MatchedNonterminal, MatchedSeq, NamedParseResult};
use crate::ext::mbe::transcribe::transcribe;
Expand All @@ -15,6 +15,7 @@ use crate::parse::parser::Parser;
use crate::parse::token::TokenKind::*;
use crate::parse::token::{self, NtTT, Token};
use crate::parse::{Directory, ParseSess};
use crate::print::pprust;
use crate::symbol::{kw, sym, Symbol};
use crate::tokenstream::{DelimSpan, TokenStream, TokenTree};

Expand Down Expand Up @@ -371,10 +372,6 @@ pub fn compile_declarative_macro(
tt.clone().into(),
true,
sess,
features,
&def.attrs,
edition,
def.id,
)
.pop()
.unwrap();
Expand All @@ -398,10 +395,6 @@ pub fn compile_declarative_macro(
tt.clone().into(),
false,
sess,
features,
&def.attrs,
edition,
def.id,
)
.pop()
.unwrap();
Expand Down Expand Up @@ -1184,3 +1177,15 @@ impl TokenTree {
parse(cx.parse_sess(), tts, mtch, Some(directory), true)
}
}

/// Generates an appropriate parsing failure message. For EOF, this is "unexpected end...". For
/// other tokens, this is "unexpected token...".
fn parse_failure_msg(tok: &Token) -> String {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not

fn parse_failure_msg(tok: &Token) -> String {
    if let token::Eof = tok.kind {
        return "unexpected end of macro invocation".to_string();
    }
    format!("no rules expected the token `{}`", pprust::token_to_string(tok))
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just haven't looked into the function body at all :-) I think that the best way to format it would probably be

fn parse_failure_msg(tok: &Token) -> String {
    match tok.kind {
        token::Eof => "unexpected end of macro invocation".to_string(),
        _ => format!("no rules expected the token `{}`", pprust::token_to_string(tok)),
    }
}

Not a fan of if let with an else case

match tok.kind {
token::Eof => "unexpected end of macro invocation".to_string(),
_ => format!(
"no rules expected the token `{}`",
pprust::token_to_string(tok),
),
}
}
31 changes: 4 additions & 27 deletions src/libsyntax/ext/mbe/quoted.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,15 @@
use crate::ast;
use crate::ast::NodeId;
use crate::ext::mbe::macro_parser;
use crate::ext::mbe::{TokenTree, KleeneOp, KleeneToken, SequenceRepetition, Delimited};
use crate::feature_gate::Features;
use crate::parse::token::{self, Token};
use crate::parse::ParseSess;
use crate::print::pprust;
use crate::symbol::kw;
use crate::tokenstream;

use syntax_pos::{edition::Edition, Span};
use syntax_pos::Span;

use rustc_data_structures::sync::Lrc;
use std::iter::Peekable;

/// Takes a `tokenstream::TokenStream` and returns a `Vec<self::TokenTree>`. Specifically, this
/// takes a generic `TokenStream`, such as is used in the rest of the compiler, and returns a
Expand All @@ -39,17 +36,13 @@ pub(super) fn parse(
input: tokenstream::TokenStream,
expect_matchers: bool,
sess: &ParseSess,
features: &Features,
attrs: &[ast::Attribute],
edition: Edition,
macro_node_id: NodeId,
) -> Vec<TokenTree> {
// Will contain the final collection of `self::TokenTree`
let mut result = Vec::new();

// For each token tree in `input`, parse the token into a `self::TokenTree`, consuming
// additional trees if need be.
let mut trees = input.trees().peekable();
let mut trees = input.trees();
while let Some(tree) = trees.next() {
// Given the parsed tree, if there is a metavar and we are expecting matchers, actually
// parse out the matcher (i.e., in `$id:ident` this would parse the `:` and `ident`).
Expand All @@ -58,10 +51,6 @@ pub(super) fn parse(
&mut trees,
expect_matchers,
sess,
features,
attrs,
edition,
macro_node_id,
);
match tree {
TokenTree::MetaVar(start_sp, ident) if expect_matchers => {
Expand Down Expand Up @@ -109,13 +98,9 @@ pub(super) fn parse(
/// unstable features or not.
fn parse_tree(
tree: tokenstream::TokenTree,
trees: &mut Peekable<impl Iterator<Item = tokenstream::TokenTree>>,
trees: &mut impl Iterator<Item = tokenstream::TokenTree>,
expect_matchers: bool,
sess: &ParseSess,
features: &Features,
attrs: &[ast::Attribute],
edition: Edition,
macro_node_id: NodeId,
) -> TokenTree {
// Depending on what `tree` is, we could be parsing different parts of a macro
match tree {
Expand All @@ -135,10 +120,6 @@ fn parse_tree(
tts.into(),
expect_matchers,
sess,
features,
attrs,
edition,
macro_node_id,
);
// Get the Kleene operator and optional separator
let (separator, kleene) = parse_sep_and_kleene_op(trees, span.entire(), sess);
Expand Down Expand Up @@ -192,10 +173,6 @@ fn parse_tree(
tts.into(),
expect_matchers,
sess,
features,
attrs,
edition,
macro_node_id,
),
}),
),
Expand Down Expand Up @@ -244,7 +221,7 @@ fn parse_kleene_op(
/// operator and separator, then a tuple with `(separator, KleeneOp)` is returned. Otherwise, an
/// error with the appropriate span is emitted to `sess` and a dummy value is returned.
fn parse_sep_and_kleene_op(
input: &mut Peekable<impl Iterator<Item = tokenstream::TokenTree>>,
input: &mut impl Iterator<Item = tokenstream::TokenTree>,
span: Span,
sess: &ParseSess,
) -> (Option<Token>, KleeneToken) {
Expand Down