How do I tokenize input using Java's Scanner class and regular expressions?

Question

Just for my own purposes, I\'m trying to build a tokenizer in Java where I can define a regular grammar and have it tokenize input based on that. The StringTokenizer class i

Answer 1

If this is for a simple project (for learning how things work), then go with what Balint Pato said.

If this is for a larger project, consider using a scanner generator like JFlex instead. Somewhat more complicated, but faster and more powerful.