Java StringTokenizer, empty null tokens

Question

I am trying to split a string into 29 tokens..... stringtokenizer won\'t return null tokens. I tried string.split, but I believe I am doing something wrong:

Answer 1

use this org.springframework.util.StringUtils

org.springframework.util.StringUtils.delimitedListToStringArray(data, delimit);

This class delivers some simple functionality provides easy-to-use methods to convert between delimited strings, such as CSV strings, and collections and arrays.

Answer 2

If you want the trailing empty strings to be kept, but you don't want to give a magic number for maximum, use a negative limit:

line.split(",", -1)

If line.equals("a,,c"), then line.split(",", -1)[1].isEmpty(); it's not null. This is because when "," is the delimiter, then ",," has an empty string between the two delimiters, not null.

---

Example:

Using the explanation above, consider the following example: ",,"

Although you might expect ",", null, and ",".

The actual result is ",", "" and ","

---

If you want null instead of empty strings in the array returned by split, then you'd have to manually scan the array and replace them with null. I'm not sure why s == null is better than s.isEmpty(), though.

See also

• Java String.indexOf and empty strings

Answer 3

If you want empty tokens to be retained string.split() won't work satisfactorily. StringTokenizer will also not work. I have come with following method, which might be helpful for you:

public static String[] splitTotokens(String line, String delim){
    String s = line;
    int i = 0;

    while (s.contains(delim)) {
        s = s.substring(s.indexOf(delim) + delim.length());
        i++;
    }
    String token = null;
    String remainder = null;
    String[] tokens = new String[i];

    for (int j = 0; j < i; j++) {
        token = line.substring(0, line.indexOf(delim));
        // System.out.print("#" + token + "#");
        tokens[j] = token;
        remainder = line.substring(line.indexOf(delim) + delim.length());
        //System.out.println("#" + remainder + "#");

        line = remainder;
    }
    return tokens;
}

Answer 4

If you want empty tokens to be retained string.split won't work satisfactorily. StringTokenizer will also no work. I have come with following method, which might be helpful for you

public static String[] splitTotokens(String line, String delim){
  String s = line;
  int i = 0;

  while (s.contains(delim)) {
  s = s.substring(s.indexOf(delim) + delim.length());
      i++;
  }
  String token = null;
  String remainder = null;
  String[] tokens = new String[i];

  for (int j = 0; j < i; j++) {
    token = line.substring(0, line.indexOf(delim));
    //System.out.print("#" + token + "#");
    tokens[j] = token;
    remainder = line.substring(line.indexOf(delim) + delim.length());
    //System.out.println("#" + remainder + "#");
    line = remainder;
    }

  return tokens;`  
 }

Answer 5

Use StringUtils.splitPreserveAllTokens() in Apache Commons Lang library