#include <cstringtokenizer.h>

Description

String tokenizer class, modelled after strtok().

It considers the input string to consist of tokens, separated by one or more delimiter characters. Repeated calls to nextToken() will enumerate the tokens in the string, returning nullptr after the last token. The function hasMoreTokens() can be used to find out whether there are more tokens without consuming one.

Limitations: this class does not honor quotes, apostrophes or backslash quoting; nor does it return empty tokens if it encounters multiple delimiter characters in a row (so setting the delimiter to "," does not produce the desired results). This behaviour is consistent with strtok().

Example 1:

const char *str = "one two three four";
cStringTokenizer tokenizer(str);
while (tokenizer.hasMoreTokens())
    EV << " [" << tokenizer.nextToken() << "]";

Example 2:

const char *str = "42 13 46 72 41";
std::vector<int> array = cStringTokenizer(str).asIntVector();

Public Member Functions
	cStringTokenizer (const char str, const char delimiters=nullptr)

	cStringTokenizer (const cStringTokenizer &other)

	~cStringTokenizer ()

cStringTokenizer &	operator= (const cStringTokenizer &other)

void	setDelimiter (const char *s)

bool	hasMoreTokens ()

const char *	nextToken ()

std::vector< std::string >	asVector ()

std::vector< int >	asIntVector ()

std::vector< double >	asDoubleVector ()

Constructor & Destructor Documentation

◆ cStringTokenizer() [1/2]

cStringTokenizer	(	const char *	str,
		const char *	delimiters = `nullptr`
	)

Constructor. The class will make its own copy of the input string and of the delimiters string. The delimiters default to all whitespace characters (space, tab, CR, LF, FF).

◆ cStringTokenizer() [2/2]

cStringTokenizer ( const cStringTokenizer & other )

Copy constructor. It copies the current state of the tokenizer (i.e. does not re-position it to the first token.)

◆ ~cStringTokenizer()

~cStringTokenizer ( )

Destructor.

Member Function Documentation

◆ operator=()

cStringTokenizer& operator= ( const cStringTokenizer & other )

Assignment operator. It copies the current state of the tokenizer (i.e. does not re-position it to the first token.)

◆ setDelimiter()

void setDelimiter ( const char * s )

Change delimiters. This allows for switching delimiters during tokenization.

◆ hasMoreTokens()

bool hasMoreTokens ( )

Returns true if there're more tokens (i.e. the next nextToken() call won't return nullptr).

◆ nextToken()

const char* nextToken ( )

Returns the next token. The returned pointers will stay valid as long as the tokenizer object exists. If there're no more tokens, nullptr will be returned.

◆ asVector()

std::vector<std::string> asVector ( )

Utility function: tokenizes the full input string at once, and returns the tokens in a string vector.

◆ asIntVector()

std::vector<int> asIntVector ( )

Utility function: converts all tokens to int, and returns them in a vector.

◆ asDoubleVector()

std::vector<double> asDoubleVector ( )

Utility function: converts all tokens to double, and returns them in a vector.

The documentation for this class was generated from the following file:

cstringtokenizer.h