Navigation: Developing Apps > BINETIX® AGE™ API > The Node > Toolkits > age_lists
tokens/2
Description
tokens/2 is an Erlang function exported by "age_lists" module.
Simple tokenizer to groups terms by selected separator.
This function could be used for analyzing ASCII strings as list of bytes.
References
None
Parameters
Name
Type
Input
list() | string()
Input list of terms or a string sequence.
Separator
integer()
Integer separator.
Result
Value or Type
list()
List of grouped tokens.
Code Sample
> age_lists:tokens([a, 32, b, 3, 32, c], 32). [[a], [b,3], [c]] > age_lists:tokens("a=1;b=2;c=3", ";"). ["a=1", "b=2", "c=3"]