strtok

Selected parts of strings

Syntax

token = strtok(str)

token = strtok(str,delimiters)

[token,remain] = strtok(___)

Description

token = strtok(str) parses str from left to right, using whitespace characters as delimiters, and returns part or all of the text in token. First, strtok ignores any leading whitespace in str. Then, strtok starts at the first character that is not whitespace, and includes all characters up to, but not including, the next whitespace character. strtok returns that part of the text in token. If strtok does not find any whitespace to use as a delimiter, then token includes all characters up to, and including, the end of str.

token = strtok(str,delimiters) parses str using the characters in delimiters. If delimiters includes more than one character, then strtok treats each character in delimiters as a separate delimiter. Because the delimiters are individual characters, delimiters can be any size, and the characters within delimiters can be in any order.

In this syntax, whitespace characters are not delimiters unless you include them within delimiters.

[token,remain] = strtok(___) returns the remaining text, if any, in remain. If strtok finds a delimiter, then it is included at the start of remain. If strtok finds no delimiters in str, then it returns the whole of str, except for leading delimiters, in token, and remain has no characters. You can use this syntax with any of the input arguments of the previous syntaxes.

Examples

Return Token from Character Vector

Create a character vector. Return all text up to the first whitespace character that is a delimiter. strtok discards leading whitespace characters.

chr = '     Four score and seven years ago'

chr = 
'     Four score and seven years ago'

token = strtok(chr)

token = 
'Four'

Return Token Using Delimiter

Create a string.

str = "A horse! A horse! My kingdom for a horse!"

str = 
"A horse! A horse! My kingdom for a horse!"

Return the first part of the string using the '!' character as a delimiter.

token = strtok(str,'!')

token = 
"A horse"

Return Tokens and Remaining Text

Create a string array.

str = ["all in good time";
       "my dog has fleas";
       "leave no stone unturned"]

str = 3x1 string
    "all in good time"
    "my dog has fleas"
    "leave no stone unturned"

Return tokens in a string array, and return the remaining text in a second string array.

[token,remain] = strtok(str)

token = 3x1 string
    "all"
    "my"
    "leave"

remain = 3x1 string
    " in good time"
    " dog has fleas"
    " no stone unturned"

Parse Delimited Text into String Array

Create a string that contains sample HTML code. Break it down into segments delimited by the < and > characters. Store the segments in a string array.

You can create strings using double quotes. To concatenate several strings into one string, use the plus operator, +.

str = "<ul class=continued><li class=continued>" + ...
      "<pre><a name=""13474""></a>token = strtok" + ...
      "(str,delimiter)<a name=""13475""></a>" + ...
      "token = strtok(str)"

str = 
"<ul class=continued><li class=continued><pre><a name="13474"></a>token = strtok(str,delimiter)<a name="13475"></a>token = strtok(str)"

Create an empty string array to contain the code segments.

segments = strings(0)

segments = 

  0x0 empty string array

Break str into segments. Write a while loop that repeatedly calls strtok on the remaining HTML text. The while loop exits when there is no more text to parse.

remain = str;
while (remain ~= "")
   [token,remain] = strtok(remain, '<>');
   segments = [segments ; token];
end

Display the code segments.

segments

segments = 9x1 string
    "ul class=continued"
    "li class=continued"
    "pre"
    "a name="13474""
    "/a"
    "token = strtok(str,delimiter)"
    "a name="13475""
    "/a"
    "token = strtok(str)"

Input Arguments

str — Input text

string array | character vector | cell array of character vectors

Input text, specified as a string array, a character vector, or a cell array of character vectors.

Data Types: string | char | cell

delimiters — Delimiting characters

string array | character vector | cell array of character vectors

Delimiting characters, specified as a string array, a character vector, or a cell array of character vectors. Because strtok treats individual characters as delimiters, delimiters can be any size and can contain characters in any order.

Example: strtok(str,{'YZ','X'}) treats 'X', 'Y', and 'Z' as separate delimiters. It does not treat 'YZ' as a delimiter.

Data Types: string | char | cell

Output Arguments

token — Selected part of text

string array | character vector | cell array of character vectors

Selected part of the text, returned as a string array, a character vector, or a cell array of character vectors. token includes all text starting at the first character that is not a delimiter and ending at, but not including, the next delimiter. str and token are the same data type.

Data Types: string | char | cell

remain — Remainder of text

string array | character vector | cell array of character vectors

Remainder of the text, returned as a string array, a character vector, or a cell array of character vectors. If strtok finds a delimiter in str, then remain includes all text starting at, and including, that delimiter and ending at the end of the text. str and remain are the same data type.

Data Types: string | char | cell

Tips

Do not specify an escape-character sequence as a delimiter. strtok does not translate escape character sequences. Instead, you can use the char function to specify such characters. For example, to specify a tab as a delimiter use char(9) instead of '\t'.

Extended Capabilities

C/C++ Code Generation

Generate C and C++ code using MATLAB® Coder™.

Usage notes and limitations:

• str and delimiters must be a string scalar or a character vector.

• Generated code returns an empty output as a 1-by-0 character array.

Thread-Based Environment

Run code in the background using MATLAB® backgroundPool or accelerate code with Parallel Computing Toolbox™ ThreadPool.

This function fully supports thread-based environments. For more information, see Run MATLAB Functions in Thread-Based Environment.

Version History

Introduced before R2006a