erasePunctuation — Erase Unicode punctuation and symbol characters from text or selected token types in tokenized documents.
erasePunctuation removes characters in the Unicode punctuation (P*) and symbol (S*) categories from documented text containers. For tokenized documents, it removes punctuation only from the selected token types and removes tokens that become empty.
Syntax
newStr = erasePunctuation(str)
newDocuments = erasePunctuation(documents)
newDocuments = erasePunctuation(documents, 'TokenTypes', types)Inputs
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
text | Any | Yes | — | Input text. |
arg | Any | Variadic | — | Additional arguments. |
Returns
| Name | Type | Description |
|---|---|---|
newStr | Any | Text with matching content erased. |
newDocuments | Any | Updated tokenized documents. |
Errors
| Identifier | When | Message |
|---|---|---|
RunMat:erasePunctuation:InvalidInput | Input is not documented host text or a tokenizedDocument object. | erasePunctuation: input must be text or tokenizedDocument |
RunMat:erasePunctuation:InvalidOption | TokenTypes is malformed, empty, or used with non-document text. | erasePunctuation: invalid TokenTypes option |
How erasePunctuation works
newStr = erasePunctuation(str)accepts a string array, character vector, or cell array of character vectors and returns the same documented container type.newDocuments = erasePunctuation(documents)processes token typespunctuationandotherby default, preserving complex tokens such as web addresses and email addresses.newDocuments = erasePunctuation(documents,'TokenTypes',types)processes only the supplied built-in or custom token types.- RunMat preserves document shape and explicit token type details in its current scalar compatibility-object representation.
- All integer and logical values are invalid in text, document, option-name, and TokenTypes roles; no implicit numeric-to-text conversion occurs.
- RunMat can process character matrices row-wise and nested or non-cellstr text cells, but those containers are independently gated RunMat-only extensions beyond the documented character-vector and cellstr surface.
GPU memory and residency
Host-only text processing; no provider hook or resident text representation.
Examples
Erase punctuation from text
newStr = erasePunctuation("it's one and/or two.")Expected output:
newStr = "its one andor two"Preserve complex document tokens by default
documents = tokenizedDocument("Visit https://example.com.");
newDocuments = erasePunctuation(documents)Expected output:
The web-address token remains intact while the sentence punctuation token is removed.Using erasePunctuation with coding agents
Open a RunMat example with live inputs, then ask the agent to explain how erasePunctuation changes the result.
Run a small erasePunctuation example, explain the result, then change one input and compare the output.
FAQ
Does erasePunctuation run on a GPU?⌄
No. It is host text processing and has no provider kernel. Resident numeric values are invalid text and reject without gathering.
Are tokenizedDocument arrays represented exactly?⌄
RunMat currently uses one scalar compatibility object carrying document cells and Shape metadata. First-class object-array identity remains a general runtime infrastructure gap.
Related Strings functions
Transform
append · deblank · erase · eraseBetween · eraseURLs · extractAfter · extractBefore · extractBetween · insertAfter · insertBefore · join · lower · pad · replace · replaceBetween · reverse · split · splitlines · strcat · strip · strjoin · strjust · strrep · strsplit · strtrim · upper
Text Analytics
addDependencyDetails · addEntityDetails · addLemmaDetails · addPartOfSpeechDetails · addSentenceDetails · addTypeDetails · bagOfNgrams · bagOfWords · cosineSimilarity · doc2sequence · encode · extractFileText · extractHTMLText · fastTextWordEmbedding · findElement · getAttribute · htmlTree · ind2word · isVocabularyWord · normalizeWords · readWordEmbedding · removeLongWords · removeShortWords · removeStopWords · removeWords · stopWords · tokenDetails · tokenizedDocument · trainWordEmbedding · vaderSentimentScores · vec2word · word2ind · word2vec · wordEncoding · writeWordEmbedding
Core
blanks · char · compose · convertCharsToStrings · convertContainedStringsToChars · convertStringsToChars · genvarname · int2str · isletter · isspace · isStringScalar · isstrprop · mat2str · native2unicode · newline · num2str · sprintf · sscanf · str2double · str2num · strcmp · strcmpi · string · string.empty · strings · strlength · strncmp · strncmpi · strtok · unicode2native
Search
contains · endsWith · matches · startsWith · strfind
Pattern
digitsPattern · lettersPattern · pattern · regexpPattern · textBoundary · wildcardPattern
Open-source implementation
Unlike proprietary runtimes, every RunMat function is open-source. Read exactly how erasePunctuation is executed, line by line, in Rust.
- View the source for erasePunctuation in Rust on GitHub
- Learn how the RunMat runtime works
- Found a bug? Open an issue with a minimal reproduction.
About RunMat
RunMat is an open-source runtime that executes MATLAB-syntax code blazing on any GPU. It is licensed under the Apache 2.0 license.
- RunMat automatically optimizes your math for GPU execution on Apple, Nvidia, and AMD hardware. No code changes needed. Simulations that took hours now take minutes.
- Start running code in seconds. RunMat runs in the browser, on the desktop, or from the CLI. No license server, no IT ticket.