erasePunctuation — Remove Unicode punctuation and symbol characters from text and tokenized documents.
erasePunctuation(text) removes characters in Unicode punctuation and symbol categories from string, character, and cell text inputs while preserving the input container shape and type. erasePunctuation(documents) filters punctuation and symbol tokens from RunMat tokenizedDocument compatibility objects.
Syntax
out = erasePunctuation(textOrDocuments, Name, Value, ...)Inputs
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
text | Any | Yes | — | Input text. |
arg | Any | Variadic | — | Additional arguments. |
Returns
| Name | Type | Description |
|---|---|---|
out | Any | Output value. |
How erasePunctuation works
- String scalars and string arrays return string values with punctuation and symbol characters removed from each nonmissing element.
- Character vectors and character arrays return character arrays; rows are padded after punctuation removal when needed to preserve rectangular char storage.
- Cell arrays are processed recursively for string and character text elements and retain the original cell shape.
- Missing string elements are preserved.
erasePunctuation(documents)removes punctuation and symbol characters from tokens with derived typepunctuationorotherand removes tokens that become empty.erasePunctuation(documents,'TokenTypes',types)removes punctuation and symbols only from the specified derived token types. RunMat uses the same lightweight token-type classifier exposed bytokenDetailsandaddTypeDetails.- For tokenized documents, URLs, email addresses, hashtags, mentions, and emoticons are preserved by default.
GPU memory and residency
erasePunctuation operates on host text and has no provider kernel.
Examples
Erase Punctuation From A String
erasePunctuation("it's one and/or two.")Expected output:
"its one andor two"Erase Punctuation And Symbols From A Character Vector
erasePunctuation('cost: $5.00!')Expected output:
'cost 500'Preserve Cell Array Shape
erasePunctuation({'alpha,beta!','C++ and C#'})Expected output:
{'alphabeta','C and C'}Erase Punctuation From Tokenized Documents
documents = tokenizedDocument("An example: visit https://example.com.");
newDocuments = erasePunctuation(documents)Expected output:
`newDocuments` contains `An example visit https://example.com`.Using erasePunctuation with coding agents
Open a RunMat example with live inputs, then ask the agent to explain how erasePunctuation changes the result.
Run a small erasePunctuation example, explain the result, then change one input and compare the output.
FAQ
Does erasePunctuation remove spaces?⌄
No. Whitespace is not in a Unicode punctuation or symbol category, so spaces are preserved.
Does this implement tokenizedDocument token-type filtering?⌄
Yes for RunMat's lightweight tokenizedDocument objects. It supports default punctuation/other filtering and explicit TokenTypes; full custom token-detail tables and linguistic detail columns remain tracked separately.
Related Strings functions
Transform
append · deblank · erase · eraseBetween · eraseURLs · extractAfter · extractBefore · extractBetween · insertAfter · insertBefore · join · lower · pad · replace · replaceBetween · reverse · split · splitlines · strcat · strip · strjoin · strjust · strrep · strsplit · strtrim · upper
Text Analytics
addDependencyDetails · addEntityDetails · addLemmaDetails · addPartOfSpeechDetails · addSentenceDetails · addTypeDetails · bagOfNgrams · bagOfWords · cosineSimilarity · doc2sequence · encode · extractFileText · extractHTMLText · fastTextWordEmbedding · findElement · getAttribute · htmlTree · ind2word · isVocabularyWord · normalizeWords · readWordEmbedding · removeLongWords · removeShortWords · removeStopWords · removeWords · stopWords · tokenDetails · tokenizedDocument · trainWordEmbedding · vaderSentimentScores · vec2word · word2ind · word2vec · wordEncoding · writeWordEmbedding
Core
blanks · char · compose · convertCharsToStrings · convertContainedStringsToChars · convertStringsToChars · genvarname · int2str · isletter · isspace · isStringScalar · isstrprop · mat2str · native2unicode · newline · num2str · sprintf · sscanf · str2double · str2num · strcmp · strcmpi · string · string.empty · strings · strlength · strncmp · strncmpi · strtok · unicode2native
Search
contains · endsWith · matches · startsWith · strfind
Pattern
digitsPattern · lettersPattern · pattern · regexpPattern · textBoundary · wildcardPattern
Open-source implementation
Unlike proprietary runtimes, every RunMat function is open-source. Read exactly how erasePunctuation is executed, line by line, in Rust.
- View the source for erasePunctuation in Rust on GitHub
- Learn how the RunMat runtime works
- Found a bug? Open an issue with a minimal reproduction.
About RunMat
RunMat is an open-source runtime that executes MATLAB-syntax code blazing on any GPU. It is licensed under the Apache 2.0 license.
- RunMat automatically optimizes your math for GPU execution on Apple, Nvidia, and AMD hardware. No code changes needed. Simulations that took hours now take minutes.
- Start running code in seconds. RunMat runs in the browser, on the desktop, or from the CLI. No license server, no IT ticket.