removeLongWords — Remove long words from tokenized documents or bag-of-words models.
removeLongWords(documents, len) removes tokens whose character length is greater than or equal to len from RunMat tokenizedDocument compatibility objects. removeLongWords(bag, len) removes matching vocabulary columns from RunMat bagOfWords objects.
Syntax
newDocumentsOrBag = removeLongWords(documentsOrBag, len)Inputs
documentsOrBag- Type
Any- Required
- Yes
- Description
- tokenizedDocument or bagOfWords object.
len- Type
NumericScalar- Required
- Yes
- Description
- Minimum word length to remove.
Returns
newDocumentsOrBag- Type
Any- Description
- Filtered tokenizedDocument or bagOfWords object.
Errors
RunMat:removeLongWords:InvalidInput- When
- Inputs do not match a supported removeLongWords form.
- Message
removeLongWords: invalid input
RunMat:removeLongWords:TooManyOutputs- When
- The requested output count exceeds the canonical signatures.
- Message
removeLongWords: too many output arguments
Supported forms
newDocuments = removeLongWords(documents, len)
newBag = removeLongWords(bag, len)
newDocuments is a tokenizedDocument compatibility object. newBag is a bagOfWords compatibility object.
len must be positive and integer-valued.
Input objects must be RunMat Text Analytics compatibility objects from this module.
Behavior
documents must be a RunMat tokenizedDocument object created by tokenizedDocument.
bag must be a RunMat bagOfWords object created by bagOfWords.
len must be a positive integer scalar and accepts every integer class without a floating conversion.
For tokenized documents, the result preserves Shape, TokenizeMethod, and Language metadata while recomputing document lengths and vocabulary.
For bag-of-words models, the result preserves document row count and removes count columns for words with length greater than or equal to len.
Execution and residency
removeLongWords filters host text-model objects. Automatic residency for a numeric control gathers transparently; explicit gpuArray host fallback is a declared RunMat extension.
removeLongWords filters host text/model objects and has no provider kernel.
Implementation boundary
The host path counts Unicode characters in each token and filters model vocabulary/count columns. It does not introduce sparse bag storage or a provider text-filter kernel.
Finite host allocation
The current implementation requires an Unbounded host allocation policy. Under Managed finite allocation, a shared family guard rejects before gather, model cloning, mutation or filesystem work because complete text-model scratch and metadata ownership is not yet implemented. Authority is checked again after suspension and before publication, and failures are terminal rather than permission to replay. This refusal is an implementation frontier, not completion of finite resource support.
Examples
Filter Tokenized Documents
documents = tokenizedDocument("An example of a short sentence");
newDocuments = removeLongWords(documents, 7)Expected output:
`newDocuments` contains `An of a short`.Filter A Bag Of Words
documents = tokenizedDocument(["an example of a short sentence"; "a second short sentence"]);
bag = bagOfWords(documents);
newBag = removeLongWords(bag, 7)Expected output:
`newBag.Vocabulary` excludes `example` and `sentence`.Using removeLongWords with coding agents
Open a RunMat example with live inputs, then ask the agent to explain how removeLongWords changes the result.
Run a small removeLongWords example, explain the result, then change one input and compare the output.
FAQ
Does removeLongWords remove words longer than len or longer than or equal to len?⌄
It removes words with character length greater than or equal to len, matching the MATLAB Text Analytics contract.
Does removeLongWords support raw strings?⌄
No. Convert raw text with tokenizedDocument first, or build a bagOfWords model and filter that object.
Does removeLongWords execute on the GPU?⌄
No. It filters host text/model metadata and has no runmat-accelerate provider path.
Related Strings functions
Text Analytics
addDependencyDetails · addEntityDetails · addLemmaDetails · addPartOfSpeechDetails · addSentenceDetails · addTypeDetails · bagOfNgrams · bagOfWords · cosineSimilarity · doc2sequence · encode · extractFileText · extractHTMLText · fastTextWordEmbedding · findElement · getAttribute · htmlTree · ind2word · isVocabularyWord · normalizeWords · readWordEmbedding · removeShortWords · removeStopWords · removeWords · stopWords · tokenDetails · tokenizedDocument · trainWordEmbedding · vaderSentimentScores · vec2word · word2ind · word2vec · wordEncoding · writeWordEmbedding
Transform
append · deblank · erase · eraseBetween · erasePunctuation · eraseURLs · extractAfter · extractBefore · extractBetween · insertAfter · insertBefore · join · lower · pad · replace · replaceBetween · reverse · split · splitlines · strcat · strip · strjoin · strjust · strrep · strsplit · strtrim · upper
Core
blanks · char · compose · convertCharsToStrings · convertContainedStringsToChars · convertStringsToChars · genvarname · int2str · isletter · isspace · isStringScalar · isstrprop · mat2str · native2unicode · newline · num2str · sprintf · sscanf · str2double · str2num · strcmp · strcmpi · string · string.empty · strings · strlength · strncmp · strncmpi · strtok · unicode2native
Search
contains · endsWith · matches · startsWith · strfind
Pattern
digitsPattern · lettersPattern · pattern · regexpPattern · textBoundary · wildcardPattern
Source you can read
Every RunMat function's source is public. Read exactly how removeLongWords is executed, line by line, in Rust.
- View the source for removeLongWords in Rust on GitHub
- Learn how the RunMat runtime works
- Found a bug? Open an issue with a minimal reproduction.
About RunMat
RunMat is a runtime that executes MATLAB-syntax code fast on any GPU. The runtime is free for individuals, including freelancers; students and faculty at an academic institution, for teaching or research there; contributors to an open-source project, for work on that project; organizations with 250 or fewer people and US$10M or less in annual revenue; and anyone, for non-production use. See runmat.com/license.
- RunMat automatically optimizes your math for GPU execution on Apple, Nvidia, and AMD hardware. No code changes needed. Simulations that took hours now take minutes.
- Start running code in seconds. RunMat runs in the browser, on the desktop, or from the CLI. No license server, no IT ticket.