bagOfNgrams — Create bag-of-n-grams model objects.
bagOfNgrams(documents) creates a lightweight RunMat bagOfNgrams object from a tokenizedDocument compatibility object. bagOfNgrams(uniqueNgrams, counts) creates a model from explicit n-grams and count matrix data.
Syntax
bag = bagOfNgrams
bag = bagOfNgrams(documents)
bag = bagOfNgrams(___, 'NgramLengths', lengths)
bag = bagOfNgrams(uniqueNgrams, counts)
bag = bagOfNgrams(uniqueNgrams, counts, 'NgramLengths', lengths)Inputs
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
documents | Any | Yes | — | Tokenized documents or a single-document word vector. |
NameValue | Any | Variadic | — | Name-value option: NgramLengths. |
uniqueNgrams | Any | Yes | — | Unique n-gram string matrix. |
counts | Any | Yes | — | N-gram counts per document. |
Returns
| Name | Type | Description |
|---|---|---|
bag | Any | Bag-of-n-grams model object. |
Errors
| Identifier | When | Message |
|---|---|---|
RunMat:bagOfNgrams:InvalidInput | Inputs do not match a supported bagOfNgrams form. | bagOfNgrams: invalid input |
How bagOfNgrams works
bagOfNgramswith no input creates an empty model using default bigram length metadata.bagOfNgrams(documents)accepts RunMattokenizedDocumentobjects created bytokenizedDocument.bagOfNgrams(words)accepts a word vector as one pre-tokenized document.bagOfNgrams(___, 'NgramLengths', lengths)accepts a positive integer scalar or vector of n-gram lengths. Duplicate length values are collapsed in first-seen order. For explicituniqueNgrams, countsinput, this option keeps only rows whose nonempty word count matches the requested lengths.bagOfNgrams(uniqueNgrams, counts)accepts a string matrix or text cell matrix whose rows are unique n-grams and a numeric count matrix whose columns match the n-gram rows.- Explicit
countsandNgramLengthsaccept all eight integer classes. Values remain authoritative through exact validation and filtering; counts then cross one deliberate conversion into the doubleCountsproperty, and lengths become bounded host indices before returning as double metadata. - When
NgramLengthsis not supplied for explicituniqueNgrams, countsinput, RunMat infers the property from the nonmissing n-gram rows in first-seen order. - Duplicate nonmissing n-grams raise
RunMat:bagOfNgrams:InvalidInput. Rows containing missing strings are ignored along with their corresponding count columns. - The returned object stores
Ngrams,Counts,NgramLengths,Vocabulary,NumNgrams, andNumDocumentsproperties. - N-gram rows are stored in
Ngrams; shorter n-grams are padded with empty strings when mixed lengths are present. - Counts are stored as a dense host tensor with one row per document and one column per n-gram. Very large dense count matrices are rejected with a controlled error until native sparse bag storage is implemented.
- GPU-resident counts and lengths reject before provider access because
bagOfNgramsexposes no interactive GPU-array input surface. - Native sparse bag storage, object-array parity, and the full Text Analytics model method surface remain broader Text Analytics work.
GPU memory and residency
bagOfNgrams stores host text-model metadata and dense host double counts in this slice; resident inputs reject before provider access.
Examples
Create A Bigram Bag
documents = tokenizedDocument(["a b a"; "a b c"]);
bag = bagOfNgrams(documents)Expected output:
`bag.Ngrams` contains the bigrams `["a" "b"]`, `["b" "a"]`, and `["b" "c"]`.Use Multiple Ngram Lengths
documents = tokenizedDocument("a b c", "TokenizeMethod", "unicode");
bag = bagOfNgrams(documents, "NgramLengths", [1 3])Expected output:
`bag.NgramLengths` is `[1 3]`, and the model contains unigrams plus the three-word n-gram.Create A Bag From Counts
uniqueNgrams = ["a" "b"; "b" "c"];
bag = bagOfNgrams(uniqueNgrams, [2 1; 0 3])Expected output:
`bag.Counts` is the supplied `2 x 2` count matrix.Using bagOfNgrams with coding agents
Open a RunMat example with live inputs, then ask the agent to explain how bagOfNgrams changes the result.
Run a small bagOfNgrams example, explain the result, then change one input and compare the output.
FAQ
Does bagOfNgrams use sparse matrices?⌄
Not in this slice. The compatibility object stores dense host counts and rejects very large dense count matrices. Sparse storage and provider-resident sparse paths remain broader runtime/model work.
Can bagOfNgrams consume raw document strings directly?⌄
No. Raw document arrays should first be converted with tokenizedDocument. A raw word vector is accepted as a single pre-tokenized document.
Does bagOfNgrams execute on the GPU?⌄
No. It builds host text-model metadata and has no runmat-accelerate provider path. Resident inputs reject rather than gathering.
Related Strings functions
Text Analytics
addDependencyDetails · addEntityDetails · addLemmaDetails · addPartOfSpeechDetails · addSentenceDetails · addTypeDetails · bagOfWords · cosineSimilarity · doc2sequence · encode · extractFileText · extractHTMLText · fastTextWordEmbedding · findElement · getAttribute · htmlTree · ind2word · isVocabularyWord · normalizeWords · readWordEmbedding · removeLongWords · removeShortWords · removeStopWords · removeWords · stopWords · tokenDetails · tokenizedDocument · trainWordEmbedding · vaderSentimentScores · vec2word · word2ind · word2vec · wordEncoding · writeWordEmbedding
Transform
append · deblank · erase · eraseBetween · erasePunctuation · eraseURLs · extractAfter · extractBefore · extractBetween · insertAfter · insertBefore · join · lower · pad · replace · replaceBetween · reverse · split · splitlines · strcat · strip · strjoin · strjust · strrep · strsplit · strtrim · upper
Core
blanks · char · compose · convertCharsToStrings · convertContainedStringsToChars · convertStringsToChars · genvarname · int2str · isletter · isspace · isStringScalar · isstrprop · mat2str · native2unicode · newline · num2str · sprintf · sscanf · str2double · str2num · strcmp · strcmpi · string · string.empty · strings · strlength · strncmp · strncmpi · strtok · unicode2native
Search
contains · endsWith · matches · startsWith · strfind
Pattern
digitsPattern · lettersPattern · pattern · regexpPattern · textBoundary · wildcardPattern
Open-source implementation
Unlike proprietary runtimes, every RunMat function is open-source. Read exactly how bagOfNgrams is executed, line by line, in Rust.
- View the source for bagOfNgrams in Rust on GitHub
- Learn how the RunMat runtime works
- Found a bug? Open an issue with a minimal reproduction.
About RunMat
RunMat is an open-source runtime that executes MATLAB-syntax code blazing on any GPU. It is licensed under the Apache 2.0 license.
- RunMat automatically optimizes your math for GPU execution on Apple, Nvidia, and AMD hardware. No code changes needed. Simulations that took hours now take minutes.
- Start running code in seconds. RunMat runs in the browser, on the desktop, or from the CLI. No license server, no IT ticket.