bagOfNgrams — Create bag-of-n-grams model objects.

bagOfNgrams(documents) creates a lightweight RunMat bagOfNgrams object from a tokenizedDocument compatibility object. bagOfNgrams(uniqueNgrams, counts) creates a model from explicit n-grams and count matrix data.

Syntax

bag = bagOfNgrams
bag = bagOfNgrams(documents)
bag = bagOfNgrams(___, 'NgramLengths', lengths)
bag = bagOfNgrams(uniqueNgrams, counts)
bag = bagOfNgrams(uniqueNgrams, counts, 'NgramLengths', lengths)

Inputs

Returns

Errors

How bagOfNgrams works

• bagOfNgrams with no input creates an empty model using default bigram length metadata.
• bagOfNgrams(documents) accepts RunMat tokenizedDocument objects created by tokenizedDocument.
• bagOfNgrams(words) accepts a word vector as one pre-tokenized document.
• bagOfNgrams(___, 'NgramLengths', lengths) accepts a positive integer scalar or vector of n-gram lengths. Duplicate length values are collapsed in first-seen order. For explicit uniqueNgrams, counts input, this option keeps only rows whose nonempty word count matches the requested lengths.
• bagOfNgrams(uniqueNgrams, counts) accepts a string matrix or text cell matrix whose rows are unique n-grams and a numeric count matrix whose columns match the n-gram rows.
• When NgramLengths is not supplied for explicit uniqueNgrams, counts input, RunMat infers the property from the nonmissing n-gram rows in first-seen order.
• Duplicate nonmissing n-grams raise RunMat:bagOfNgrams:InvalidInput. Rows containing missing strings are ignored along with their corresponding count columns.
• The returned object stores Ngrams, Counts, NgramLengths, Vocabulary, NumNgrams, and NumDocuments properties.
• N-gram rows are stored in Ngrams; shorter n-grams are padded with empty strings when mixed lengths are present.
• Counts are stored as a dense host tensor with one row per document and one column per n-gram. Very large dense count matrices are rejected with a controlled error until native sparse bag storage is implemented.
• Native sparse bag storage, object-array parity, and the full Text Analytics model method surface remain broader Text Analytics work.

GPU memory and residency

bagOfNgrams stores host text-model metadata and dense host counts in this slice.

Examples

Create A Bigram Bag

documents = tokenizedDocument(["a b a"; "a b c"]);
bag = bagOfNgrams(documents)

Expected output:

`bag.Ngrams` contains the bigrams `["a" "b"]`, `["b" "a"]`, and `["b" "c"]`.

Use Multiple Ngram Lengths

documents = tokenizedDocument("a b c", "TokenizeMethod", "unicode");
bag = bagOfNgrams(documents, "NgramLengths", [1 3])

Expected output:

`bag.NgramLengths` is `[1 3]`, and the model contains unigrams plus the three-word n-gram.

Create A Bag From Counts

uniqueNgrams = ["a" "b"; "b" "c"];
bag = bagOfNgrams(uniqueNgrams, [2 1; 0 3])

Expected output:

`bag.Counts` is the supplied `2 x 2` count matrix.

Using bagOfNgrams with coding agents

Open a RunMat example with live inputs, then ask the agent to explain how bagOfNgrams changes the result.

FAQ

Related Strings functions

Open-source implementation

Unlike proprietary runtimes, every RunMat function is open-source. Read exactly how bagOfNgrams is executed, line by line, in Rust.

• View the source for bagOfNgrams in Rust on GitHub
• Learn how the RunMat runtime works
• Found a bug? Open an issue with a minimal reproduction.

About RunMat

RunMat is an open-source runtime that executes MATLAB-syntax code blazing on any GPU. It is licensed under the Apache 2.0 license.

• RunMat automatically optimizes your math for GPU execution on Apple, Nvidia, and AMD hardware. No code changes needed. Simulations that took hours now take minutes.
• Start running code in seconds. RunMat runs in the browser, on the desktop, or from the CLI. No license server, no IT ticket.

Getting started · Benchmarks · Pricing