addLemmaDetails — Add lemma forms to tokenized documents.

addLemmaDetails(documents) adds lemma metadata to a RunMat tokenizedDocument compatibility object. Use tokenDetails to view the resulting Lemma column.

Syntax

updatedDocuments = addLemmaDetails(documents)
updatedDocuments = addLemmaDetails(documents,'DiscardKnownValues',tf)

Inputs

Returns

Errors

How addLemmaDetails works

• documents must be a RunMat tokenizedDocument object.
• addLemmaDetails(documents,'DiscardKnownValues',tf) controls whether existing lemma details are preserved or recomputed.
• DiscardKnownValues defaults to false, so existing known lemma details are preserved while missing or empty entries are filled.
• DiscardKnownValues, true recomputes every stored lemma.
• For English documents, RunMat applies the same lightweight English lemmatizer used by normalizeWords(...,'Style','lemma'), including common irregulars such as ran to run, plural reduction, and simple verb-suffix normalization.
• Punctuation, numbers, URLs, email addresses, hashtags, mentions, emoji, and other non-word tokens keep their original token text as the lemma.
• If an existing tokenizedDocument object already carries Japanese or Korean language metadata, RunMat preserves token text as the lemma. Public Japanese/Korean tokenizedDocument construction still requires MeCab-compatible tokenization and remains tracked. German documents are rejected because MATLAB addLemmaDetails does not document German support.

GPU memory and residency

addLemmaDetails updates host tokenizedDocument metadata and has no provider kernel.

Examples

Add Lemma Details

documents = tokenizedDocument(["The dogs ran after the cat."; "I am building a house."]);
documents = addLemmaDetails(documents);
tdetails = tokenDetails(documents)

Expected output:

`tdetails.Lemma` contains values such as `"dog"`, `"run"`, `"be"`, and `"build"`.

Recompute Lemma Details

documents = addLemmaDetails(documents, "DiscardKnownValues", true)

Expected output:

`documents` has refreshed lemma details.

Using addLemmaDetails with coding agents

Open a RunMat example with live inputs, then ask the agent to explain how addLemmaDetails changes the result.

FAQ

Related Strings functions

Open-source implementation

Unlike proprietary runtimes, every RunMat function is open-source. Read exactly how addLemmaDetails is executed, line by line, in Rust.

• View the source for addLemmaDetails in Rust on GitHub
• Learn how the RunMat runtime works
• Found a bug? Open an issue with a minimal reproduction.

About RunMat

RunMat is an open-source runtime that executes MATLAB-syntax code blazing on any GPU. It is licensed under the Apache 2.0 license.

• RunMat automatically optimizes your math for GPU execution on Apple, Nvidia, and AMD hardware. No code changes needed. Simulations that took hours now take minutes.
• Start running code in seconds. RunMat runs in the browser, on the desktop, or from the CLI. No license server, no IT ticket.

Getting started · Benchmarks · Pricing