addLemmaDetails — Add lemma forms to tokenized documents.
addLemmaDetails(documents) adds lemma metadata to a RunMat tokenizedDocument compatibility object. Use tokenDetails to view the resulting Lemma column.
Syntax
updatedDocuments = addLemmaDetails(documents)
updatedDocuments = addLemmaDetails(documents,'DiscardKnownValues',tf)Inputs
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
documents | Any | Yes | — | tokenizedDocument object. |
Name | StringScalar | Yes | DiscardKnownValues | DiscardKnownValues option name. |
tf | LogicalArray | Yes | false | Whether to recompute existing lemma details. |
Returns
| Name | Type | Description |
|---|---|---|
updatedDocuments | Any | Updated tokenized document object. |
Errors
| Identifier | When | Message |
|---|---|---|
RunMat:addLemmaDetails:InvalidInput | Input is not a supported tokenizedDocument object or option form. | addLemmaDetails: invalid input |
How addLemmaDetails works
documentsmust be a RunMattokenizedDocumentobject.addLemmaDetails(documents,'DiscardKnownValues',tf)controls whether existing lemma details are preserved or recomputed.DiscardKnownValuesdefaults tofalse, so existing known lemma details are preserved while missing or empty entries are filled.DiscardKnownValues,truerecomputes every stored lemma.- For English documents, RunMat applies the same lightweight English lemmatizer used by
normalizeWords(...,'Style','lemma'), including common irregulars such asrantorun, plural reduction, and simple verb-suffix normalization. - Punctuation, numbers, URLs, email addresses, hashtags, mentions, emoji, and other non-word tokens keep their original token text as the lemma.
- If an existing tokenizedDocument object already carries Japanese or Korean language metadata, RunMat preserves token text as the lemma. Public Japanese/Korean tokenizedDocument construction still requires MeCab-compatible tokenization and remains tracked. German documents are rejected because MATLAB
addLemmaDetailsdoes not document German support.
GPU memory and residency
addLemmaDetails updates host tokenizedDocument metadata and has no provider kernel.
Examples
Add Lemma Details
documents = tokenizedDocument(["The dogs ran after the cat."; "I am building a house."]);
documents = addLemmaDetails(documents);
tdetails = tokenDetails(documents)Expected output:
`tdetails.Lemma` contains values such as `"dog"`, `"run"`, `"be"`, and `"build"`.Recompute Lemma Details
documents = addLemmaDetails(documents, "DiscardKnownValues", true)Expected output:
`documents` has refreshed lemma details.Using addLemmaDetails with coding agents
Open a RunMat example with live inputs, then ask the agent to explain how addLemmaDetails changes the result.
Run a small addLemmaDetails example, explain the result, then change one input and compare the output.
FAQ
Does addLemmaDetails use a full NLP model?⌄
No. This compatibility slice uses RunMat's deterministic English lemmatizer and records exact model parity as remaining Text Analytics work.
Does addLemmaDetails execute on the GPU?⌄
No. It updates host tokenizedDocument metadata.
Related Strings functions
Text Analytics
addDependencyDetails · addEntityDetails · addPartOfSpeechDetails · addSentenceDetails · addTypeDetails · bagOfNgrams · bagOfWords · cosineSimilarity · doc2sequence · encode · extractFileText · extractHTMLText · fastTextWordEmbedding · findElement · getAttribute · htmlTree · ind2word · isVocabularyWord · normalizeWords · readWordEmbedding · removeLongWords · removeShortWords · removeStopWords · removeWords · stopWords · tokenDetails · tokenizedDocument · trainWordEmbedding · vaderSentimentScores · vec2word · word2ind · word2vec · wordEncoding · writeWordEmbedding
Transform
append · deblank · erase · eraseBetween · erasePunctuation · eraseURLs · extractAfter · extractBefore · extractBetween · insertAfter · insertBefore · join · lower · pad · replace · replaceBetween · reverse · split · splitlines · strcat · strip · strjoin · strjust · strrep · strsplit · strtrim · upper
Core
blanks · char · compose · convertCharsToStrings · convertContainedStringsToChars · convertStringsToChars · genvarname · int2str · isletter · isspace · isStringScalar · isstrprop · mat2str · native2unicode · newline · num2str · sprintf · sscanf · str2double · str2num · strcmp · strcmpi · string · string.empty · strings · strlength · strncmp · strncmpi · strtok · unicode2native
Search
contains · endsWith · matches · startsWith · strfind
Pattern
digitsPattern · lettersPattern · pattern · regexpPattern · textBoundary · wildcardPattern
Open-source implementation
Unlike proprietary runtimes, every RunMat function is open-source. Read exactly how addLemmaDetails is executed, line by line, in Rust.
- View the source for addLemmaDetails in Rust on GitHub
- Learn how the RunMat runtime works
- Found a bug? Open an issue with a minimal reproduction.
About RunMat
RunMat is an open-source runtime that executes MATLAB-syntax code blazing on any GPU. It is licensed under the Apache 2.0 license.
- RunMat automatically optimizes your math for GPU execution on Apple, Nvidia, and AMD hardware. No code changes needed. Simulations that took hours now take minutes.
- Start running code in seconds. RunMat runs in the browser, on the desktop, or from the CLI. No license server, no IT ticket.