addSentenceDetails — Add sentence numbers to tokenized documents.

addSentenceDetails(documents) detects sentence boundaries in a RunMat tokenizedDocument compatibility object and stores one SentenceNumber value per token. Use tokenDetails to view the resulting table column.

Syntax

newDocuments = addSentenceDetails(documents)
newDocuments = addSentenceDetails(documents,Name,Value)

Inputs

Returns

Errors

How addSentenceDetails works

• documents must be a RunMat tokenizedDocument object.
• RunMat assigns sentence numbers from terminating punctuation tokens ., !, ?, and common Japanese/Korean equivalents.
• Abbreviations accepts a character vector, string array, cell array of character vectors, or a table with Abbreviation and Usage variables.
• Text abbreviation lists are treated as regular abbreviations. Table Usage values can be text or categorical values of regular, inner, reference, or unit.
• Starters accepts a character vector, string array, cell array of character vectors, or categorical vector and controls whether regular abbreviations break before capitalized starter words. The default starter list is based on RunMat's English stopWords list plus common sentence starters.
• DiscardKnownValues defaults to false; existing complete positive integer sentence-number details are preserved. Set it to true to recompute every sentence number.
• Single-letter initialisms such as U.S.A. are treated as regular abbreviations and are not split inside the initialism.
• RunMat's sentence detector is token-level and deterministic. Exact MathWorks statistical sentence boundary models, emoji-after-terminator placement, line-provenance-aware splitting, and Japanese/Korean MeCab behavior remain broader Text Analytics work.

GPU memory and residency

addSentenceDetails updates host tokenizedDocument metadata and has no provider kernel.

Examples

Add Sentence Details

documents = tokenizedDocument(["This is an example document. It has two sentences."; "This document has one sentence."]);
documents = addSentenceDetails(documents);
tdetails = tokenDetails(documents)

Expected output:

`tdetails` includes a `SentenceNumber` column.

Use Custom Abbreviations

documents = tokenizedDocument("Book an appt. We'll meet then.");
documents = addSentenceDetails(documents, "Abbreviations", "appt", "Starters", "we'll")

Expected output:

`appt.` is treated as a regular abbreviation and breaks only before the configured starter.

Recompute Sentence Details

documents = addSentenceDetails(documents, "DiscardKnownValues", true)

Expected output:

`documents` has refreshed sentence-number details.

Using addSentenceDetails with coding agents

Open a RunMat example with live inputs, then ask the agent to explain how addSentenceDetails changes the result.

FAQ

Related Strings functions

Open-source implementation

Unlike proprietary runtimes, every RunMat function is open-source. Read exactly how addSentenceDetails is executed, line by line, in Rust.

• View the source for addSentenceDetails in Rust on GitHub
• Learn how the RunMat runtime works
• Found a bug? Open an issue with a minimal reproduction.

About RunMat

RunMat is an open-source runtime that executes MATLAB-syntax code blazing on any GPU. It is licensed under the Apache 2.0 license.

• RunMat automatically optimizes your math for GPU execution on Apple, Nvidia, and AMD hardware. No code changes needed. Simulations that took hours now take minutes.
• Start running code in seconds. RunMat runs in the browser, on the desktop, or from the CLI. No license server, no IT ticket.

Getting started · Benchmarks · Pricing