unicode2native — Convert Unicode text to encoded uint8 bytes in MATLAB and RunMat.
unicode2native(unicodestr) converts a string scalar or character vector into a 1xn uint8 row vector using UTF-8 by default. unicode2native(unicodestr, encoding) uses a named character encoding.
Syntax
bytes = unicode2native(unicodestr)
bytes = unicode2native(unicodestr, encoding)Inputs
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
unicodestr | StringScalar | Yes | — | Unicode text, provided as a string scalar or character vector. |
encoding | StringScalar | No | "UTF-8" | Character encoding name. |
Returns
| Name | Type | Description |
|---|---|---|
bytes | NumericArray | uint8 row vector containing encoded bytes. |
Errors
| Identifier | When | Message |
|---|---|---|
RunMat:unicode2native:InvalidInput | Input text is not a string scalar or character vector. | unicode2native: input must be a string scalar or character vector |
RunMat:unicode2native:InvalidEncoding | Encoding name is unsupported or malformed. | unicode2native: unsupported character encoding |
RunMat:unicode2native:InvalidArgument | Too many arguments were supplied. | unicode2native: invalid argument |
RunMat:unicode2native:InternalError | Internal byte-vector tensor construction failed. | unicode2native: internal error |
How unicode2native works
- String scalars and character row vectors are accepted as input text.
- The optional encoding argument can be a string scalar or character row vector.
- The default encoding is UTF-8.
- Encoding names are resolved through the runtime encoding library plus explicit Unicode encoders, including common aliases such as
UTF-8,latin1,US-ASCII,windows-1252,Shift_JIS,UTF-16, andUTF-32. UTF-16emits a little-endian byte-order mark followed by UTF-16LE bytes, matching MATLAB's documented output form.UTF-16LEandUTF-16BEproduce UTF-16 bytes without a byte-order mark.UTF-32emits a little-endian byte-order mark followed by UTF-32LE bytes.UTF-32LEandUTF-32BEproduce UTF-32 bytes without a byte-order mark.- Characters that cannot be represented in a legacy byte encoding are substituted with
?(63) instead of HTML numeric character references. - The result is always a host
uint8tensor with shape1xn, including1x0for empty input. - Non-scalar string arrays, multi-row character arrays, unsupported encodings, non-text inputs, and extra arguments raise descriptive errors.
Examples
Convert Text With The Default Encoding
bytes = unicode2native("hello")Expected output:
bytes =
1x5 uint8 row vector
104 101 108 108 111Encode Text As UTF-8
bytes = unicode2native("hé", "UTF-8")Expected output:
bytes =
1x3 uint8 row vector
104 195 169Encode Text As Latin-1
bytes = unicode2native('café', 'latin1')Expected output:
bytes =
1x4 uint8 row vector
99 97 102 233Encode Text As UTF-16
bytes = unicode2native("hi", "UTF-16")Expected output:
bytes =
1x6 uint8 row vector
255 254 104 0 105 0Encode Text As UTF-32
bytes = unicode2native("A", "UTF-32")Expected output:
bytes =
1x8 uint8 row vector
255 254 0 0 65 0 0 0Using unicode2native with coding agents
Open a RunMat example with live inputs, then ask the agent to explain how unicode2native changes the result.
Run a small unicode2native example, explain the result, then change one input and compare the output.
FAQ
What does unicode2native return?⌄
It returns a uint8 row vector containing the encoded byte values.
What is the default encoding?⌄
RunMat uses UTF-8 as the default native encoding.
Can I pass a string array?⌄
Only scalar string arrays are accepted. MATLAB documents unicodestr as a string scalar or character vector, so nonscalar string arrays raise an error.
What happens when a character cannot be represented?⌄
RunMat substitutes ? (63) for characters that cannot be represented in byte-oriented legacy encodings. This avoids HTML numeric character-reference bytes and matches MATLAB-style byte substitution.
Does this run on the GPU?⌄
No. This is a text conversion sink: inputs are gathered to host memory and encoded on the CPU.
Related Strings functions
Core
blanks · char · compose · convertCharsToStrings · convertContainedStringsToChars · convertStringsToChars · genvarname · int2str · isletter · isspace · isStringScalar · isstrprop · mat2str · native2unicode · newline · num2str · sprintf · sscanf · str2double · str2num · strcmp · strcmpi · string · string.empty · strings · strlength · strncmp · strncmpi · strtok
Text Analytics
addDependencyDetails · addEntityDetails · addLemmaDetails · addPartOfSpeechDetails · addSentenceDetails · addTypeDetails · bagOfNgrams · bagOfWords · cosineSimilarity · doc2sequence · encode · extractFileText · extractHTMLText · fastTextWordEmbedding · findElement · getAttribute · htmlTree · ind2word · isVocabularyWord · normalizeWords · readWordEmbedding · removeLongWords · removeShortWords · removeStopWords · removeWords · stopWords · tokenDetails · tokenizedDocument · trainWordEmbedding · vaderSentimentScores · vec2word · word2ind · word2vec · wordEncoding · writeWordEmbedding
Transform
append · deblank · erase · eraseBetween · erasePunctuation · eraseURLs · extractAfter · extractBefore · extractBetween · insertAfter · insertBefore · join · lower · pad · replace · replaceBetween · reverse · split · splitlines · strcat · strip · strjoin · strjust · strrep · strsplit · strtrim · upper
Search
contains · endsWith · matches · startsWith · strfind
Pattern
digitsPattern · lettersPattern · pattern · regexpPattern · textBoundary · wildcardPattern
Open-source implementation
Unlike proprietary runtimes, every RunMat function is open-source. Read exactly how unicode2native is executed, line by line, in Rust.
- View the source for unicode2native in Rust on GitHub
- Learn how the RunMat runtime works
- Found a bug? Open an issue with a minimal reproduction.
About RunMat
RunMat is an open-source runtime that executes MATLAB-syntax code blazing on any GPU. It is licensed under the Apache 2.0 license.
- RunMat automatically optimizes your math for GPU execution on Apple, Nvidia, and AMD hardware. No code changes needed. Simulations that took hours now take minutes.
- Start running code in seconds. RunMat runs in the browser, on the desktop, or from the CLI. No license server, no IT ticket.