adamupdate — Update numeric parameters using the Adam optimizer rule.

adamupdate(params, grad, averageGrad, averageSqGrad, iteration) applies the Adam optimizer update to host numeric arrays, host dlarray compatibility objects, and plain resident gpuArray tensors.

Syntax

[params,avg,avgSq] = adamupdate(params, grad, avg, avgSq, iteration)
[params,avg,avgSq] = adamupdate(params, grad, avg, avgSq, iteration, learnRate, gradDecay, sqGradDecay, epsilon)

How adamupdate works

• Parameters, gradients, and non-empty moving-average inputs must have matching shapes.
• Empty averageGrad or averageSqGrad inputs initialize the corresponding moving average to zeros with the parameter shape.
• Optional positional arguments are learnRate, gradDecay, sqGradDecay, and epsilon; defaults are 0.001, 0.9, 0.999, and 1e-8; learning rate and epsilon must be positive, and decay factors must be in [0, 1).
• A single unbracketed call returns updated parameters directly; bracketed calls return up to three outputs: updated parameters, updated first-moment average, and updated second-moment average.
• Tensor inputs must be floating-point tensors; integer tensor optimizer state is rejected rather than relabeled after fractional updates.
• When params and grad are resident gpuArray tensors, matching resident optimizer-state tensors or empty host numeric state route through the acceleration provider and return resident gpuArray outputs without downloading full tensors.
• GPU-backed dlarray wrappers, model parameter structures, tables, function-handle training loops, and automatic differentiation remain future work.

GPU memory and residency

adamupdate preserves residency for plain real gpuArray parameter, gradient, and optimizer-state tensors through the runmat-accelerate adam_update provider hook. Empty moving-average state is initialized on device. Host-backed dlarray remains host-compatible, and GPU-backed dlarray metadata/autodiff support remains future work.

Example

Update Parameters

[params,avg,avgSq] = adamupdate(params, gradients, avg, avgSq, iteration)

Expected output:

Updated numeric parameters and Adam moving-average state.

Using adamupdate with coding agents

Open a RunMat example with live inputs, then ask the agent to explain how adamupdate changes the result.

Related Deep Learning functions

About RunMat

RunMat is an open-source runtime that executes MATLAB-syntax code blazing on any GPU. It is licensed under the Apache 2.0 license.

• RunMat automatically optimizes your math for GPU execution on Apple, Nvidia, and AMD hardware. No code changes needed. Simulations that took hours now take minutes.
• Start running code in seconds. RunMat runs in the browser, on the desktop, or from the CLI. No license server, no IT ticket.

Getting started · Benchmarks · Pricing