public final class InferenceOptions

Object
InferenceOptions

Configures how a reusable InferenceSession is created.

Accelerator requests are portable preferences rather than promises. With fallback enabled, a backend may execute on CPU when the requested delegate is unavailable. With fallback disabled, opening the session fails instead of silently changing the execution target. Android’s NNAPI runtime can mix NPU and CPU operations without reporting full delegation, and the iOS Core ML delegate can schedule work across the Neural Engine, GPU, and CPU. Both mobile backends therefore reject Accelerator.NPU when fallback is disabled. iOS also rejects strict Accelerator.CORE_ML sessions because the delegate does not report whether unsupported model operations remained on LiteRT’s CPU path.

Nested types

enum InferenceOptions.Accelerator Execution targets understood by the portable inference API.

Constructors

public InferenceOptions()

Methods

`public` `InferenceOptions accelerator(InferenceOptions.Accelerator value)`	Requests an execution target for the model.
`public` `InferenceOptions threads(int value)`	Sets the CPU worker count.
`public` `InferenceOptions allowFallback(boolean value)`	Controls whether opening may fall back from an unavailable accelerator to CPU execution.
`public` `InferenceOptions.Accelerator getAccelerator()`
`public int getThreads()`
`public boolean isFallbackAllowed()`

Inherited methods

From Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait, clone

Constructor details

InferenceOptions

public InferenceOptions()

Method details

accelerator

public InferenceOptions accelerator(InferenceOptions.Accelerator value)

Requests an execution target for the model.

Parameters

value InferenceOptions.Accelerator: requested target; null restores Accelerator.AUTO

Returns

this options object

threads

public InferenceOptions threads(int value)

Sets the CPU worker count. Non-positive values let the native runtime choose its default.

Parameters

value int: requested worker count

Returns

this options object

allowFallback

public InferenceOptions allowFallback(boolean value)

Controls whether opening may fall back from an unavailable accelerator to CPU execution.

On Android and iOS, setting this to false with Accelerator.NPU rejects session creation. On iOS it also rejects Accelerator.CORE_ML. Neither LiteRT’s NNAPI delegate nor its Core ML delegate can prove that every operation ran on the requested accelerator instead of CPU or another processor.

Parameters

value boolean: true to permit CPU fallback

Returns

this options object

getAccelerator

public InferenceOptions.Accelerator getAccelerator()

Returns

the requested accelerator, never null

getThreads

public int getThreads()

Returns

the requested CPU worker count, or a non-positive runtime default

isFallbackAllowed

public boolean isFallbackAllowed()

Returns

whether an unavailable accelerator may fall back to CPU