TIEPass Class Reference

Inherits from NSObject
Declared in TIEPass.h
TIEPass.mm

Overview

TIEPass is a class responsible for performing image enhance rendering on input images using the Metal framework.

TIEPass is responsible for initializing the image enhance rendering process with the specified input image size. It also provides a render method that applies the image enhance process to the input image and returns the processed image as an MTLTexture object.

Note:

  1. This class uses the Metal framework for performing image enhance rendering and requires a device that supports Metal.
  2. All methods of TSRPass must be called within the same thread.
  3. The render:commandBuffer: and renderWithPixelBuffer: method should not be called before the initWithDevice:inputWidth:inputHeight:algorithmType:initStatusCode: method, and it should not be called after the deInit method.

Usage:

  1. Create a TIEPass instance using the initWithDevice:inputWidth:inputHeight:algorithmType:initStatusCode: method.
  2. Call the render:commandBuffer: method with the input image’s MTLTexture and an MTLCommandBuffer to perform image enhance rendering.
  3. Obtain the processed image as an MTLTexture from the return value of the render:commandBuffer: method.
  4. deInit method should be called when the TIEPass object is no longer needed, to free up the memory and other resources.

– initWithTIEAlgorithmType:device:inputWidth:inputHeight:initStatusCode:

Init TIEPass object for image enhance.

- (instancetype)initWithTIEAlgorithmType:(TIEAlgorithmType)algorithmType device:(id<MTLDevice>)device inputWidth:(int32_t)inputWidth inputHeight:(int32_t)inputHeight initStatusCode:(TIEInitStatusCode *)initStatusCode

Parameters

algorithmType

The TIEAlgorithmType enum value representing the algorithm running mode to be used.
device

The MTLDevice which you are using.
inputWidth

The width of the input image. This value should be between 8 to 4096.
inputHeight

The height of the input image. This value should be between 8 to 4096.
initStatusCode

A pointer to a TIEInitStatusCode enum value that will store the result of the initialization process. If initialization is successful, the value will be TIEInitStatusCodeSuccess. If initialization fails, the value will indicate the specific error that occurred.

Discussion

Init TIEPass object for image enhance.

Declared In

TIEPass.h

– reInit:inputHeight:

Reinitializes the TIEPass object for image enhancement. The input parameters specify the new dimensions of the input image to be enhanced.

- (TIEInitStatusCode)reInit:(int32_t)inputWidth inputHeight:(int32_t)inputHeight

Parameters

inputWidth

The new width of the input image to be enhanced. This value should be between [8, 4096].
inputHeight

The new height of the input image to be enhanced. This value should be between [8, 4096].

Return Value

TIEInitStatusCode enum value that will store the result of the reinitialization process. If reinitialization is successful, the value will be TIEInitStatusCodeSuccess. If reinitialization fails, the value will indicate the specific error that occurred.

Discussion

Reinitializes the TIEPass object for image enhancement. The input parameters specify the new dimensions of the input image to be enhanced.

Declared In

TIEPass.h

– render:commandBuffer:

Performs the image enhance rendering operation on the input image.
This method applies the image enhance rendering process to the input image, improving its quality. The processed image is rendered onto a MTLTexture within the TIEPass object. And the return is the MTLTexture which has preformed image enhance rendering .

- (id<MTLTexture>)render:(id<MTLTexture>)texture commandBuffer:(id<MTLCommandBuffer>)commandBuffer

Parameters

texture

The MTLTexture of the input image that needs to be processed for image enhance. The format of the texture is BGRA.
commandBuffer

The MTLCommandBuffer which you are using.

Return Value

The MTLTexture that will be performed image enhance rendering, which is stored inside the TIEPass object.

Discussion

Performs the image enhance rendering operation on the input image.
This method applies the image enhance rendering process to the input image, improving its quality. The processed image is rendered onto a MTLTexture within the TIEPass object. And the return is the MTLTexture which has preformed image enhance rendering .

Declared In

TIEPass.h

– renderWithPixelBuffer:

Performs the image enhance rendering operation on the input pixel buffer.

- (CVPixelBufferRef)renderWithPixelBuffer:(CVPixelBufferRef)pixelBuffer

Parameters

pixelBuffer

The CVPixelBufferRef of the input image that needs to be processed for image enhance. The width and height of the pixel buffer must match the inputWidth and inputHeight set during initialization, and the pixel format must be BGRA.

Return Value

The CVPixelBufferRef that has been performed image enhance rendering. The size of the output pixel buffer is (inputWidth, inputHeight).

Discussion

Performs the image enhance rendering operation on the input pixel buffer.

This method applies the image enhance rendering process to the input pixel buffer, improving its quality. The processed pixel buffer is created and returned as a new CVPixelBufferRef.

Note: The SDK internally handles the lifecycle of the returned CVPixelBufferRef, therefore, the caller absolutely should not attempt to manually release the returned CVPixelBufferRef.

Declared In

TIEPass.h

– enableProIEAutoFallback:timeoutDurationMs:fallbackListener:

Configures and activates the automatic fallback mechanism for image enhancement processing.

- (void)enableProIEAutoFallback:(int)consecutiveTimeoutFrames timeoutDurationMs:(int)timeoutDurationMs fallbackListener:(FallbackListener)fallbackListener

Parameters

consecutiveTimeoutFrames

The initial threshold for the number of consecutive timeout frames before triggering a fallback. This value may be dynamically adjusted based on performance.
timeoutDurationMs

The duration in milliseconds that defines the timeout threshold for processing. If processing exceeds this duration, it is considered a timeout.
fallbackListener

The listener that will be notified when a fallback event occurs. The listener’s onFallback method will be called with the relevant parameters.

Discussion

Configures and activates the automatic fallback mechanism for image enhancement processing.

This method will determine a timeout when the processing time exceeds the specified timeoutDurationMs. If the number of consecutive timeout frames reaches the configured threshold consecutiveTimeoutFrames, an automatic fallback operation will be triggered.

The threshold for consecutive timeout frames, consecutiveTimeoutFrames, is dynamically adjusted based on the device’s performance. In cases of lower device performance, the threshold will be reduced. The adjustment mechanism is as follows:

  • If the processing time is twice timeoutDurationMs, the threshold will be halved.
  • If the processing time is three times timeoutDurationMs, the threshold will be reduced to one-third, and so on.

The minimum value for the threshold is 2. The formula for updating the threshold is: consecutiveTimeoutFrames = max(2, consecutiveTimeoutFrames / (timeConsume / timeoutDurationMs)), where timeConsume is the actual processing time.

Through this dynamic adjustment mechanism, the system can flexibly respond to the current performance conditions, ensuring stability and efficiency during the processing.

Declared In

TIEPass.h

– disableProIEAutoFallback

Disables the automatic fallback behavior for the image enhancement process. This method should be called to turn off the automatic fallback feature that was previously enabled using enableProIEAutoFallback:timeoutDurationMs:fallbackListener:. Once this method is invoked, the system will no longer trigger a fallback based on the configured parameters.

- (void)disableProIEAutoFallback

Discussion

Disables the automatic fallback behavior for the image enhancement process. This method should be called to turn off the automatic fallback feature that was previously enabled using enableProIEAutoFallback:timeoutDurationMs:fallbackListener:. Once this method is invoked, the system will no longer trigger a fallback based on the configured parameters.

Declared In

TIEPass.h

– benchmarkProIE:inputHeight:

Evaluates the rendering time consumption of the PROFESSIONAL algorithm. This method assesses the execution time in milliseconds for the PROFESSIONAL algorithm based on the given input dimensions. This method should not be called on the main thread, as it may take approximately 2 to 5 seconds to complete. This method only takes effect if the TIEAlgorithmType used to create the TIEPass is not set to TIEAlgorithmTypeStandard. If the algorithm type is STANDARD, the evaluation may not be applicable. If the execution of the algorithm fails for any reason, this method will return -1.

- (int)benchmarkProIE:(int)inputWidth inputHeight:(int)inputHeight

Parameters

inputWidth

The width of the input.
inputHeight

The height of the input.

Return Value

The estimated rendering time consumption in milliseconds, or -1 if the execution fails.

Discussion

Evaluates the rendering time consumption of the PROFESSIONAL algorithm. This method assesses the execution time in milliseconds for the PROFESSIONAL algorithm based on the given input dimensions. This method should not be called on the main thread, as it may take approximately 2 to 5 seconds to complete. This method only takes effect if the TIEAlgorithmType used to create the TIEPass is not set to TIEAlgorithmTypeStandard. If the algorithm type is STANDARD, the evaluation may not be applicable. If the execution of the algorithm fails for any reason, this method will return -1.

Declared In

TIEPass.h

– forceProIEFallback:

Switches between the PROFESSIONAL and STANDARD algorithms. This method enables or disables the use of the STANDARD algorithm. When enable is true, the system will switch to the STANDARD algorithm; otherwise, it will use the PROFESSIONAL algorithm. This method only takes effect if the TIEAlgorithmType used to create the TIEPass is not set to TIEAlgorithmTypeStandard.

- (void)forceProIEFallback:(bool)enable

Parameters

enable

true to switch to the STANDARD algorithm; false to use the PROFESSIONAL algorithm.

Discussion

Switches between the PROFESSIONAL and STANDARD algorithms. This method enables or disables the use of the STANDARD algorithm. When enable is true, the system will switch to the STANDARD algorithm; otherwise, it will use the PROFESSIONAL algorithm. This method only takes effect if the TIEAlgorithmType used to create the TIEPass is not set to TIEAlgorithmTypeStandard.

Declared In

TIEPass.h

– releaseMetal

将CVMat转为CGImage

- (void)releaseMetal

Discussion

将CVMat转为CGImage

Declared In

TIEPass.mm

– deInit

Releases the resources.
This method should be called when the TIEPass object is no longer needed, to free up the memory and other resources.

- (void)deInit

Discussion

Releases the resources.
This method should be called when the TIEPass object is no longer needed, to free up the memory and other resources.

Declared In

TIEPass.h