类 TIEPass

java.lang.Object
com.tencent.mps.tie.api.TIEPass

public class TIEPass extends Object
TIEPass is a class responsible for performing image enhancement rendering on input images. The class creates a TIEPass object and provides methods for initialization, rendering, reinitialization, and releasing resources.

Note:

  • All methods of TIEPass must be called within the same GLThread. Always ensure to call all methods of TIEPass within the same GLThread to guarantee correct execution and avoid potential issues.
  • The render(int) method should not be called before the init(int, int) method, and it should not be called after the deInit() method.
  • The reInit(int, int) method can be used to update the dimensions of the input image without needing to create a new TIEPass instance.

Usage:

  1. (In glThread) Create a TIEPass object using the constructor.
  2. (In glThread) Initialize the TIEPass object by calling the init(int, int) method. This method sets the input image width and height.
  3. (In glThread) If the input image dimensions change and you need to update the TIEPass configuration without creating a new instance, call the reInit(int, int) method.
  4. (In glThread) Perform image enhancement rendering on an input OpenGL texture by calling the render(int) method. This method returns the enhanced texture ID.
  5. (In glThread) Release resources when the TIEPass object is no longer needed by calling the deInit() method. This ensures proper release of OpenGL-related resources.

Example:


 // The code below must be executed in the same glThread.
 //----------------------GL Thread---------------------//

 // Create a TIEPass object using the constructor.
 TIEPass tiePass = new TIEPass(TIEPass.TIEAlgorithmType.PROFESSIONAL_HIGH_QUALITY);

 // Initialize TIEPass and set the input image width and height.
 tiePass.init(inputWidth, inputHeight);

 // If the input image dimensions change, reinitialize TIEPass with new dimensions.
 tiePass.reInit(newInputWidth, newInputHeight);

 // Perform image enhancement rendering on the input OpenGL texture and get the enhanced texture ID.
 int outputTextureId = tiePass.render(inputTextureId);

 // Release resources when the TIEPass object is no longer needed.
 tiePass.deInit();
 //----------------------GL Thread---------------------//

 
  • 嵌套类概要

    嵌套类
    修饰符和类型
    说明
    static interface 
    A listener interface for handling fallback events in the image enhancement process.
    static enum 
    The TIEAlgorithmType enum represents the algorithm running mode to be used for image enhancement.
    static enum 
    Enum representing various initialization errors for TIEPass.
  • 构造器概要

    构造器
    构造器
    说明
    Constructs a TIEPass object for image enhancement.
  • 方法概要

    修饰符和类型
    方法
    说明
    int
    benchmarkProIE(int inputWidth, int inputHeight)
    Evaluates the rendering time consumption of the PROFESSIONAL algorithm.
    configureProIEMaxInputResolution(int maxInputWidth, int maxInputHeight)
    Configures the maximum input resolution for the image enhancement process.
    void
    Releases the resources associated with the TIEPass object.
    void
    Disables the automatic fallback behavior for the image enhancement process.
    void
    enableProIEAutoFallback(int consecutiveTimeoutFrames, int timeoutDurationMs, TIEPass.FallbackListener listener)
    Configures and activates the automatic fallback mechanism for image enhancement processing.
    void
    forceProIEFallback(boolean enable)
    Switches between the PROFESSIONAL and STANDARD algorithms.
    init(int inputWidth, int inputHeight)
    Initializes the TIEPass object for image enhancement.
    reInit(int inputWidth, int inputHeight)
    Reinitializes the TIEPass object for image enhancement.
    int
    render(int textureId)
    Performs the image enhancement rendering operation on the input image.
    int
    render(int textureId, float[] transformMatrix)
    Performs the image enhancement rendering operation on the input image with a specified transform matrix.

    从类继承的方法 java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
  • 构造器详细资料

    • TIEPass

      public TIEPass(TIEPass.TIEAlgorithmType algorithmType)
      Constructs a TIEPass object for image enhancement. This constructor should be called after TSRSdk.init(long, int, TSRSdk.TSRSdkLicenseVerifyResultCallback, TSRLogger) to ensure proper initialization. The created TIEPass object provides image enhancement functionality based on the specified TIEPass.TIEAlgorithmType.
      参数:
      algorithmType - The TIEAlgorithmType enum value representing the algorithm running mode to be used for image enhancement. It can be either PROFESSIONAL_FAST or PROFESSIONAL_HIGH_QUALITY.
      • PROFESSIONAL_FAST mode prioritizes better performance in terms of speed, possibly sacrificing some image quality.
      • PROFESSIONAL_HIGH_QUALITY mode prioritizes better image quality, even if the running speed is slower.
  • 方法详细资料

    • configureProIEMaxInputResolution

      public TIEPass.TIEInitStatusCode configureProIEMaxInputResolution(int maxInputWidth, int maxInputHeight)
      Configures the maximum input resolution for the image enhancement process. This method is used to pre-allocate memory for the underlying neural network inference framework, helping to avoid performance overhead when resetting the resolution or issues with image enhancement not taking effect during reinitialization.

      This method can be called from a non-GL thread and must be invoked before the init(int, int) and reInit(int, int) methods; otherwise, those methods will be ineffective.

      参数:
      maxInputWidth - The maximum width of the input image. This value should be between [8, 4096].
      maxInputHeight - The maximum height of the input image. This value should be between [8, 4096].
      返回:
      A TIEPass.TIEInitStatusCode enum value indicating the result of the configuration.
    • init

      public TIEPass.TIEInitStatusCode init(int inputWidth, int inputHeight)
      Initializes the TIEPass object for image enhancement. This method must be called from the same GL thread and after configureProIEMaxInputResolution(int, int) has been successfully executed. The input parameters specify the dimensions of the input image to be enhanced.
      参数:
      inputWidth - The new width of the input image to be enhanced. This value should be between [8, min(maxInputWidth, 4096)].
      inputHeight - The new height of the input image to be enhanced. This value should be between [8, min(maxInputHeight, 4096)].
      返回:
      A TIEPass.TIEInitStatusCode enum value indicating the result of the initialization.
    • reInit

      public TIEPass.TIEInitStatusCode reInit(int inputWidth, int inputHeight)
      Reinitializes the TIEPass object for image enhancement. This method must be called from the same GL thread and after configureProIEMaxInputResolution(int, int) has been successfully executed. The input parameters specify the new dimensions of the input image to be enhanced.
      参数:
      inputWidth - The new width of the input image to be enhanced. This value should be between [8, min(maxInputWidth, 4096)].
      inputHeight - The new height of the input image to be enhanced. This value should be between [8, min(maxInputHeight, 4096)].
      返回:
      A TIEPass.TIEInitStatusCode enum value indicating the result of the reinitialization.
    • enableProIEAutoFallback

      public void enableProIEAutoFallback(int consecutiveTimeoutFrames, int timeoutDurationMs, TIEPass.FallbackListener listener)
      Configures and activates the automatic fallback mechanism for image enhancement processing.

      This method will determine a timeout when the processing time exceeds the specified timeoutDurationMs. If the number of consecutive timeout frames reaches the configured threshold consecutiveTimeoutFrames, an automatic fallback operation will be triggered.

      The threshold for consecutive timeout frames, consecutiveTimeoutFrames, is dynamically adjusted based on the device's performance. In cases of lower device performance, the threshold will be reduced. The adjustment mechanism is as follows:

      • If the processing time is twice timeoutDurationMs, the threshold will be halved.
      • If the processing time is three times timeoutDurationMs, the threshold will be reduced to one-third, and so on.

      The minimum value for the threshold is 2. The formula for updating the threshold is: consecutiveTimeoutFrames = max(2, consecutiveTimeoutFrames / (timeConsume / timeoutDurationMs)), where timeConsume is the actual processing time.

      Through this dynamic adjustment mechanism, the system can flexibly respond to the current performance conditions, ensuring stability and efficiency during the processing.

      参数:
      consecutiveTimeoutFrames - The initial threshold for the number of consecutive timeout frames before triggering a fallback. This value may be dynamically adjusted based on performance.
      timeoutDurationMs - The duration in milliseconds that defines the timeout threshold for processing. If processing exceeds this duration, it is considered a timeout.
      listener - The listener that will be notified when a fallback event occurs. The listener's onFallback method will be called with the relevant parameters.
    • disableProIEAutoFallback

      public void disableProIEAutoFallback()
      Disables the automatic fallback behavior for the image enhancement process.

      This method should be called to turn off the automatic fallback feature that was previously enabled using enableProIEAutoFallback(int, int, FallbackListener). Once this method is invoked, the system will no longer trigger a fallback based on the configured parameters.

    • benchmarkProIE

      public int benchmarkProIE(int inputWidth, int inputHeight)
      Evaluates the rendering time consumption of the PROFESSIONAL algorithm.

      This method assesses the execution time in milliseconds for the PROFESSIONAL algorithm based on the given input dimensions. This method should not be called on the main thread, as it may take approximately 2 to 5 seconds to complete.

      This method only takes effect if the TIEPass.TIEAlgorithmType used to create the TIEPass is not set to TIEPass.TIEAlgorithmType.STANDARD. If the algorithm type is STANDARD, the evaluation may not be applicable.

      If the execution of the algorithm fails for any reason, this method will return -1.

      参数:
      inputWidth - The width of the input.
      inputHeight - The height of the input.
      返回:
      The estimated rendering time consumption in milliseconds, or -1 if the execution fails.
    • forceProIEFallback

      public void forceProIEFallback(boolean enable)
      Switches between the PROFESSIONAL and STANDARD algorithms.

      This method enables or disables the use of the STANDARD algorithm. When enable is true, the system will switch to the STANDARD algorithm; otherwise, it will use the PROFESSIONAL algorithm.

      This method only takes effect if the TIEPass.TIEAlgorithmType used to create the TIEPass is not set to TIEPass.TIEAlgorithmType.STANDARD.

      参数:
      enable - true to switch to the STANDARD algorithm; false to use the PROFESSIONAL algorithm.
    • render

      public int render(int textureId)
      Performs the image enhancement rendering operation on the input image. This method must be called within the GLThread.

      This method applies the image enhancement rendering process to the input image, improving its quality. The processed image is rendered onto a texture within the TIEPass object, and the texture's ID is returned.

      NOTE:

      • The type of the texture corresponding to the textureId you pass in must be texture2D.
      • The format of the input OpenGL texture must be GL_RGBA.
      • The format of the output textureId is also GL_RGBA.

      参数:
      textureId - The OpenGL textureId of the input image that needs to be processed for image enhancement. The corresponding texture is a texture2D texture with GL_RGBA format.
      返回:
      The textureId of the texture2D after image enhancement rendering, which is stored inside the TIEPass object. The format of the output texture is GL_RGBA. The size of the output texture is (inputWidth, inputHeight).
    • render

      public int render(int textureId, float[] transformMatrix)
      Performs the image enhancement rendering operation on the input image with a specified transform matrix. This method must be called within the GLThread.

      This method applies the image enhancement rendering process to the input image, improving its quality, and uses the given transform matrix to manipulate the image during the rendering process. The processed image is rendered onto a texture within the TIEPass object, and the texture's ID is returned.

      NOTE:

      • The type of the texture corresponding to the textureId you pass in must be texture2D.
      • The format of the input OpenGL texture must be GL_RGBA.
      • The format of the output textureId is also GL_RGBA.

      参数:
      textureId - The OpenGL textureId of the input image that needs to be processed for image enhancement. The corresponding texture is a texture2D texture with GL_RGBA format.
      transformMatrix - A float array representing the transform matrix to be applied to the input image during the rendering process. The matrix should be in column-major order and of size 16. The transform matrix can be used to apply various transformations, such as rotation, scaling, and translation, to the input image during the rendering process.
      返回:
      The textureId of the texture2D after image enhancement rendering and applying the transform matrix, which is stored inside the TIEPass object. The format of the output texture is GL_RGBA. The size of the output texture is (inputWidth, inputHeight).
    • deInit

      public void deInit()
      Releases the resources associated with the TIEPass object.

      This method should be called when the TIEPass object is no longer needed, to free up memory and other resources. Failing to call this method may lead to memory leaks or other issues.

      After calling this method, the TIEPass object should not be used for any further operations.