程序包 com.tencent.mps.tie.api

类 TSRPass

java.lang.Object
com.tencent.mps.tie.api.TSRPass
public class TSRPass extends Object
TSRPass is a class responsible for performing super-resolution rendering on input images. The class creates a TSRPass object and provides methods for initialization, reinitialization, rendering, and releasing resources.

Note:

  • All methods of TSRPass must be called within the same GLThread.
  • The render(int) method should not be called before the init(int, int, float) method, and it should not be called after the deInit() method.
  • The reInit(int, int, float) method can be used to update the dimensions and scaling factor of the input image without needing to create a new TSRPass instance.

Usage:

  1. (In glThread) Create a TSRPass object using the constructor.
  2. (In glThread) Initialize the TSRPass object by calling the init(int, int, float) method.
  3. (In glThread) If the input image dimensions or scaling factor change and you need to update the TSRPass configuration without creating a new instance, call the reInit(int, int, float) method.
  4. (In glThread) Perform super-resolution rendering on an input OpenGL texture by calling the render(int) method.
  5. (In glThread) Release resources when the TSRPass object is no longer needed by calling the deInit() method.

STANDARD SDK Example:

 // The code below must be executed in the same glThread.
 //----------------------GL Thread---------------------//

 // Create a TSRPass object using the constructor.
 TSRPass tsrPass = new TSRPass(TSRPass.TSRAlgorithmType.STANDARD);

 // Initialize TSRPass and set the input image width, height and srRatio.
 tsrPass.init(inputWidth, inputHeight, srRatio);

 // If the input image dimensions or srRatio change, reinitialize TSRPass with new parameters.
 tsrPass.reInit(newInputWidth, newInputHeight, newSrRatio);

 // Perform super-resolution rendering on the input OpenGL texture and get the enhanced texture ID.
 int outputTextureId = tsrPass.render(inputTextureId);

 // Release resources when the TSRPass object is no longer needed.
 tsrPass.deInit();

 //----------------------GL Thread---------------------//

PROFESSIONAL SDK Example:

 // The code below must be executed in the same glThread.
 //----------------------GL Thread---------------------//

 // Create a TSRPass object using the constructor.
 TSRPass tsrPass = new TSRPass(TSRPass.TSRAlgorithmType.PROFESSIONAL_HIGH_QUALITY);

 // Initialize TSRPass and set the input image width, height and srRatio.
 tsrPass.init(inputWidth, inputHeight, srRatio);

 // If the input image dimensions or srRatio change, reinitialize TSRPass with new parameters.
 tsrPass.reInit(newInputWidth, newInputHeight, newSrRatio);

 // Perform super-resolution rendering on the input OpenGL texture and get the enhanced texture ID.
 int outputTextureId = tsrPass.render(inputTextureId);

 // Release resources when the TSRPass object is no longer needed.
 tsrPass.deInit();

 //----------------------GL Thread---------------------//

  • 嵌套类概要

    嵌套类
    修饰符和类型
    类
    说明
    static interface 
    TSRPass.FallbackListener
    A listener interface for handling fallback events in the super-resolution process.
    static enum 
    TSRPass.TSRAlgorithmType
    The TSRAlgorithmType enum represents the algorithm running mode to be used for super-resolution rendering in the TSRPass class.
    static enum 
    TSRPass.TSRInitStatusCode
    Enum representing various initialization errors for TSRPass.

  • 构造器概要

    构造器
    构造器
    说明
    TSRPass(TSRPass.TSRAlgorithmType algorithmType)
    Creates a TSRPass object for super-resolution rendering.

  • 方法概要

    修饰符和类型
    方法
    说明
    int
    benchmarkProSR(int inputWidth, int inputHeight, float srRatio)
    Evaluates the rendering time consumption of the PROFESSIONAL algorithm.
    TSRPass.TSRInitStatusCode
    configureProSRMaxInputResolution(int maxInputWidth, int maxInputHeight)
    Configures the maximum input resolution for the super-resolution process.
    void
    deInit()
    Releases the resources associated with the TSRPass object.
    void
    disableProSRAutoFallback()
    Disables the automatic fallback behavior for the super-resolution process.
    void
    enableProSRAutoFallback(int consecutiveTimeoutFrames, int timeoutDurationMs, TSRPass.FallbackListener listener)
    Configures and activates the automatic fallback mechanism for super-resolution processing.
    void
    forceProSRFallback(boolean enable)
    Switches between the PROFESSIONAL and STANDARD algorithms.
    TSRPass.TSRInitStatusCode
    init(int inputWidth, int inputHeight, float srRatio)
    Initializes the TSRPass object for super-resolution rendering.
    TSRPass.TSRInitStatusCode
    reInit(int inputWidth, int inputHeight, float srRatio)
    Reinitializes the TSRPass object for super-resolution rendering.
    int
    render(int textureId)
    Performs the super-resolution rendering operation on the input image.
    int
    render(int textureId, float[] transformMatrix)
    Performs the super-resolution rendering operation on the input image with a specified transform matrix.
    void
    setParameters(float brightness, float saturation, float contrast, float sharpness)
    Set the parameters of the rendering operator.

    从类继承的方法 java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • 构造器详细资料

    • TSRPass

      public TSRPass(TSRPass.TSRAlgorithmType algorithmType)
      Creates a TSRPass object for super-resolution rendering.

      This constructor should be called after initializing the TSRSdk using TSRSdk.init(long, int, TSRSdk.TSRSdkLicenseVerifyResultCallback, TSRLogger). The created TSRPass object will use the specified TSRPass.TSRAlgorithmType for its rendering operations.

      参数:
      algorithmType - The TSRAlgorithmType enum value representing the algorithm running mode to be used. It can be either STANDARD (fast mode) or PROFESSIONAL (quality mode). STANDARD mode prioritizes better performance in terms of speed, possibly sacrificing some image quality. PROFESSIONAL mode prioritizes better image quality, even if the running speed is slower.
      另请参阅:

  • 方法详细资料

    • configureProSRMaxInputResolution

      public TSRPass.TSRInitStatusCode configureProSRMaxInputResolution(int maxInputWidth, int maxInputHeight)
      Configures the maximum input resolution for the super-resolution process. This method is used to pre-allocate memory for the underlying neural network inference framework, helping to avoid performance overhead when resetting the resolution or issues with super-resolution not taking effect during reinitialization.

      This method can be called from a non-GL thread and must be invoked before the init(int, int, float) and reInit(int, int, float) methods; otherwise, those methods will be ineffective.

      Note: This method is only required for the following algorithm types:

      The TSRPass.TSRAlgorithmType.STANDARD mode does not require this method to be called.

      参数:
      maxInputWidth - The maximum width of the input image. This value should be between [8, 4096].
      maxInputHeight - The maximum height of the input image. This value should be between [8, 4096].
      返回:
      A TSRPass.TSRInitStatusCode enum value indicating the result of the configuration.

    • init

      public TSRPass.TSRInitStatusCode init(int inputWidth, int inputHeight, float srRatio)
      Initializes the TSRPass object for super-resolution rendering. This method must be called within the GL thread and after configureProSRMaxInputResolution(int, int) has been successfully executed. The input parameters specify the dimensions of the input image to be processed and the magnification factor for super-resolution.

      The valid ranges for the input dimensions depend on the selected TSRPass.TSRAlgorithmType:

      参数:
      inputWidth - The new width of the input image to be processed. This value should be within the valid range based on the algorithm type.
      inputHeight - The new height of the input image to be processed. This value should be within the valid range based on the algorithm type.
      srRatio - The magnification factor for super-resolution. This value should be greater than zero and determines the scaling applied to the input image during the rendering process.
      返回:
      A TSRPass.TSRInitStatusCode enum value indicating the result of the initialization.

    • reInit

      public TSRPass.TSRInitStatusCode reInit(int inputWidth, int inputHeight, float srRatio)
      Reinitializes the TSRPass object for super-resolution rendering. This method must be called within the GL thread and after configureProSRMaxInputResolution(int, int) has been successfully executed. The input parameters specify the new dimensions of the input image to be processed and the new magnification factor for super-resolution.

      The valid ranges for the input dimensions depend on the selected TSRPass.TSRAlgorithmType:

      参数:
      inputWidth - The new width of the input image to be processed. This value should be within the valid range based on the algorithm type.
      inputHeight - The new height of the input image to be processed. This value should be within the valid range based on the algorithm type.
      srRatio - The new magnification factor for super-resolution. This value should be greater than zero and determines the scaling applied to the input image during the rendering process.
      返回:
      A TSRPass.TSRInitStatusCode enum value indicating the result of the reinitialization.

    • enableProSRAutoFallback

      public void enableProSRAutoFallback(int consecutiveTimeoutFrames, int timeoutDurationMs, TSRPass.FallbackListener listener)
      Configures and activates the automatic fallback mechanism for super-resolution processing.

      This method will determine a timeout when the processing time exceeds the specified timeoutDurationMs. If the number of consecutive timeout frames reaches the configured threshold consecutiveTimeoutFrames, an automatic fallback operation will be triggered.

      The threshold for consecutive timeout frames, consecutiveTimeoutFrames, is dynamically adjusted based on the device's performance. In cases of lower device performance, the threshold will be reduced. The adjustment mechanism is as follows:

      • If the processing time is twice timeoutDurationMs, the threshold will be halved.
      • If the processing time is three times timeoutDurationMs, the threshold will be reduced to one-third, and so on.

      The minimum value for the threshold is 2. The formula for updating the threshold is: consecutiveTimeoutFrames = max(2, consecutiveTimeoutFrames / (timeConsume / timeoutDurationMs)), where timeConsume is the actual processing time.

      Through this dynamic adjustment mechanism, the system can flexibly respond to the current performance conditions, ensuring stability and efficiency during the processing.

      参数:
      consecutiveTimeoutFrames - The initial threshold for the number of consecutive timeout frames before triggering a fallback. This value may be dynamically adjusted based on performance.
      timeoutDurationMs - The duration in milliseconds that defines the timeout threshold for processing. If processing exceeds this duration, it is considered a timeout.
      listener - The listener that will be notified when a fallback event occurs. The listener's onFallback method will be called with the relevant parameters.

    • disableProSRAutoFallback

      public void disableProSRAutoFallback()
      Disables the automatic fallback behavior for the super-resolution process.

      This method should be called to turn off the automatic fallback feature that was previously enabled using enableProSRAutoFallback(int, int, FallbackListener). Once this method is invoked, the system will no longer trigger a fallback based on the configured parameters.

    • benchmarkProSR

      public int benchmarkProSR(int inputWidth, int inputHeight, float srRatio)
      Evaluates the rendering time consumption of the PROFESSIONAL algorithm.

      This method assesses the execution time in milliseconds for the PROFESSIONAL algorithm based on the given input dimensions. This method should not be called on the main thread, as it may take approximately 2 to 5 seconds to complete.

      This method only takes effect if the TSRPass.TSRAlgorithmType used to create the TSRPass is not set to TSRPass.TSRAlgorithmType.STANDARD. If the algorithm type is STANDARD, the evaluation may not be applicable.

      If the execution of the algorithm fails for any reason, this method will return -1.

      参数:
      inputWidth - The width of the input.
      inputHeight - The height of the input.
      返回:
      The estimated rendering time consumption in milliseconds, or -1 if the execution fails.

    • forceProSRFallback

      public void forceProSRFallback(boolean enable)
      Switches between the PROFESSIONAL and STANDARD algorithms.

      This method enables or disables the use of the STANDARD algorithm. When enable is true, the system will switch to the STANDARD algorithm; otherwise, it will use the PROFESSIONAL algorithm.

      This method only takes effect if the TSRPass.TSRAlgorithmType used to create the TSRPass is not set to TSRPass.TSRAlgorithmType.STANDARD.

      参数:
      enable - true to switch to the STANDARD algorithm; false to use the PROFESSIONAL algorithm.

    • setParameters

      public void setParameters(float brightness, float saturation, float contrast, float sharpness)
      Set the parameters of the rendering operator. It needs to take effect after init(int, int, float) is called.
      参数:
      brightness - The brightness level to apply. Valid values are between 0 and 100, with a default value of 50.
      saturation - The saturation level to apply. Valid values are between 0 and 100, with a default value of 50.
      contrast - The contrast level to apply. Valid values are between 0 and 100, with a default value of 50.
      sharpness - The sharpness factor to apply Valid values are between 0 and 100, with a default value of 0.

    • render

      public int render(int textureId)
      Performs the super-resolution rendering operation on the input image. This method must be called within the GLThread.

      This method applies the super-resolution rendering process to the input image, improving its quality by increasing its resolution. The processed image is rendered onto a texture within the TSRPass object, and the texture's ID is returned.

      NOTE:

      • The type of the texture corresponding to the textureId you pass in must be texture2D.
      • The format of the input OpenGL texture must be GL_RGBA.
      • The format of the output textureId is also GL_RGBA.

      参数:
      textureId - The OpenGL textureId of the input image that needs to be processed for super-resolution. The corresponding texture is a texture2D texture with GL_RGBA format.
      返回:
      The textureId of the texture2D after super-resolution rendering, which is stored inside the TSRPass object. The format of the output texture is GL_RGBA. The size of the output texture is (inputWidth * srRatio, inputHeight * srRatio).

    • render

      public int render(int textureId, float[] transformMatrix)
      Performs the super-resolution rendering operation on the input image with a specified transform matrix. This method must be called within the GLThread.

      This method applies the super-resolution rendering process to the input image, improving its quality by increasing its resolution. Additionally, it uses the given transform matrix to manipulate the image during the rendering process. The processed image is rendered onto a texture within the TSRPass object, and the texture's ID is returned.

      NOTE:

      • The type of the texture corresponding to the textureId you pass in must be texture2D.
      • The format of the input OpenGL texture must be GL_RGBA.
      • The format of the output textureId is also GL_RGBA.

      参数:
      textureId - The OpenGL textureId of the input image that needs to be processed for super-resolution. The corresponding texture is a texture2D texture with GL_RGBA format.
      transformMatrix - A float array representing the transform matrix to be applied to the input image during the rendering process. The matrix should be in column-major order and of size 16.
      返回:
      The textureId of the texture2D after super-resolution rendering and applying the transform matrix, which is stored inside the TSRPass object. The format of the output texture is GL_RGBA. The size of the output texture is (inputWidth * srRatio, inputHeight * srRatio).

    • deInit

      public void deInit()
      Releases the resources associated with the TSRPass object.

      This method should be called when the TSRPass object is no longer needed, to free up memory and other resources. After calling this method, the TSRPass object should not be used for any further operations.