Android 13 - Programmable shaders

• New APIs from Android 13
  • RuntimeShader

Android 13 adds support for programmable RuntimeShader objects, with behavior defined using the Android Graphics Shading Language (AGSL).

 

Definition

Programmerble shaders

• As you know OpenGL draws faster because of using GPU.
• It means we can do per-pixel lighting and other neat effects, like cartoon-cel shading at runtime. 

 

Vertices

• A set of independent points.
• We can build objects by using vertices.

 

Shader

• Shaders are defined in GLSL(OpenGL’s shading language). - similar to C language.
• We can draw objects by using shaders.
• Small programs that tell OpenGL how to draw an object.
• A vertex shader
  • generates the final position of each vertex and is run once per vertex. Once the final positions are known, OpenGL will take the visible set of vertices and assemble them into points, lines, and triangles.
  • AirHockey1/res/raw/simple_vertex_shader.glsl
    attribute vec4 a_Position;
    void main() {
        gl_Position = a_Position;
    }
    This vertex shader will be called once for every single vertex that we’ve defined. When it’s called, it will receive the current vertex’s position in the a_Position attribute, which is defined to be a vec4.
    A vec4 is a vector consisting of four components. A fragment shader
• A fragment shader
  • generates the final color of each fragment of a point, line, or triangle and is run once per fragment. A fragment is a small, rectangular area of a single color, analogous to a pixel on a computer screen.
  • AirHockey1/res/raw/simple_fragment_shader.glsl
    precision mediump float;
    uniform vec4 u_Color;
    void main() {
        gl_FragColor = u_Color;
    }
    Your mobile display is composed of thousands to millions of small, individual components known as pixels. Each of these pixels appears to be capable of displaying a single color out of a range of millions of different colors. However, this is actually a visual trick: most displays can’t actually create millions of different colors, so instead each pixel is usually composed of just three indi- vidual subcomponents that emit red, green, and blue light, and because each pixel is so small, our eyes blend the red, green, and blue light together to create a huge range of possible colors. Put enough of these individual pixels together and we can show a page of text or the Mona Lisa.
    OpenGL creates an image that we can map onto the pixels of our mobile dis- play by breaking down each point, line, and triangle into a bunch of small fragments through a process known as rasterization. These fragments are analogous to the pixels on your mobile display, and each one also consists of a single solid color. To represent this color, each fragment has four compo- nents: red, green, and blue for color, and alpha for transparency.
    Rasterization: generating fragments
    fragment of the primitive, so if a triangle maps onto 10,000 fragments, then the fragment shader will be called 10,000 times.

 

AGSL (Android Graphics Shading Language)

• To define the behavior of programmable RuntimeShader objects.
• AGSL and GLSL are very similar in syntax, allowing many GLSL fragment shader effects to be brought over to Android with minimal changes. 
• It works within the Android graphics rendering system to both customize painting within Canvas and filter View content.

 

LinearGradient	RadialGradient	SweepGradient	RuntimeShader
API level 1			API level 33

Android internally uses these shaders to implement ripple effects, blur, and stretch overscroll, and Android 13 enables you to create similar advanced effects for your app. 

 

How to use RuntimeShader?

 

A simple AGSL shader

Your shader code is called for each drawn pixel, and returns the color the pixel should be painted with. An extremely simple shader is one that always returns a single color; this example uses red. The shader is defined inside of a String.

private const val COLOR_SHADER_SRC =
   """half4 main(float2 fragCoord) {
      return half4(1,0,0,1);
   }"""

The next step is to create a RuntimeShader object initialized with your shader string. This also compiles the shader.

val fixedColorShader = RuntimeShader(COLOR_SHADER_SRC)

Your RuntimeShader can be used anywhere a standard Android shader can. As an example, you can use it to draw into a custom View using a Canvas.

val paint = Paint()
paint.shader = fixedColorShader
override fun onDrawForeground(canvas: Canvas?) {
   canvas?.let {
      canvas.drawPaint(paint) // fill the Canvas with the shader
   }
}

Result image: 

This draws a red View. You can use a uniform to pass a color parameter into the shader to be drawn. First, add the color uniform to the shader:

private const val COLOR_SHADER_SRC =
"""layout(color) uniform half4 iColor;
   half4 main(float2 fragCoord) {
      return iColor;
   }"""

Then, call setColorUniform from your custom View to pass the desired color into the AGSL shader.

fixedColorShader.setColorUniform("iColor", Color.GREEN )

Now, you get a green View; the View color is controlled using a parameter from code in your custom View instead of being embedded in the shader.

Result image: 

 

Drawing the gradient

You can create a color gradient effect instead. You'll first need to change the shader to accept the View resolution as input:

private const val COLOR_SHADER_SRC =
"""uniform float2 iResolution;
   half4 main(float2 fragCoord) {
      float2 scaled = fragCoord/iResolution.xy;
      return half4(scaled, 0, 1);
   }"""

This shader does something slightly fancy. For each pixel, it creates a float2 vector that contains the x and y coordinates divided by the resolution, which will create a value between zero and one. It then uses that scaled vector to construct the red and green components of the return color.

You pass the resolution of the View into an AGSL shader uniform by calling setFloatUniform.

val paint = Paint()
paint.shader = fixedColorShader
override fun onDrawForeground(canvas: Canvas?) {
   canvas?.let {
      fixedColorShader.setFloatUniform("iResolution", width.toFloat(), height.toFloat())
      canvas.drawPaint(paint)
   }
}

Red and green gradient:

 

Full source codes

// activity_simple_shader.xml
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".SimpleShaderActivity">

    <com.example.opengles.custom.SimpleShaderImageView
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        app:shaderName="gradient" /> <!-- simple, gradient --> 
</androidx.constraintlayout.widget.ConstraintLayout>

// SimpleShaderImageView.kt
@RequiresApi(33)
class SimpleShaderImageView @JvmOverloads constructor(
    context: Context, attrs: AttributeSet? = null
) : AppCompatImageView(context, attrs) {

    private var shaderName: String?
    private val simpleShader: RuntimeShader = RuntimeShader(
        """
            layout(color) uniform half4 iColor;
            half4 main(float2 fragCoord) {
               return iColor;
            }
        """.trimMargin()
    )
    private val gradientShader: RuntimeShader = RuntimeShader(
        """
            uniform float2 iResolution;
            half4 main(float2 fragCoord) {
                float2 scaled = fragCoord / iResolution.xy; // both of x = 0~1080, y = 0~1668
                return half4(scaled, 0, 1);
            }
        """.trimMargin()
    )
    private val paint: Paint = Paint()

    init {
        context.theme.obtainStyledAttributes(
            attrs,
            R.styleable.ShaderImageView,
            0, 0
        ).apply {
            try {
                shaderName = getString(R.styleable.ShaderImageView_shaderName)
            } finally {
                recycle()
            }
        }
    }

    override fun onDrawForeground(canvas: Canvas?) {
        super.onDrawForeground(canvas)
        val requiredShader: RuntimeShader = when (shaderName) {
            "gradient" -> {
                gradientShader.setFloatUniform("iResolution", width.toFloat(), height.toFloat())
                gradientShader
            }
            else -> {
                simpleShader.setColorUniform("iColor", Color.GREEN)
                simpleShader
            }
        }
        paint.shader = requiredShader
        canvas?.drawPaint(paint)
    }
}

 

Animating the shader

You can use a similar technique to animate the shader by modifying it to receive iTime and iDuration uniforms. The shader will use these values to create a triangular wave for the colors, causing them to cycle back and forth across their gradient values.

private const val DURATION = 4000f
private const val COLOR_SHADER_SRC = """
   uniform float2 iResolution;
   uniform float iTime;
   uniform float iDuration;
   half4 main(in float2 fragCoord) {
      float2 scaled = abs(1.0-mod(fragCoord/iResolution.xy+iTime/(iDuration/2.0),2.0)); // 0~1
      return half4(scaled, 0, 1.0);
   }
"""

 

From the custom view source code, a ValueAnimator updates the iTime uniform.

// declare the ValueAnimator
private val shaderAnimator = ValueAnimator.ofFloat(0f, DURATION)

// use it to animate the time uniform
shaderAnimator.duration = DURATION.toLong()
shaderAnimator.repeatCount = ValueAnimator.INFINITE
shaderAnimator.repeatMode = ValueAnimator.RESTART
shaderAnimator.interpolator = LinearInterpolator()

animatedShader.setFloatUniform("iDuration", DURATION )
shaderAnimator.addUpdateListener { animation ->
    animatedShader.setFloatUniform("iTime", animation.animatedValue as Float )
}
shaderAnimator.start()

Red and Green animated gradient:

 

 

Full source codes: 

@RequiresApi(33)
class AnimatedGradientImageView @JvmOverloads constructor(
    context: Context, attrs: AttributeSet? = null
) : AppCompatImageView(context, attrs) {
    private val paint: Paint = Paint()
    private val shaderAnimator: ValueAnimator = ValueAnimator.ofFloat(0f, ANIMATION_DURATION)
    private val animatedGradientShader: RuntimeShader = RuntimeShader(
        """
           uniform float2 iResolution;
           uniform float iTime;
           uniform float iDuration;
           half4 main(in float2 fragCoord) {
              float2 scaled = abs(1.0-mod(fragCoord/iResolution.xy+iTime/(iDuration/2.0),2.0));
              return half4(scaled, 0, 1.0);
           }
        """.trimMargin()
    )

    init {
        // use it to animate the time uniform
        shaderAnimator.duration = ANIMATION_DURATION.toLong()
        shaderAnimator.repeatCount = ValueAnimator.INFINITE
        shaderAnimator.repeatMode = ValueAnimator.RESTART
        shaderAnimator.interpolator = LinearInterpolator()

        animatedGradientShader.setFloatUniform(
            "iDuration",
            ANIMATION_DURATION
        )
        shaderAnimator.addUpdateListener { animation ->
            animatedGradientShader.setFloatUniform("iTime", animation.animatedValue as Float)
            invalidate()
        }
        paint.apply {
            shader = animatedGradientShader
        }
        shaderAnimator.start()
    }

    override fun onLayout(changed: Boolean, left: Int, top: Int, right: Int, bottom: Int) {
        super.onLayout(changed, left, top, right, bottom)

        animatedGradientShader.setFloatUniform("iResolution", width.toFloat(), height.toFloat())
    }

    override fun onDraw(canvas: Canvas?) {
        super.onDraw(canvas)

        canvas?.drawPaint(paint)
    }

    companion object {
        private const val ANIMATION_DURATION = 4000f
    }
}

 

Painting complex objects

You don't have to draw the shader to fill the background; it can be used in any place that accepts a Paint object, such as drawText.

canvas.drawText(ANIMATED_TEXT, TEXT_MARGIN_DP, TEXT_MARGIN_DP + bounds.height(),
   paint)

 

Red and Green animated gradient text:

 

Full source codes:

@RequiresApi(33)
class AnimatedTextImageView @JvmOverloads constructor(
    context: Context, attrs: AttributeSet? = null
) : AppCompatImageView(context, attrs) {
    private val paint: Paint = Paint()
    private val shaderAnimator: ValueAnimator = ValueAnimator.ofFloat(0f, ANIMATION_DURATION)
    private val animatedGradientShader: RuntimeShader = RuntimeShader(
        """
           uniform float2 iResolution;
           uniform float iTime;
           uniform float iDuration;
           half4 main(in float2 fragCoord) {
              float2 scaled = abs(1.0-mod(fragCoord/iResolution.xy+iTime/(iDuration/2.0),2.0));
              return half4(scaled, 0, 1.0);
           }
        """.trimMargin()
    )

    init {
        // use it to animate the time uniform
        shaderAnimator.duration = ANIMATION_DURATION.toLong()
        shaderAnimator.repeatCount = ValueAnimator.INFINITE
        shaderAnimator.repeatMode = ValueAnimator.RESTART
        shaderAnimator.interpolator = LinearInterpolator()

        animatedGradientShader.setFloatUniform(
            "iDuration",
            ANIMATION_DURATION
        )
        shaderAnimator.addUpdateListener { animation ->
            animatedGradientShader.setFloatUniform("iTime", animation.animatedValue as Float)
            invalidate()
        }
        paint.apply {
            shader = animatedGradientShader
        }
        shaderAnimator.start()
    }

    override fun onLayout(changed: Boolean, left: Int, top: Int, right: Int, bottom: Int) {
        super.onLayout(changed, left, top, right, bottom)

        animatedGradientShader.setFloatUniform("iResolution", width.toFloat(), height.toFloat())
    }

    override fun onDraw(canvas: Canvas?) {
        super.onDraw(canvas)

        paint.textSize = 250f
        canvas?.drawText("SHADER", 0f, 20f + paint.textSize, paint)
    }

    companion object {
        private const val ANIMATION_DURATION = 4000f
    }
}

 

 

Shading and Canvas transformations

You can apply additional Canvas transformations on your shaded text, such as rotation. In the ValueAnimator, you can update a matrix for 3D rotations using the built-in [android.graphics.Camera](/reference/android/graphics/Camera) class.

// in the ValueAnimator
camera.rotate(0.0f, animation.animatedValue as Float / DURATION * 360f, 0.0f)

Since you want to rotate the text from the center axis rather than from the corner, get the text bounds and then use preTranslate and postTranslate to alter the matrix to translate the text so that 0,0 is the center of the rotation without changing the position the text is drawn on the screen.

linearColorPaint.getTextBounds(ANIMATED_TEXT, 0, ANIMATED_TEXT.length, bounds)
camera.getMatrix(rotationMatrix)
val centerX = (bounds.width().toFloat())/2
val centerY = (bounds.height().toFloat())/2
rotationMatrix.preTranslate(-centerX, -centerY)
rotationMatrix.postTranslate(centerX, centerY)
canvas.save()
canvas.concat(rotationMatrix)
canvas.drawText(ANIMATED_TEXT, 0f, 0f + bounds.height(), paint)
canvas.restore()

Red and Green rotating animated gradient text: 

Full source codes:

@RequiresApi(33)
class RotatedTextImageView @JvmOverloads constructor(
    context: Context, attrs: AttributeSet? = null
) : AppCompatImageView(context, attrs) {
    private val camera: Camera = Camera()
    private val linearColorPaint: Paint = Paint()
    private val paint: Paint = Paint()
    private val rotationMatrix = Matrix()
    private val bounds = Rect()
    private val shaderAnimator: ValueAnimator = ValueAnimator.ofFloat(0f, ANIMATION_DURATION)
    private val animatedGradientShader: RuntimeShader = RuntimeShader(
        """
           uniform float2 iResolution;
           uniform float iTime;
           uniform float iDuration;
           half4 main(in float2 fragCoord) {
              float2 scaled = abs(1.0-mod(fragCoord/iResolution.xy+iTime/(iDuration/2.0),2.0));
              return half4(scaled, 0, 1.0);
           }
        """.trimMargin()
    )

    init {
        // use it to animate the time uniform
        shaderAnimator.duration = ANIMATION_DURATION.toLong()
        shaderAnimator.repeatCount = ValueAnimator.INFINITE
        shaderAnimator.repeatMode = ValueAnimator.RESTART
        shaderAnimator.interpolator = LinearInterpolator()

        animatedGradientShader.setFloatUniform(
            "iDuration",
            ANIMATION_DURATION
        )
        shaderAnimator.addUpdateListener { animation ->
            animatedGradientShader.setFloatUniform("iTime", animation.animatedValue as Float)
            // TODO More slowly
            camera.rotate(0.0f, animation.animatedValue as Float / ANIMATION_DURATION * 360f, 0.0f)
            invalidate()
        }
        paint.apply {
            shader = animatedGradientShader
        }
        shaderAnimator.start()
    }

    override fun onLayout(changed: Boolean, left: Int, top: Int, right: Int, bottom: Int) {
        super.onLayout(changed, left, top, right, bottom)

        animatedGradientShader.setFloatUniform("iResolution", width.toFloat(), height.toFloat())
    }

    override fun onDraw(canvas: Canvas?) {
        super.onDraw(canvas)

        paint.textSize = 250f

        canvas ?: return

        paint.getTextBounds(ANIMATED_TEXT, 0, ANIMATED_TEXT.length, bounds)
        camera.getMatrix(rotationMatrix)
        val centerX = (bounds.width().toFloat()) / 2
        val centerY = (bounds.height().toFloat()) / 2
        rotationMatrix.preTranslate(-centerX, -centerY)
        rotationMatrix.postTranslate(centerX, centerY)

        canvas.save()
        canvas.concat(rotationMatrix)
        canvas.drawText(ANIMATED_TEXT, 0f, 50f + bounds.height(), paint)
        canvas.restore()
    }

    companion object {
        private const val ANIMATED_TEXT = "SHADER"
        private const val ANIMATION_DURATION = 4000f
    }
}

 

 

Using RuntimeShader with Jetpack Compose

It's even easier to use RuntimeShader if you're rendering your UI using Jetpack Compose. Starting with the same gradient shader from before:

private const val COLOR_SHADER_SRC =
    """uniform float2 iResolution;
   half4 main(float2 fragCoord) {
   float2 scaled = fragCoord/iResolution.xy;
   return half4(scaled, 0, 1);
}"""

You can apply that shader to a ShaderBrush. You then use the ShaderBrush as a parameter to the drawing commands within your Canvas's draw scope.

// created as top level constants
val colorShader = RuntimeShader(COLOR_SHADER_SRC)
val shaderBrush = ShaderBrush(colorShader)

Canvas(
   modifier = Modifier.fillMaxSize()
) {
   colorShader.setFloatUniform("iResolution",
   size.width, size.height)
   drawCircle(brush = shaderBrush)
}

 Result image: 

Full source codes:

// app/build.gradle
dependencies {
    implementation 'androidx.compose.ui:ui:1.1.1'
}

android {
    compileSdkPreview "Tiramisu"
    defaultConfig {
        targetSdkPreview "Tiramisu"
    }
}

// build.gradle
plugins {
    id 'org.jetbrains.kotlin.android' version '1.5.31' apply false
}

// ComposeActivity.kt
@RequiresApi(33)
class ComposeActivity : ComponentActivity() {
    private val colorShader: RuntimeShader = RuntimeShader(
        """uniform float2 iResolution;
           half4 main(float2 fragCoord) {
           float2 scaled = fragCoord/iResolution.xy;
           return half4(scaled, 0, 1);
        }""".trimMargin()
    )

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        val shaderBrush = ShaderBrush(colorShader)
        setContent {
            Canvas(modifier = Modifier.fillMaxSize()) {
                colorShader.setFloatUniform("iResolution", size.width, size.height)
                drawCircle(brush = shaderBrush)
            }
        }
    }
}

 

Using RuntimeShader with RenderEffect

You can use RenderEffect to apply a RuntimeShader to a parent View and all child views. This is more expensive than drawing a custom View, but it allows you to easily create an effect that incorporates what would have originally been drawn using createRuntimeShaderEffect.

Background	Bitmap	Result

 

Full source codes

// activity_render_effect.xml
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:id="@+id/parent_view"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".RenderEffectActivity">

    <ImageView
        android:id="@+id/image_view"
        android:layout_width="200dp"
        android:layout_height="200dp"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

</androidx.constraintlayout.widget.ConstraintLayout>

// RenderEffectActivity.kt
@RequiresApi(33)
class RenderEffectActivity : AppCompatActivity() {

    private val mixShader: RuntimeShader = RuntimeShader(
        """
            uniform float2 iResolution;
            uniform shader background;
             half4 main(float2 fragCoord) {
                float2 scaled = fragCoord / iResolution.xy; // both of x = 0~1080, y = 0~1668
                return mix(half4(scaled, 0, 1), background.eval(fragCoord), 0.5);
            }
        """.trimMargin()
    )

    private lateinit var binding: ActivityRenderEffectBinding

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        binding = ActivityRenderEffectBinding.inflate(layoutInflater)
        setContentView(binding.root)

        binding.imageView.setBackgroundResource(R.drawable.ic_launcher_foreground)
        setResolutionToShader()
        setRenderEffectToParentView()
    }

    private fun setRenderEffectToParentView() {
        val renderEffect = RenderEffect.createRuntimeShaderEffect(mixShader, "background")
        binding.parentView.setRenderEffect(renderEffect)
    }

    private fun setResolutionToShader() {
        val width = windowManager.defaultDisplay.width
        val height = windowManager.defaultDisplay.height
        mixShader.setFloatUniform("iResolution", width.toFloat(), height.toFloat())
    }
}

 

Conclusion

• Draw complicated animations with GPU on Canvas by using RuntimeShader.
• Need to understand AGSL.

 

RenderScript

• Starting with Android 12, the RenderScript APIs are deprecated. 
• Although the RenderScript intrinsics functions will continue to function after RenderScript deprecation, they may execute only on the CPU rather than the GPU.
• https://developer.android.com/guide/topics/renderscript/migrate#intrinsics
  • Vulkan
  • https://developer.android.com/reference/android/graphics/RenderEffect
  • RuntimeShader

 

• https://developer.android.com/guide/topics/graphics/agsl
• https://developer.android.com/guide/topics/graphics/agslhttps://developer.android.com/codelabs/advanced-android-kotlin-training-shaders?hl=en#6 
• https://developer.android.com/reference/android/graphics/RuntimeShader
• https://developer.android.com/about/versions/13/features#graphics
• https://yggr.medium.com/exploring-android-13-programmable-shaders-db91683127e3