How do the pieces of Android's (2D) Canvas drawing pipeline fit together?

I would like to have a better understanding of how the components of Android’s (2D) Canvas drawing pipeline fit together.

For example, how do XferMode, Shader, MaskFilter and ColorFilter interact? The reference docs for these classes are pretty sparse and the docs for Canvas and Paint don’t really add any useful explanation.

  • “Canvas: trying to draw too large bitmap” when Android N Display Size set larger than Small
  • Keep annotated class in Proguard
  • How to Work With Different Screen Resolutions
  • Make Preference look disabled, but still register clicks
  • layout folder name for devices 720x1280 like samsung galaxy s3
  • Month name as a string
  • It’s also not entirely clear to me how drawing operations that have intrinsic colors (eg: drawBitmap, versus the “vector” primitives like drawRect) fit into all of this — do they always ignore the Paint‘s color and use use their intrinsic color instead?

    I was also surprised by the fact that one can do something like this:

    Paint eraser = new Paint();
    eraser.setXfermode(new PorterDuffXfermode(PorterDuff.Mode.CLEAR));
    canvas.drawOval(rectF, eraser);
    

    This erases an oval. Before I noticed this my mental-model was that drawing to a canvas (conceptually) draws to a separate “layer” and then that layer is composed with the Canvas’s Bitmap using the Paint’s transfer mode. If it were as simple as that then the above code would erase the entire Bitmap (within the clipping region) as CLEAR always sets the color (and alpha) to 0 regardless of the source’s alpha. So this implies that there’s an additional sort of masking going on to constrain the erasing to an oval.

    I did find the API demos but each demo works “in a vacuum” and doesn’t show how the thing it focusses on (eg: XferModes) interacts with other stuff (eg: ColorFilters).

    With enough time and effort I could empirically figure out how these pieces relate or go decipher the source, but I’m hoping that someone else has already worked this out, or better yet that there’s some actual documentation of the pipeline/drawing-model that I missed.

    This question was inspired by seeing the code in this answer to another SO question.

    Update

    While looking around for some documentation it occurred to me that since much the stuff I’m interested in here seems to be a pretty thin veneer on top of skia, maybe there’s some skia documentation that would be helpful. The best thing I could find is the documentation for SkPaint which says:

    There are 6 types of effects that can
    be assigned to a paint:

    • SkPathEffect – modifications to the geometry (path) before it generates an
      alpha mask (e.g. dashing)
    • SkRasterizer – composing custom mask layers (e.g. shadows)
    • SkMaskFilter – modifications to the alpha mask before it is colorized and
      drawn (e.g. blur, emboss)
    • SkShader – e.g. gradients (linear, radial, sweep), bitmap patterns
      (clamp, repeat, mirror)
    • SkColorFilter – modify the source color(s) before applying the xfermode
      (e.g. color matrix)
    • SkXfermode – e.g. porter-duff transfermodes, blend modes

    It isn’t stated explicitly, but I’m guessing that the order of the effects here is the order they appear in the pipeline.

    Related posts:

    How to get single preview frame in Camera2 API Android 5.0?
    Installation error: INSTALL_FAILED_INSUFFICIENT_STORAGE
    How to know that an app is going to be uninstalled in android?
    How to send a high-priority GCM?
    Namespace is not Bound in Android Studio
    Get id token on Android app and verify it on backend server (How to use id token?)
  • MP3 Decoding on Android
  • Google Play services out of date. Requires 9256000 but found 9080470
  • What is the difference between ANR and crash in Android?
  • How to collect native stacktrace without a scary READ_LOGS permission up front?
  • Using String selectionArgs in SQLiteDatabase.query()
  • glow when touch the screen in android?
  • 3 Solutions collect form web for “How do the pieces of Android's (2D) Canvas drawing pipeline fit together?”

    Like Romain Guy said, “This question is difficult to answer on StackOverflow”. There wasn’t really any complete documentation, and complete documentation would be kind of large to include here.

    I ended up reading through the source and doing a bunch of experiments. I took notes along the way, and ended up turning them into a document which you can see here:

    • Android’s 2D Canvas Rendering Pipeline

    as well as this diagram:

    http://xenomachina.com/android-canvas-pipeline.png

    It’s “unofficial”, obviously, so the normal caveats apply.

    Based on the above, here are answers to some of the “sub-questions”:

    It’s also not entirely clear to me how
    drawing operations that have intrinsic
    colors (eg: drawBitmap, versus the
    “vector” primitives like drawRect)
    fit into all of this — do they always
    ignore the Paint‘s color and use use
    their intrinsic color instead?

    The “source colors” come from the Shader. In drawBitmap the Shader is temporarily replaced by a BitmapShader if a non-ALPHA_8 Bitmap is used. In other cases, if no Shader is specified a Shader that just generates a solid color, the Paint‘s color, is used.

    I was also surprised by the fact that
    one can do something like this:

    Paint eraser = new Paint();
    eraser.setXfermode(new PorterDuffXfermode(PorterDuff.Mode.CLEAR));
    canvas.drawOval(rectF, eraser);
    

    This erases an oval. Before I noticed
    this my mental-model was that drawing
    to a canvas (conceptually) draws to a
    separate “layer” and then that layer
    is composed with the Canvas’s Bitmap
    using the Paint’s transfer mode. If it
    were as simple as that then the above
    code would erase the entire Bitmap
    (within the clipping region) as CLEAR
    always sets the color (and alpha) to 0 regardless of the source’s alpha. So
    this implies that there’s an
    additional sort of masking going on to
    constrain the erasing to an oval.

    The XferMode applies to the “source colors” (from the Shader) and the “destination colors” (from the Canvas‘s Bitmap). The result is then blended with the destination using the mask computed in Rasterization. See the Transfer phase in the above document for more details.

    This question is difficult to answer on StackOverflow. Before I get started however, note that shapes (drawRect() for instance) do NOT have an intrinsic color. The color information always comes from the Paint object.

    This erases an oval. Before I noticed
    this my mental-model was that drawing
    to a canvas (conceptually) draws to a
    separate “layer” and then that layer
    is composed with the Canvas’s Bitmap
    using the Paint’s transfer mode. If it
    were as simple as that then the above
    code would erase the entire Bitmap
    (within the clipping region) as CLEAR
    always sets the color (and alpha) to 0
    regardless of the source’s alpha. So
    this implies that there’s an
    additional sort of masking going on to
    constrain the erasing to an oval.

    Your model is a bit off. The oval is not drawn into a separate layer (unless you call Canvas.saveLayer()), it is drawn directly onto the Canvas’ backing bitmap. The Paint’s transfer mode is applied to every pixel drawn by the primitive. In this case, only the result of the rasterization of an oval affects the Bitmap. There’s no special masking going on, the oval itself is the mask.

    Anyhow, here is a simplified view of the pipeline:

    1. Primitive (rect, oval, path, etc.)
    2. PathEffect
    3. Rasterization
    4. MaskFilter
    5. Color/Shader/ColorFilter
    6. Xfermode

    (I just saw your update and yes, what you found describes the stages of the pipeline in order.)

    The pipeline becomes just a little bit more complicated when using layers (Canvas.saveLayer()), as the pipeline doubles. You first go through the pipeline to render your primitive(s) inside an offscreen bitmap (the layer), and the offscreen bitmap is then applied to the Canvas by going through the pipeline.

    SkPathEffect – modifications to the geometry (path) before it generates an alpha mask (e.g. dashing)
    SkRasterizer – composing custom mask layers (e.g. shadows)
    SkMaskFilter – modifications to the alpha mask before it is colorized and drawn (e.g. blur)
    SkShader – e.g. gradients (linear, radial, sweep), bitmap patterns (clamp, repeat, mirror)
    SkColorFilter – modify the source color(s) before applying the xfermode (e.g. color matrix)
    SkXfermode – e.g. porter-duff transfermodes, blend modes

    http://imgur.com/0X5Yqod

    Android Babe is a Google Android Fan, All about Android Phones, Android Wear, Android Dev and Android Games Apps and so on.