X Tutup
Skip to content

Add blend modes and blend groups for compositing#31162

Open
ayshih wants to merge 20 commits intomatplotlib:mainfrom
ayshih:agg_compositing
Open

Add blend modes and blend groups for compositing#31162
ayshih wants to merge 20 commits intomatplotlib:mainfrom
ayshih:agg_compositing

Conversation

@ayshih
Copy link
Contributor

@ayshih ayshih commented Feb 15, 2026

PR summary

This PR adds support for blend modes beyond alpha blending (e.g., "screen" or "hard light"), so closes #6210. With this PR, all artists can specify blend_mode, and they are supported by Agg-based and Cairo-based backends, and mostly supported by SVG/PDF/PGF backends.

Of course, mplcairo provides access to these blend modes, but this PR provides blend-mode support without needing cairo.

Update: This PR uses this functionality to fix a long-standing bug (e.g., fixes #27016) with Agg rendering of Gouraud shading, where the edges of triangles would become visible when transparency is involved.

Update: This PR adds support for isolated transparency groups for every backend that supports non-normal blending.


Blend modes Agg Cairo SVG PDF PGF PS
normal
multiply, screen, overlay, darken, lighten,
color dodge, color burn, hard light, soft light,
difference, exclusion
hue, saturation, color, luminosity ✅*
knockout, erase, clear, atop, xor, plus
  • "normal" is the normal alpha blending (also known as "over" or "source over")
  • "multiply" through "exclusion" are separable blend modes (color channels are independent)
  • "hue" through "luminosity" are non-separable blend modes
  • "knockout" through "plus" are Porter Duff compositing operators, where "knockout" is also known as "source" and "erase" is also known as "destination out"
  • ✅ = supported, ❌ = not supported
  • * Text artists disappear on Windows, likely a Cairo bug

Agg showcase

  • Gouraud shading can look wrong with some blend modes, for the same reason it can look wrong under normal blend mode when alpha < 1, due to overlapping triangles Now fixed
agg

Cairo showcase

  • On Windows, text is missing in non-separable blend modes, which is presumably a bug in Cairo
  • Gouraud shading is apparently not supported by the Cairo backend, so I commented out the pcolormesh call I added support for Gouraud shading

Windows

cairo

macOS

cairo-macos

SVG showcase

  • These results may not render as intended depending on the SVG renderer (try non-mobile web browsers)
  • The Porter Duff compositing operators normally use a different mechanism to command the renderer, which is inaccessible through SVG XML, so are currently disabled (and fall back to "normal" with a warning)

Figure_1

PDF showcase

  • I haven't figured out how to implement the Porter Duff compositing operators

Figure_1.pdf

PGF showcase

  • The PGF and PGF->PDF output looks fine, but the PGF->PNG output via pdftocairo (on Windows) appears to screw up some colors for the non-separable blend modes
  • Gouraud shading is apparently not supported by the PGF backend, so I commented out the pcolormesh call
  • No Porter Duff compositing operators yet again

Figure_1.pgf.pdf

Generating code

import matplotlib
#matplotlib.use('TkCairo')

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.patches import Circle, Rectangle

N = 10
data = np.arange(N**2).reshape((N, N)) % (N-1)

fig, axs = plt.subplots(3, 8, figsize=(10, 5.5), layout="tight")
axs = axs.flatten()
fig.set_facecolor("none")

blend_modes = ["normal", "multiply", "screen", "overlay",
               "darken", "lighten", "color dodge", "color burn",
               "hard light", "soft light", "difference", "exclusion",
               "hue", "saturation", "color", "luminosity",
               "knockout", "erase", "clear", "atop", "xor", "plus"]

for ax in axs:
    ax.set_facecolor("none")
    ax.set_xlim(0, 1)
    ax.set_ylim(0, 1.2)
    ax.set_axis_off()

for i, blend_mode in enumerate(blend_modes):
    axs[i].imshow(data, cmap='Reds', alpha=0.75, extent=(0, 0.8, 0, 0.8))
    axs[i].imshow(data[::-1, :], cmap='Blues', alpha=0.75, extent=(0.2, 1, 0.4, 1.2),
                  blend_mode=blend_mode)
    axs[i].pcolormesh(*np.meshgrid(np.linspace(0.6, 0.9, 5), np.linspace(0.7, 1, 5)),
                      data[:5, :5], cmap='Spectral', alpha=0.75, shading='gouraud',
                      blend_mode=blend_mode)
    axs[i].text(0.05, 0.15, "Test", weight="bold", color="c",
                blend_mode=blend_mode)
    axs[i].plot([0.1, 0.1, 0.1, 0.1, 0.2, 0.2, 0.2, 0.2], [0.7, 0.8, 0.9, 1, 0.7, 0.8, 0.9, 1],
                'p', markersize=15, markeredgecolor="orange", markerfacecolor="purple", alpha=0.75,
                blend_mode=blend_mode)
    axs[i].plot([0, 1], [1.2, 0], color="y",
                blend_mode=blend_mode)
    circ = Circle((.65, 0.5), .3, facecolor='g', alpha=0.5,
                  blend_mode=blend_mode, zorder=2)
    axs[i].add_artist(circ)

    rect = Rectangle((0, 1.2), 1, .3, facecolor='lightgray', clip_on=False)
    axs[i].add_artist(rect)
    axs[i].set_title(blend_mode)

plt.show()

Still to do:

  • Add a documentation page for isolated transparency groups
  • Write tests

Put off to future work:

  • Change the default of pcolormesh.snap to False
  • Change the default antialiasing behavior of contourf()/pcolor()/pcolormesh() to be True
  • Agg: investigate alpha edge around Gouraud shading
  • Cairo: investigate missing text for non-separable blend modes
  • SVG: somehow implement the six Porter Duff operators
  • PDF: implement the six Porter Duff operators
  • Consider implementing more Porter Duff operators

PR checklist

@ayshih ayshih changed the title WIP: Add blend modes for compositing, supported by Agg backend WIP: Add blend modes for compositing, supported by Agg-based backends Feb 15, 2026
@github-actions github-actions bot added topic: mpl_toolkit Documentation: API files in lib/ and doc/api labels Feb 15, 2026
@timhoffm
Copy link
Member

This looks interesting. Thanks for working on it.

Since I’m not into the topic I can dare to ask the stupid questions:

  • Is it correct that an Artist and its blend mode define completely how they blend with “the background” I.e. all previously drawn artists? In particular, this does not depend on the blend mode of the other artists.
  • Are all these blend modes parameter-less?

@ayshih
Copy link
Contributor Author

ayshih commented Feb 15, 2026

  • Is it correct that an Artist and its blend mode define completely how they blend with “the background” I.e. all previously drawn artists? In particular, this does not depend on the blend mode of the other artists.

Yup, that is correct: the history of how that "background" was constructed has no bearing on how the next Artist is blended in using its specific blend mode.

It's also important to remember that that the "empty" background of an Axes is solid white, and thus not actually empty as far as these blend modes are concerned. For example, using "screen" to blend in an image on a truly empty background will just return the image, but on a white background will return solid white, which can make it look instead like the image call failed. That's why I turn off the face colors in my example above.

What I still need to investigate is how my changes interact with collections of Artists. A user may want "over" blending (the default) within the collection before using a different blend mode for the collection as a whole.

  • Are all these blend modes parameter-less?

Yes. In principle, the transformation functions for the hue/saturation/color/luminosity operators could have more than one possibility, but in practice I think everyone has simply used the same functions for decades (as defined in the PDF specification).

@ayshih ayshih force-pushed the agg_compositing branch 2 times, most recently from 2221d69 to 6d49bcf Compare February 16, 2026 04:43
@anntzer
Copy link
Contributor

anntzer commented Feb 16, 2026

This is pretty cool :-)

It's also important to remember that that the "empty" background of an Axes is solid white, and thus not actually empty as far as these blend modes are concerned. For example, using "screen" to blend in an image on a truly empty background will just return the image, but on a white background will return solid white, which can make it look instead like the image call failed. That's why I turn off the face colors in my example above.

What I still need to investigate is how my changes interact with collections of Artists. A user may want "over" blending (the default) within the collection before using a different blend mode for the collection as a whole.

Actually I suspect that another possibility is to want some nonstandard blending between multiple artists, then "over" blending of the result over the background.

In general I suspect this would be related to adding support for temporary, intermediate rendering buffers, which is also something that would be useful for other purposes e.g. contour label overplotting (#26971 (comment)).

@ayshih
Copy link
Contributor Author

ayshih commented Feb 16, 2026

By the way, I decided to rename "over" to "normal". That mode of blending is referred to as "normal" often enough, and it makes it readily apparent to users that it is the standard choice (and the default).

@ayshih ayshih force-pushed the agg_compositing branch 4 times, most recently from f49e8d0 to f7af1e4 Compare February 17, 2026 14:15
@ayshih ayshih changed the title WIP: Add blend modes for compositing, supported by Agg-based backends WIP: Add blend modes for compositing, fully supported by Agg backend and mostly supported by others Feb 17, 2026
@ayshih ayshih changed the title WIP: Add blend modes for compositing, fully supported by Agg backend and mostly supported by others WIP: Add blend modes for compositing, fully supported by Agg backends and mostly supported by other major backends Feb 17, 2026
@story645
Copy link
Member

story645 commented Mar 3, 2026

but what I'm currently trying out for plotting-level API is a helper class: ArtistGroup in #31162 (comment),

If I'm following that correctly than the PathCollection would need to get wrapped in an ArtistGroup, so something like splatterplot would be implemented as a standalone method rather than an argument to scatter?

Which is fine/ a good example to post to the gallery, just thinking through where/how this is exposed.

@ayshih ayshih force-pushed the agg_compositing branch 4 times, most recently from b04119a to a12b397 Compare March 6, 2026 05:07
@ayshih
Copy link
Contributor Author

ayshih commented Mar 6, 2026

Per the discussion at today's dev meeting, the renderer methods to open/close an isolated transparency group are implemented as new methods open_blend_group()/close_blend_group() rather than expanding the existing methods open_group()/close_group(). The separation is definitely cleaner, especially with the docstring.

My niggling concern is that both open_group() and open_blend_group() accept a string as the sole required argument, a developer could get tripped up by accidentally typing, e.g., open_group("difference") instead of open_blend_group("difference") and being mystified why it's not blending as intended. Do folks think it would be reasonable for me to have open_group() check whether the specified group name is equal to the name of a blend mode, and if it is, emit a warning that the developer that open_blend_group() is probably what was intended to be called?

@anntzer
Copy link
Contributor

anntzer commented Mar 6, 2026

The problem is that the main(?) point of open_group is to allow naming of svg groups (<g>); I assume that if third-parties are using those the group names tend to be programmatically set (e.g. matching a preexisting object hierarchy) so it would be annoying to hit warnings. If you are really worried about the potential confusion (I'm not so sure about that) I would suggest making the parameters to open_group keyword-only, which should clarify the matter...

@ayshih
Copy link
Contributor Author

ayshih commented Mar 6, 2026

Yeah, I think you're right on not putting in a warning. The false positive of emitting a warning when someone legitimately wants to name an SVG group something totally reasonable like "overlay" is not worth mitigating the hypothetical confusion of someone mistakenly calling open_group() instead of open_blend_group(). When the group isn't blending as they desire, hopefully they pull up the docstring on the method they are calling and that will immediately reveal the error to them.

@ayshih
Copy link
Contributor Author

ayshih commented Mar 9, 2026

I've implemented two more Porter-Duff compositing operators – "source" and "clear" – for both Agg and Cairo backends, but I think "source" is too confusing of a name. Since "source" provides the similar behavior as a knockout group, I have renamed it "knockout" for blend_mode.

@ayshih
Copy link
Contributor Author

ayshih commented Mar 10, 2026

I've now implemented that a blend group can be an isolated group, a knockout group, both, or neither. Isolated groups are supported by all non-PS backends. Knockout groups are supported by only the PDF and PGF backends.

Even though Agg/Cairo do not support proper knockout groups, it is straightforward to simulate an isolated + knockout group by using blend_mode="knockout" instead. A non-isolated + knockout group would be more cumbersome to assemble.

Agg

Top-right panel is not supported
agg

Cairo

Top-right panel is not supported
cairo

SVG

Right column is not supported
Figure_1

PDF

Figure_1.pdf

Generating code

import matplotlib
#matplotlib.use('TkCairo')

import numpy as np

import matplotlib.pyplot as plt
from matplotlib.artist import Artist
from matplotlib.patches import Circle
from matplotlib.backends.backend_agg import RendererAgg
from matplotlib.backends.backend_cairo import RendererCairo

class ArtistGroup(Artist):
    def __init__(self, artist_list, *, group_blend_mode=None, group_alpha=1, knockout=False):
        self._artist_list = artist_list
        self._group_blend_mode = group_blend_mode
        self._group_alpha = group_alpha
        self._knockout = knockout
        super().__init__()

    def draw(self, renderer):
        alternate = isinstance(renderer, (RendererAgg, RendererCairo)) and self._knockout and self._group_blend_mode is not None
        knockout = False if alternate else self._knockout
        renderer.open_blend_group(self._group_blend_mode, alpha=self._group_alpha, knockout=knockout)
        for a in self._artist_list:
            if not a.is_transform_set():
                a.set_transform(self.get_transform())
            if alternate:
                saved_blend_mode = a.get_blend_mode()
                a.set_blend_mode('knockout')
            a.draw(renderer)
            if alternate:
                a.set_blend_mode(saved_blend_mode)
        renderer.close_blend_group()


fig, axs = plt.subplots(2, 2, figsize=(6, 6), layout='constrained')
fig.set_facecolor("none")

for i, group_blend_mode in enumerate([None, "normal"]):
    for j, knockout in enumerate([False, True]):
        axs[i, j].set_facecolor("none")
        axs[i, j].set_xlim(-1, 1)
        axs[i, j].set_ylim(-1, 1)
        axs[i, j].set_aspect("equal")

        axs[i, j].imshow(np.arange(20*20).reshape((20, 20)) % 19,
                      cmap='Spectral', extent=[-1, 1, -1, 1])

        left = Circle((-0.25, 0), 0.6, fc='y', alpha=0.75, blend_mode='multiply')
        right = Circle((0.25, 0), 0.6, fc='g', alpha=0.75, blend_mode='multiply')

        both = ArtistGroup([left, right], group_blend_mode=group_blend_mode, knockout=knockout)
        axs[i, j].add_artist(both)

axs[0, 0].set_title('non-knockout')
axs[0, 1].set_title('knockout')
axs[0, 0].set_ylabel('non-isolated')
axs[1, 0].set_ylabel('isolated')

plt.show()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: diagonal lines in pcolormesh with Gouraud shading and transparency Alternative compositing methods

4 participants

X Tutup