Video with Alpha Transparency on the Web

> There's a feature request for the open source & cross-platform x265 codec to support transparency, but it doesn't seem to be going anywhere.

x265 has added support for alpha very recently, but only using their standalone CLI. https://bitbucket.org/multicoreware/x265_git/commits/c8c9d22...

I feel this, and not even in a situation that had complex transparency needs. I went down a path that needed the surrounding background perfectly matched to the video background, and it ended up being easier to render the video to a canvas, get the rendered pixel value, and update the surrounding background. Still necessary even knowing what the video background color should be given browser differences in rendering the video’s color. Prompted a fun conversation internally, though[1].

[1] https://www.mux.com/blog/your-browser-and-my-browser-see-dif...

I recently downloaded an "avif" thinking that I was downloading a gif. A little annoyed, I started poking around at it with ffmpeg and discovered that the file contained two video streams. I extracted both individually with -c copy into individual mkv's so I could play them individually.

The first video was the content I desired, the second video was solid white but for the same length of time as the first video. I was honestly a little flummoxed about the white stream for about 15 seconds before it hit me "this must be an alpha channel".

I don't suspect I have ever seen it in action, so I am extremely curious how well alpha channels can possibly turn out with lossy video formats where the result of any given frame is a matter of interpretation? In lossless formats like GIF the borders of objects at any given frame are perfectly defined, but lossy formats, especially ones using discrete cosine transform, where the object ends and background begins is not clear cut.

Where alpha is a requirement, this is very clever.

However, in many cases, the requirement isn't actually transparency, the requirement is a video background seamless with the containing page.

With most pages white or off-white, this can be done at production. Even responsive dark mode can be done in production if two clips are made.

We used this simpler technique for borderless animated video (e.g., a streaming video spokesperson walking across your e-commerce product page) 20 years ago.

The optical illusion of borderless transparency works so surprisingly well it's unbelievable it's not seen widely.

Spent hours solving the same problem, landed on this tool: https://rotato.app/tools/converter

It will take transparent MP4/MOV as input and output compatible versions for Safari (HEVC) and Chrome (WEBM). Then you simply use a tag that includes both e.g:

    <video playsinline preload="metadata">
      <source src="video.mp4" type="video/mp4; codecs=hvc1">
      <source src="video.webm" type="video/webm">
    </video>

It has always both surprised and dismayed me that transparent video on the web is still so damn hard. It's nice to see someone managing to make it work and then sharing their solution.

It's frustrating to see full vs limited range is still an issue in 2024. I was hoping it would have been figured out for web video by now, but we're still in a situation (at least on Windows) where you can end up having to go into the nvidia control panel and mess with settings to make video look correct in webpages... And then in the case where you manually split the alpha channel out, limited range could mess up the alpha channel. Painful.

I wonder if SVG filters would let you do the "manual" approach without JavaScript. IIRC they're hardware-accelerated by default in Chrome, and they usually still work in browsers that disable WebGL for security/privacy reasons.

What about animated PNG?

https://apng.onevcat.com/demo/

Or animated webp?

https://mathiasbynens.be/demo/animated-webp

https://cobalt.googlesource.com/cobalt/+/refs/tags/trunk.unn...

> Here's a demo, but… don't get your hopes up

-- Continues to render perfectly fine and smooth with 60 fps on my Windows/Firefox/Thinkpad

Ah, the problems of good software running on high-powered machines. Closed, bug not reproducable!

Here you go - problem solved - maybe? This link shows animated transparent webp:

https://static.crowdwave.link/transparentanimatedwebp.html

Seems to work on Chrome, Firefox and Safari.

The video file is here: https://static.crowdwave.link/transparentanimatedwebp.webp

I put the result of the code below here:

This python program will generate webp frames and if you have ffmpeg will convert them to transparent animated webp:

    from PIL import Image, ImageDraw
    import random
    import subprocess
    
    # Image parameters
    width, height = 640, 480
    frames = 1000
    shapes = 10
    
    def random_color():
        # Generate a random RGBA color with varying levels of transparency (alpha)
        return (random.randint(0, 255), random.randint(0, 255), random.randint(0, 255), random.randint(50, 200))
    
    def random_position():
        return random.randint(0, width - 80), random.randint(0, height - 80)
    
    def random_size():
        return random.randint(20, 80)
    
    def create_gradient(width, height):
        base = Image.new('RGBA', (width, height))
        top = Image.new('RGBA', (width, height))
        for y in range(height):
            alpha = int((y / height) * 255)  # Alpha varies from 0 to 255
            for x in range(width):
                top.putpixel((x, y), (255, 255, 255, alpha))  # White gradient
        return Image.alpha_composite(base, top)
    
    # Create a series of images
    for i in range(frames):
        # Create an image with a transparent gradient background
        img = create_gradient(width, height)
        draw = ImageDraw.Draw(img)
    
        for _ in range(shapes):
            shape_type = random.choice(['rectangle', 'ellipse'])
            x1, y1 = random_position()
            x2, y2 = x1 + random_size(), y1 + random_size()
            color = random_color()
    
            if shape_type == 'rectangle':
                draw.rectangle([x1, y1, x2, y2], fill=color)
            elif shape_type == 'ellipse':
                draw.ellipse([x1, y1, x2, y2], fill=color)
    
        # Save each frame as a PNG file
        img.save(f'frame_{i:04d}.png')
    
    print("Frames created successfully.")
    
    # Create an animated WebP from the generated frames
    subprocess.run([
        'ffmpeg', '-y', '-i', 'frame_%04d.png', '-vf', 'fps=30,scale=320:-1:flags=lanczos',
        '-loop', '0', '-pix_fmt', 'yuva420p', 'output.webp'
    ])
    
    print("Animated WebP created successfully.")

Then put the output.webp and this index.html on your local disk somewhere and load the index.html in a browser.

Pull the slider to change the background so you can see it is transparent.

    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <title>Animated WebP Display with Background Image</title>
        <style>
            body {
                display: flex;
                justify-content: center;
                align-items: center;
                height: 100vh;
                margin: 0;
                background-image: url('transparentanimatedwebp.jpg');
                background-size: cover;
                background-position: center;
                background-repeat: no-repeat;
            }
            img {
                max-width: 100%;
                height: auto;
            }
        </style>
    </head>
    <body>
    
        <img src="transparentanimatedwebp.webp" alt="Animated WebP">
    
    </body>
    </html>

> 8bit is pretty minimal when it comes to gradients, so this ~15% reduction can result in banding.

If you can already have an access to WebGL shaders, probably banding can be fixed in shaders too.

Apple ProRes 4444 deserves a mention. It's used a lot in professional contexts like digital signage, event projections etc.

Edit: sorry, ProRes is of course not "on the web".

Phew, old times, I had to solve this with a crafty solution using Flash back in the day.

(Un)surprised to know this is still an issue in 2024.