Although the algorithm I described uses only two frames (current actual camera pixels & the displayed pixels) it implicitly represents data from multiple frames over time. It is sort of like recursively defined values in mathematical proofs.

Each new pixel is the weighted average of that same pixel from all preceding frames.

(1/2)P0+ (1/4)P1 + (1/8)P2 + (1/16)P3 + …

i.e. based on this mathematical series:

1/2 + 1/4 + 1/8 + 1/16 + 1/32 + … = 1

>> bilinear filtering or for GPU anti-aliasing?

Those are essentially nearest neighbor pixel averages. They operate in one frame to smooth the image but they are not based on temporal data (at least as far as I know).

I have however experimented with the combination of my temporal smoothing with various 3×3 pixel transformations for contrast enhancement or smoothing of the current frame (and also for edge enhancement and emboss effects and so forth; btw – I’m sure you’d really enjoy experimenting with these algorithms!). The temporal filtering provides a less noisy image to start with, so the results with other effects are improved.

I was particularly fascinated with the notion that I could create green-screen effects by identifying clusters of pixels that had negligible changes, and then compositing in real time with these as transparency. It worked very nicely to some extent but, if the subject was relatively still I’d have this disembodied hand or face floating around.

Two questions –

Does what you describe share something in common with the process used for bilinear filtering or for GPU anti-aliasing?

Also, are you describing a routine that works on a single frame? I wonder if that same concept could be useful over two or more frames…

Just thought I’d comment on this one.

Several years ago I implemented a similar image stream effect for a dazzling improvement in webcam clarity, but with a slight algorithmic difference from your approach. You might find it interesting for future experimentation. Instead of a mean I used a sort of binary logarithmic convolution.

The algorithm is very simple, works in integers, and doesn’t have the roundoff issue that you had been concerned about.

for each pixel, r,g,b:
shift current camera pixel right by 1 bit (i.e. divide by two)
shift previous displayed frame pixel value right by 1 bit (again a divide by two)
add these two values together to yield a new display frame pixel value

This approach is basically a weighted average of the preceding values where each frame is twice as important the closer it is to the present frame.

The effect is similarly crisp, much faster to compute, and the artifacts from motion disappear very rapidly.