Performance of Water Simulation with Hardware Acceleration using Microsoft Frameworks

I’ve found that creating water effects that respond interactively and smoothly is not really something easy to do in WPF or Silverlight.  However because a project required it I trudged forward managed to get it working pretty nicely – buttery smooth and low CPU utilization.

The results can be seen in the video, but the reasons it wasn’t easy may be interesting to WPF/Silverlight developers so I’ll mention why below.  Can you guess what the technical obstacles are?

                              (user interaction 17 seconds in)

                              (user interaction 17 seconds in)

Graphics architecture

The first hurdle is that WPF and Silverlight primarily use a “display list” or “retained mode” graphics architecture.  This is usually not a bad thing, and allowed developers to break away from the WM_PAINT model which is used in WinForms and Win32 before that.  Those older systems encourage an “immediate mode” architecture while WPF and Silverlight encourage a retained mode, but will tolerate immediate mode in certain cases.

WPF and Silverlight

You might have nightmares about WinForms and Win32 and wish all manner of death upon them.  Death to immediate mode as well – long live WPF/Silverlight and retained mode graphics! 

Just one small problem – immediate mode graphics are well suited to certain things like games and simulations, and no one wants give those up.  In fact Direct3d itself is an immediate mode architecture which underlies most graphics in Windows 7 including WPF itself.  I’m speaking in general here and there are always exceptions.  For example I’ve written simple games and simulations in WPF, but many times it is not an ideal approach.

Animation vs. Simulation

To get more specific on why WPF/Silverlight are not great at simulating water, it’s basically about loops.  Many simulations for water and other phenomena run a programming loop over and over again throughout the simulation.  The reason is that future states of the simulation depend on the output of previous states of the simulation.  The problem is that animations are different than simulations.

In WPF you can easily animate a ball along a flat line.  WPF determines the position of the ball using a function with time as the input:

• Time elapsed 1 second  -   Move ball to 25% along the line
• Time elapsed 2 second  -   Move ball to 50% along the line
• Time elapsed 3 second  -   Move ball to 75% along the line

You could also choose any random time and the ball would be placed properly because the new position does not depend on the previous position.  It’s always just a function taking any time as an input, and spitting out a position as output.

Most animation systems work in a similar way, which is why regardless of what applications artists and animators choose it usually has a big timeline along the bottom.

So if that's how animation works, how is simulation different? 

In simulation of water our function to calculate the new position takes not only time as an input but also the previous position of the water.  This feedback loop is the key difference.

Since WPF does not encourage loops that let you feedback the output into the next input, we are kind of stuck.

What about CompositionTarget?

One ray of hope is the CompositionTarget.  This is a WPF component specifically designed to enable controlling your own rendering loop.  I consider CompositionTarget going a bit off the reservation because it’s not really a mainstream technique used by most apps, however it’s fairly clean and that’s ok if it can get us past the loop barrier.  While this lets us have precise control over the rendering loop, it still does not allow any way for output of the shaders to feed back into the loop.  

We could just not use shaders and write the code procedurally, but then the question becomes how much performance do shaders afford our simulation?

How fast are shaders in WPF?

VERY fast.  In fact to get nice performance and realistic water, custom shaders are essentially required.  It's an order(s) of magnitude difference.  Using shaders should be no problem because WPF supports shader model 3.0.  In fact in a previous blog post I showed a single drop water effect using WPF shaders and the code is simple XAML with no problems.  But again, that was an animation not a simulation.  The drops could not affect each other and you could not drag your finger through the water and create a wave as in the video above. 

To get some real coolness we need a simulation, which means using the output from the shader as the input for the next time the shader is run.  The problem is when you capture the output of shaders as data in WPF (rather than just display the output), you lose hardware acceleration and performance goes to heck in a handbasket.  One item on my wishlist for WPF 5.0 would definitely be enabling a form of shader trees where the output can be modified and looped back as input.

To keep pixel shaders running in hardware in this fashion within a WPF app requires Direct3d, which in turn requires D3DImage.  D3DImage is the class that can allow separate Direct3d C++ code to output into an image brush within a WPF app.  It also allows (requires really) the Direct3d code to have it’s own rendering loop, which allows us to feed shader output back to the input for the simulation.

With Silverlight its worse because of the lack of GPU acceleration and because the Direct3d integration is not an option.  I’ve seen a few liquid/particle simulations in Silverlight but they are quite slow, and tend to peg CPU utilization. 

So the bottom line is to create interactive effects like this in WPF really requires CompositionTarget, custom shaders, and a separate Direct3d helper DLL.  The first two techniques alone will do the same thing at about a quarter of the speed.

Every year I hope to never code in C++ again, but it looks like that time is not here quite yet.

 

Comments

[Anders]

I have been looking for a WPF shader implementation capable of doing what your video shows.
Is the source code you developed for this video available?

Thanks,
Anders

[Angelo]

Hi there,
what a cool effect!!
Did your code mix WPF, shaders and Direct3D to create the continuous touch effect?

Wondering if it will be possible to have the source code to inspect and compile myself.

Thanks in advance
Angelo

[staff]

Thanks Angelo :)

I can't give out source for this one because I did it as a consulting project, but I can talk about the approach.

Essentially it's C# taking driving the input loop (touches) and sending coordinates to custom DirectX shaders in C++. If you read through the article and have any specific questions please post back.

Regards,
Lee

[John Andrews]

Could you give me any insight to the process to complete this, and also how long it took? I've been looking into doing this kind of effect exactly, but I'm wondering if it's worth it.