Development

imp.push Beta Released

After a month and a half of exhaustion on various smallish jobs, I arrived home to find a shiny new Ableton Push 2 controller waiting for me.

There's a huge amount of potential in this device, although my thoughts are probably not in the area of it's manufacturer's intended use! I see it as a high-quality RGB grid controller and a set of re-purposable function buttons, with the massive bonus of a totally customisable built-in LED display, which runs at desktop display refresh rate.

In order to start getting some of my plans for integration with various software going, my first order of business was the prove that I could output my own video to the display. This proved nicely easy thanks to Ableton's API documents and libusb, and after a few hours of effort I have a pretty simple Max object which happily streams Jitter matrices at 60 Hz over USB.

In the interests of sharing, I've released this external on Github, and you can also download it from my max externals page. I'd like to expand the scope of the external in future to include parsing for all the button LEDs (from their raw MIDI note/controller values) and also full LED control. If you'd like to help out, head over to Github and send me a pull request.

PosiStageDotNet v0.1.0 Released

In my day job at Green Hippo I've been working on various automation tracking projects recently, implementing a variety of proprietary automation UDP protocols. However one of these was PosiStageNet, a open protocol for positional data streaming maintained by VYV.

The protocol allows transport of position, speed, orientation and status/naming information for multiple defined 'trackers' using a nicely-extensible chunk format. After some discussion with VYV I requested an addition to the specification for some extra tracker information, adding acceleration and target position (this is version 2.01 of the protocol), allowing for more accurate positional estimation on the client side.

On the official site you can download a C++ implementation, and I originally developed a C# wrapper around this. However wrappers often feel like an ugly solution to me, so I decided to replace this with a full C# implementation. In the name of standardization it seemed sensible to make it both a portable library (creating some interesting opportunities for mobile usage) and open-source. After a few scattered days work over the last month or so, I've just released the first version of Imp.PosiStageDotNet via NuGet.org.

My hope is that with this library (and the original C++ implementation) the entertainment control industry can start to unify around a single, well-defined and appropriate protocol rather than the back-of-the-envelope ASCII formats which many have been using to date. Of course, as with all standardization attempts this may be futile, but at least I have something to recommend when asked to specify a protocol!

 

Cross-Platform Drawing with Skia and WPF

Skia is a cross-platform 2D graphics library, most notably used by Google Chrome and Chromium OS. SkiaSharp is a new C# cross-platform wrapper around the library.

Why is this interesting? Well it's a massive step towards being able to make cross-platform custom GUI controls. I generally prefer to use native OS controls in my application UIs, and this is usually not too time consuming to do and additionally allows you to tailor the interface to that system. However I'm soon starting work on a project which requires a large amount of totally custom widgets, and the thought of coding those individually for each platform was making my knees weak...

Enter SkiaSharp. Now I have a 2D drawing API which I can access from a portable class library. I can now write all of my custom widgets in cross-platform code. Then I can hook them up to a shell class for each platform which exposes properties, hooks up mouse and keyboard input and renders the Skia canvas to the native GUI system.

To test this idea out, I've created the WPF version of this shell object. In this case it's an abstract control (usable from any XAML document), which hosts a Skia canvas inside a WritableBitmap, allowing composition inside a WPF UI. To implement a Skia-based control, inherit from this class and override the Draw() method. Here's the source:

    /// <summary>
    ///     Abstract class used to create WPF controls which are drawn using Skia
    /// </summary>
    [PublicAPI]
    public abstract class SkiaControl : FrameworkElement
    {
        private WriteableBitmap _bitmap;
        private SKColor _canvasClearColor;

        protected SkiaControl()
        {
            cacheCanvasClearColor();
            createBitmap();
            SizeChanged += (o, args) => createBitmap();
        }

        
        /// <summary>
        ///     Color used to clear canvas before each call to <see cref="Draw" /> if <see cref="IsClearCanvas" /> is true
        /// </summary>
        [Category("Brush")]
        [Description("Gets or sets a color used to clear canvas before each render if IsClearCanvas is true")]
        public SolidColorBrush CanvasClear
        {
            get { return (SolidColorBrush)GetValue(CanvasClearProperty); }
            set { SetValue(CanvasClearProperty, value); }
        }

        public static readonly DependencyProperty CanvasClearProperty =
            DependencyProperty.Register("CanvasClear", typeof(SolidColorBrush), typeof(SkiaControl),
                new PropertyMetadata(new SolidColorBrush(Colors.Transparent), canvasClearPropertyChanged));

        private static void canvasClearPropertyChanged(DependencyObject o, DependencyPropertyChangedEventArgs args)
        {
            ((SkiaControl)o).cacheCanvasClearColor();
        }

        /// <summary>
        ///     When enabled, canvas will be cleared before each call to <see cref="Draw" /> with the value of
        ///     <see cref="CanvasClear" />
        /// </summary>
        [Category("Appearance")]
        [Description(
            "Gets or sets a bool to determine if canvas should be cleared before each render with the value of CanvasClear")]
        public bool IsClearCanvas
        {
            get { return (bool)GetValue(IsClearCanvasProperty); }
            set { SetValue(IsClearCanvasProperty, value); }
        }

        public static readonly DependencyProperty IsClearCanvasProperty =
            DependencyProperty.Register("IsClearCanvas", typeof(bool), typeof(SkiaControl), new PropertyMetadata(true));

        /// <summary>
        /// Capture the most recent control render to an image
        /// </summary>
        /// <returns>An <see cref="ImageSource"/> containing the captured area</returns>
        [CanBeNull]
        public BitmapSource SnapshotToBitmapSource() => _bitmap?.Clone();

        protected override void OnRender(DrawingContext dc)
        {
            if (_bitmap == null)
                return;

            _bitmap.Lock();

            using (var surface = SKSurface.Create((int)_bitmap.Width, (int)_bitmap.Height, 
                SKColorType.N_32, SKAlphaType.Premul, _bitmap.BackBuffer, _bitmap.BackBufferStride))
            {
                if (IsClearCanvas)
                    surface.Canvas.Clear(_canvasClearColor);

                Draw(surface.Canvas, (int)_bitmap.Width, (int)_bitmap.Height);
            }

            _bitmap.AddDirtyRect(new Int32Rect(0, 0, (int)_bitmap.Width, (int)_bitmap.Height));
            _bitmap.Unlock();

            dc.DrawImage(_bitmap, new Rect(0, 0, ActualWidth, ActualHeight));
        }

        /// <summary>
        ///     Override this method to implement the drawing routine for the control
        /// </summary>
        /// <param name="canvas">The Skia canvas</param>
        /// <param name="width">Canvas width</param>
        /// <param name="height">Canvas height</param>
        protected abstract void Draw(SKCanvas canvas, int width, int height);

        private void createBitmap()
        {
            int width = (int)ActualWidth;
            int height = (int)ActualHeight;

            if (height > 0 && width > 0 && Parent != null)
                _bitmap = new WriteableBitmap(width, height, 96, 96, PixelFormats.Pbgra32, null);
            else
                _bitmap = null;
        }

        private void cacheCanvasClearColor()
        {
            _canvasClearColor = CanvasClear.ToSkia();
        }
    }

Looking for something interesting to demonstrate the concept with, I've written a quick C#/Skia copy of this gif by Dave Whyte, who makes some amazing 2D animations using Processing. Checkout skiasharpwpfextensions on github and the build the main solution to try it out.

I'm excited to explore further how well this concept will work, my next step will be to built a Cocoa equivalent for use in OS X.