Fun with Silverlight: Multi-touch hub navigation
Not all applications need to be boring, at least that’s my motto. I like to create applications from time to time that are a bit different in the way that they interact with the user. One of these wacky applications is an application that uses a hub navigation.
Introduction
A hub navigation is a navigation where you are moving around in a virtual space that you can compare to a box that is folded open. You start in a center square that contains navigation buttons on each side of the screen. Each of these buttons allows you to “move” over to a different square or hub of the application.
In the figure above you can see the schematic of the navigation of my recipe browser. This application allows you to gather and categorize recipes and order them into different menus. The idea is that the user always starts in the recipe hub and go from there. The user can choose to navigate to the menus hub to compose menus or go to the recipe settings hub to configure units of measurement.
When you navigate to a hub, you should be able to get back the way you came, so I choose not to “fold” the user interface. Which means that once you have reached a hub that has no more navigation options, you need to go back the way you came to enter other screens. This should prevent the user from getting lost.
In this article I will show you how you can create such a hub navigation and make it react to touch events on a multi-touch enabled device.
Implementing the navigation mechanics
The key to this application is the way screens are navigated. The logical choice here would be to go for a navigation using the Navigation framework provided by Silverlight. This isn’t really what I want, it’s really hard to write and it is far from logical. You have to take in consideration the fact here that the navigation is spatial rather than going back and forth between a set of pages. The navigation mechanics should match that pattern.
The navigation mechanics of the application are build out of two key elements. The first part is the layout of each of the hubs and the second part is the animation that is used to navigate between hubs.
Basic navigation setup
Each of the hubs will be made using a Silverlight user control that can contain one or more navigation buttons around the each of the screen. A sample of this can be seen in the screenshot below.
When a user clicks on a navigation button, the application has to determine which button is clicked and which way the user should be moved. For example, if the user clicks the settings button in the screenshot above, the application should move to the right in the layout and show the settings hub for the recipes.
To make navigation a bit more generic I’ve created a pair of interfaces that will work together to implement the navigation mechanics for the user interface.
The IHubNavigationController is responsible for keeping track of the active hub and changing the active hub. Each Silverlight user control implementing a hub will be implementing the IHubNavigationPage. This makes it possible for a hub to invoke the Navigate method on the IHubNavigationController instance to change the active hub.
Every time a hub invokes the Navigate method on the IHubNavigationController it has to provide a direction in which to navigate. Based on this direction there will be a different animation to visualize the direction in which the user is navigating.
The implementation of the IHubNavigationController is actually the main view of the application. This view contains a transitioning content control from the Silverlight toolkit. Each of the hubs will be displayed inside this transitioning content control.
The implementation of the Navigate method on the IHubNavigationController (Read, the main view) is pretty basic. A different transition is used for each of the directions the user can navigate in. I will talk about these transitions a bit later. The rest of the method is basic stuff you would expect to see in a navigation method.
1: public partial class MainPage : UserControl, IHubNavigationController
2: {
3: const string LeftTransition = "LeftTransition";
4: const string RightTransition = "RightTransition";
5: const string UpTransition = "UpTransition";
6: const string DownTransition = "DownTransition";
7: 
8: public MainPage()
9: {
10: InitializeComponent();
11: }
12:
13: /// <summary>
14: /// Navigates in the specified direction
15: /// </summary>
16: /// <param name="page"></param>
17: /// <param name="direction"></param>
18: public void Navigate(IHubNavigationPage page, NavigationDirection direction)
19: {
20: switch (direction)
21: {
22: case NavigationDirection.Up:
23: PageContent.Transition = UpTransition;
24: break;
25: case NavigationDirection.Down:
26: PageContent.Transition = DownTransition;
27: break;
28: case NavigationDirection.Left:
29: PageContent.Transition = LeftTransition;
30: break;
31: case NavigationDirection.Right:
32: PageContent.Transition = RightTransition;
33: break;
34: default:
35: break;
36: }
37: 
38: page.HubNavigationController = this;
39: PageContent.Content = page;
40: }
41: 
42: private void OnMainPageLoaded(object sender, RoutedEventArgs e)
43: {
44: Navigate(new RecipesNavigationHubPage(), NavigationDirection.Up);
45: }
46: }
The second part required to make the navigation work is to implement the hubs themselves. Each hub implements the IHubNavigationPage interface, exposing the HubNavigationController property. This property can be accessed to navigate to other pages. A sample of this is shown below.
1: /// <summary>
2: /// Navigates to the categories hub
3: /// </summary>
4: /// <param name="parameter"></param>
5: private void NavigateCategories(object parameter)
6: {
7: NavigationHub.Navigate(new CategoriesNavigationHubPage(), NavigationDirection.Left);
8: }
Adding animations to the user interface
Without any animation the hub navigation feels kind of awkward to the user. When you click on the navigation to the right, you would expect the screen to slide in that direction. This is not the case when you build the application without any custom transitions defined on the TransitionContentControl from the Silverlight toolkit.
For the animations to support the navigation principles you will need a couple of custom transitions. One for each direction the user can navigate in. The most important thing about these animations is that the page should actually slide away in the opposite direction of where the user is actually going. This gives a feeling that the current page is sliding away and a new page is sliding in.
To create the sliding animations you will need to create a new ControlTemplate for the TransitioningContentControl. This template looks very much like the original one and I recommend you use Expression Blend to extract a copy of the original control template. This saves time, because the up and down transitions are already there. All that is left, is to add the transitions for the left and right movement. You can find the left,right, up and down transitions in the code snippet below.
1: <ControlTemplate TargetType="toolkit:TransitioningContentControl" x:Key="NavigationHubTemplate">
2: <Border Background="{TemplateBinding Background}" BorderBrush="{TemplateBinding BorderBrush}" BorderThickness="{TemplateBinding BorderThickness}" CornerRadius="2">
3: <vsm:VisualStateManager.VisualStateGroups>
4: <vsm:VisualStateGroup x:Name="PresentationStates">
5: <!-- .... -->
6: <vsm:VisualState x:Name="UpTransition">
7: <Storyboard>
8: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="CurrentContentPresentationSite" Storyboard.TargetProperty="(UIElement.Opacity)">
9: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="0"/>
10: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="1"/>
11: </DoubleAnimationUsingKeyFrames>
12: 
13: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="CurrentContentPresentationSite" Storyboard.TargetProperty="(UIElement.RenderTransform).(TransformGroup.Children)[3].(TranslateTransform.Y)">
14: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="90"/>
15: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="0"/>
16: </DoubleAnimationUsingKeyFrames>
17: 
18: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="PreviousContentPresentationSite" Storyboard.TargetProperty="(UIElement.Opacity)">
19: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="1"/>
20: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="0"/>
21: </DoubleAnimationUsingKeyFrames>
22: 
23: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="PreviousContentPresentationSite" Storyboard.TargetProperty="(UIElement.RenderTransform).(TransformGroup.Children)[3].(TranslateTransform.Y)">
24: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="0"/>
25: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="-90"/>
26: </DoubleAnimationUsingKeyFrames>
27: </Storyboard>
28: </vsm:VisualState>
29: 
30: <vsm:VisualState x:Name="DownTransition">
31: <Storyboard>
32: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="CurrentContentPresentationSite" Storyboard.TargetProperty="(UIElement.Opacity)">
33: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="0"/>
34: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="1"/>
35: </DoubleAnimationUsingKeyFrames>
36: 
37: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="CurrentContentPresentationSite" Storyboard.TargetProperty="(UIElement.RenderTransform).(TransformGroup.Children)[3].(TranslateTransform.Y)">
38: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="-90"/>
39: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="0"/>
40: </DoubleAnimationUsingKeyFrames>
41: 
42: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="PreviousContentPresentationSite" Storyboard.TargetProperty="(UIElement.Opacity)">
43: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="1"/>
44: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="0"/>
45: </DoubleAnimationUsingKeyFrames>
46: 
47: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="PreviousContentPresentationSite" Storyboard.TargetProperty="(UIElement.RenderTransform).(TransformGroup.Children)[3].(TranslateTransform.Y)">
48: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="0"/>
49: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="90"/>
50: </DoubleAnimationUsingKeyFrames>
51: </Storyboard>
52: </vsm:VisualState>
53: 
54: <vsm:VisualState x:Name="LeftTransition">
55: <Storyboard>
56: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="CurrentContentPresentationSite" Storyboard.TargetProperty="(UIElement.Opacity)">
57: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="0"/>
58: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="1"/>
59: </DoubleAnimationUsingKeyFrames>
60: 
61: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="CurrentContentPresentationSite" Storyboard.TargetProperty="(UIElement.RenderTransform).(TransformGroup.Children)[3].(TranslateTransform.X)">
62: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="-90"/>
63: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="0"/>
64: </DoubleAnimationUsingKeyFrames>
65: 
66: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="PreviousContentPresentationSite" Storyboard.TargetProperty="(UIElement.Opacity)">
67: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="1"/>
68: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="0"/>
69: </DoubleAnimationUsingKeyFrames>
70: 
71: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="PreviousContentPresentationSite" Storyboard.TargetProperty="(UIElement.RenderTransform).(TransformGroup.Children)[3].(TranslateTransform.X)">
72: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="0"/>
73: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="90"/>
74: </DoubleAnimationUsingKeyFrames>
75: </Storyboard>
76: </vsm:VisualState>
77: 
78: <vsm:VisualState x:Name="RightTransition">
79: <Storyboard>
80: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="CurrentContentPresentationSite" Storyboard.TargetProperty="(UIElement.Opacity)">
81: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="0"/>
82: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="1"/>
83: </DoubleAnimationUsingKeyFrames>
84: 
85: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="CurrentContentPresentationSite" Storyboard.TargetProperty="(UIElement.RenderTransform).(TransformGroup.Children)[3].(TranslateTransform.X)">
86: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="90"/>
87: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="0"/>
88: </DoubleAnimationUsingKeyFrames>
89: 
90: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="PreviousContentPresentationSite" Storyboard.TargetProperty="(UIElement.Opacity)">
91: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="1"/>
92: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="0"/>
93: </DoubleAnimationUsingKeyFrames>
94: 
95: <DoubleAnimationUsingKeyFrames BeginTime="00:00:00" Storyboard.TargetName="PreviousContentPresentationSite" Storyboard.TargetProperty="(UIElement.RenderTransform).(TransformGroup.Children)[3].(TranslateTransform.X)">
96: <SplineDoubleKeyFrame KeyTime="00:00:00" Value="0"/>
97: <SplineDoubleKeyFrame KeyTime="00:00:00.300" Value="-90"/>
98: </DoubleAnimationUsingKeyFrames>
99: </Storyboard>
100: </vsm:VisualState>
101: 
102: </vsm:VisualStateGroup>
103: </vsm:VisualStateManager.VisualStateGroups>
104: <!-- The rest of the control template -->
105: </Border>
106: </ControlTemplate>
Each of the transitions uses a opacity cross-fade combined with a translate transform to create the desired effect. The TransitioningContentControl uses the VisualStateManager to apply the transitions between each of the hubs being displayed. This is much simpler than using separate storyboards, because all the logic needed to play the animations is contained in the VisualStateManager. Because of this, there is no need for additional code to integrate the animations into the application.
Adding multi-touch
As an extra touch of whackiness I will show you how you can add multi-touch support to the application. You don’t absolutely need this to make it a great application, but it’s a nice extra to have. At least, that’s my opinion.
Implementing a multi-touch controller
Because multi-touch is optional you don’t want to it break your application when the user doesn’t have a multi-touch device attached to his computer. So instead of removing the navigation buttons and going for a full touch approach, it’s a good idea to add gestures to the application to implement an alternative method of navigation. This way the user can either click the buttons or swipe in a direction to navigate the application.
The key in making multi-touch work is to handle the Touch.FrameReported event. Inside the eventhandler you will get a TouchFrameEventArgs instance that contains information about the touch device and one or more touch points that you can get by either invoking GetPrimaryTouchPoint or the GetTouchPoints method.
The primary touch point is the first finger the user placed on the multi-touch device (Keep in mind, this can also be the mouse or a pen device). The other points are secondary touch points.
With each touch point you will also get an action that was performed. This can either be Down, Up or Moved. The first one you will get is the Down action. This happens when the user places a finger on the touch device. The second action is Move. This action happens even if the user isn’t moving. However, if the user is moving you can detect movement by calculating the distance between the touch point that was tracked with the Down action and the touch point that was tracked with the Move action. The last action is the Up action. This action happens when the user pulls his finger off the touch device.
The combination of touch points and actions can be used to do just about everything with the user interface. The difficulty is to make sense out of the raw information provided.
For navigation you will need to detect whether the user swiped horizontally (left to right or right to left) or vertically (top to bottom or bottom to top). Depending on this movement a navigation command needs to be invoked. All of this is implemented in a custom class called MultiTouchNavigationController. This controller can later be bound to any UI element to track gestures for that element.
To create the multi-touch navigation controller, first you will need to know if the user is actually swiping his finger across the touch device. This is done by tracking the various touch points entered, when there’s only one touch point, the user is probably swiping.
1: /// <summary>
2: /// Handles the FrameReported event for the touch infrastructure
3: /// </summary>
4: /// <param name="sender"></param>
5: /// <param name="e"></param>
6: private void OnTouchFrameReported(object sender, TouchFrameEventArgs e)
7: {
8: try
9: {
10: // Retrieve the primary touch point
11: TouchPoint touchPoint = e.GetPrimaryTouchPoint(_ParentElement);
12: 
13: if (touchPoint.Action == TouchAction.Down)
14: {
15: // Track the new touch point when the user placed a finger on the touch screen.
16: _TouchPoints.Add(touchPoint.TouchDevice.Id, touchPoint);
17: 
18: // Make the new point the primary touch point
19: // if it's the only one on the device.
20: if (_TouchPoints.Count == 1)
21: {
22: _PrimaryTouchPoint = touchPoint;
23: }
24: }
25: else if (touchPoint.Action == TouchAction.Move)
26: {
27: //TODO: Process move action
28: }
29: else if (touchPoint.Action == TouchAction.Up)
30: {
31: // Stop tracking the touch point when the user removed
32: // the finger being tracked from the device
33: _TouchPoints.Remove(touchPoint.TouchDevice.Id);
34: }
35: }
36: catch
37: {
38: // Use this as an extra precaution.
39: // We don't want the application to break when multi-touch is broken for some reason.
40: }
41: }
The next step is to check if there’s a move action. If there is a move action it should still be using one finger. Otherwise the user could be zooming or rotating the view. When the user is actually swiping you can process that action and determine the direction using the following code.
1: /// <summary>
2: /// Handles the FrameReported event for the touch infrastructure
3: /// </summary>
4: /// <param name="sender"></param>
5: /// <param name="e"></param>
6: private void OnTouchFrameReported(object sender, TouchFrameEventArgs e)
7: {
8: try
9: {
10: // Retrieve the primary touch point
11: var touchPoint = e.GetPrimaryTouchPoint(_ParentElement);
12: 
13: if (touchPoint.Action == TouchAction.Down)
14: {
15: // Track the new touch point when the user placed a finger on the touch screen.
16: _TouchPoints.Add(touchPoint.TouchDevice.Id, touchPoint);
17: 
18: // Make the new point the primary touch point
19: // if it's the only one on the device.
20: if (_TouchPoints.Count == 1)
21: {
22: _PrimaryTouchPoint = touchPoint;
23: }
24: }
25: else if (touchPoint.Action == TouchAction.Move)
26: {
27: // Perform some additional logic to locate the previous touch point and the new touch point.
28: Point firstPoint = _TouchPoints[touchPoint.TouchDevice.Id].Position;
29: Point secondPoint = touchPoint.Position;
30: 
31: // Make sure that the gesture is only interpreted as a navigation gesture
32: // when the user used only one finger. Other gestures might include a zoom gesture or rotate gesture.
33: // Also make sure we don't interact with the user when he's interacting with a list or something else
34: // that should be scrollable.
35: if (_TouchPoints.Count == 1 && touchPoint.TouchDevice.DirectlyOver == _ParentElement)
36: {
37: ProcessMoveAction(firstPoint, secondPoint);
38: }
39: }
40: else if (touchPoint.Action == TouchAction.Up)
41: {
42: // Stop tracking the touch point when the user removed
43: // the finger being tracked from the device
44: _TouchPoints.Remove(touchPoint.TouchDevice.Id);
45: }
46: }
47: catch
48: {
49: // Use this as an extra precaution.
50: // We don't want the application to break when multi-touch is broken for some reason.
51: }
52: }
53: 
54: private void ProcessMoveAction(Point firstPoint, Point secondPoint)
55: {
56: // Calculate the delta's in each direction
57: double deltaX = secondPoint.X - firstPoint.X;
58: double deltaY = secondPoint.Y - firstPoint.Y;
59: 
60: // Check that the user is generally moving in a horizontal direction
61: // The movement will be interpreted as being vertical when the user moves diagonally over the screen.
62: if (Math.Abs(deltaX) > Math.Abs(deltaY))
63: {
64: // Check if the user has swiped from left to right
65: // or from right to left.
66: if (deltaX > 0)
67: {
68: TryExecuteCommand(_NavigateRightCommand);
69: }
70: else
71: {
72: TryExecuteCommand(_NavigateLeftCommand);
73: }
74: }
75: else
76: {
77: // Check if the user has swiped top to bottom
78: // or from bottom to top.
79: if (deltaY > 0)
80: {
81: TryExecuteCommand(_NavigateDownCommand);
82: }
83: else
84: {
85: TryExecuteCommand(_NavigateUpCommand);
86: }
87: }
88: }
89: 
90: /// <summary>
91: /// Tries to execute the specified command
92: /// </summary>
93: /// <param name="command"></param>
94: private void TryExecuteCommand(ICommand command)
95: {
96: if (command != null && command.CanExecute(null))
97: {
98: command.Execute(null);
99: }
100: }
Using that direction a specific command is invoked to allow the application to navigate in that direction.
The last step is to bind the MultiTouchNavigationController to the user interface. You can do this by invoking the Bind method on the controller. Each time you navigate away from a specific hub you will need to unbind the controller again using the Unbind method.
1: /// <summary>
2: /// Binds the controller to the user interface.
3: /// This enables user gesture interaction
4: /// </summary>
5: public void Bind()
6: {
7: Touch.FrameReported += OnTouchFrameReported;
8: }
9: 
10: /// <summary>
11: /// Unbinds the controller to the user interface.
12: /// This disables user gesture interaction
13: /// </summary>
14: public void Unbind()
15: {
16: Touch.FrameReported -= OnTouchFrameReported;
17: }
Binding and unbinding the controller is necessary to prevent problems. If you don’t unbind an event the multi-touch controller for one page may still try to interpret the gestures entered by the user. This can give quite a bit of chaos, not to mention the memory leaks you’re causing with this.
Testing multi-touch without multi-touch
You’re probably wondering how I’ve tested the multi-touch gestures. I don’t have a multi-touch monitor and Silverlight doesn’t take my pen tablet for a touch enabled device, so I had to improvise.
There’s a project available on CodePlex called VistaTouch that allows you to run a virtual multi-touch driver. This turns your mice (Yep, you can add more than one) into a simulated multi-touch device. Each of the mouse pointers is a potential touch point that you can interpret as a gesture.
The tutorial to get this driver up and running is kind of extensive, so I’m not going to explain how to configure it here. Instead I suggest you read a blogpost about it made by Micheal Sync. The tutorial itself is written for Windows Phone 7, but you can use it for Silverlight and WPF too. You can find the tutorial here: http://michaelsync.net/2010/04/06/step-by-step-tutorial-installing-multi-touch-simulator-for-silverlight-phone-7
What’s next?
There’s isn’t any conclusion I’d like to jump to here. The sample demonstrates that you can do a lot more with Silverlight than just a basic RIA client. And I hope people will start to think more about how they can make a user interface work for the user.
If you would like to know more about multi-touch or user experience I suggest you check out one of the following links:
http://uxdesign.com/ – More information about user experience design.
http://gallery.expression.microsoft.com/en-us/MultiTouch – Multi-touch behavior for Blend 4.