Saturday 28 April 2012

Surface project - part 23(Rich Media Map - Enhancement)

Spent a day on the enhancement of my rich media map, first version has come out.

1. move import buttons, add image, text, video into pop up panel.


when user click plus button, an additional panel will gracely move from right to left


2. move edit, delete, rearrange button into top operation panel.


3. add color picker to select push pin color


4. add confirmation panel when user click delete item


5. disable text scroll, so the entire length of document will equal to the height of text element


Wednesday 25 April 2012

Surface project - part 22(Rich Media Map - Enhancement)

After showing my major demo application to the supervisors, they want more functionalities and more concise user interface. Thus I need to make a lot of enhancement on the existing one.

  1. Automatically save push pin, so user does not need to click save button.
  2. Automatically save drawing, text, when user tab out.
  3. Remove Main tag functionality.
  4. Add author information on each pin.
  5. Allow user to select push pin color.
  6. Remove scroll ability on text element. So the entire length document will shown in each text element
  7. Move "add drawing", "add video recorder", "add text", "import file" into a pop up panel, so there will be more space on the operation panel.
  8. Move "edit", "delete" button into an individual operation panel on each element.
  9. Move "rearrange" button onto the operation panel of each element.
  10. Fix any collapsible bugs.

Sunday 1 April 2012

Surface project - part 21(Major Demo - Rich Media Map - The Demo - Part 4)

  • Introduction
  • Import Media
  • Create Media
  • Edit Media
  • Layout Mode
  • Tagging System
  • Searching System
  • Deleting Mode

Layout Mode

This application also allows user to remove any media from the push pin, and rearrange the media index as well.


When user go into the layout mode, the appearance of the operation bar will be changed


User can start the rearrange action by put two fingers on the element, which they want to drag.


In the layout mode, a deleting mark will appears on the top right corner in every media element.


Tagging System

Tagging system is fully integrated with search system, so that user can search the push pin after they assign one or more tags onto individual push pin. user can click the button indicated below to open the integrated tag browser.


In the tag mode, user can select one main tag and 5 predefined tags.


After changing the main tag, the appearance of the push pin will be changed correspondingly.


Searching System

With the help of tagging system, the application become searching enabled.


Deleting Mode

The last feature of this application is the deleting mode.


Surface project - part 20 (Major Demo - Rich Media Map - The Demo - Part 3)


  • Introduction
  • Import Media
  • Create Media
  • Edit Media
  • Layout Mode
  • Tagging System
  • Searching System
  • Deleting Mode



  • Create Media

    As being required by supervisors that they want to let user create drawing, text, video inside the application. Thus, I have implemented these functions and make the media inside the push pin in-place editable.

    To creating a drawing, user can click the button indicated below.


    A new drawing panel will be added into the push pin, the integrated drawing panel supports several features, including, color picker, stroke thickness slider, eraser and saving.


    To creating a video from web camera, user can click the button indicated below


    An integrated video recorder will be added into the push pin. After making the video, the video will be automatically imported into the push pin.


    To creating a text media, user can click the button indicated below



    A simple text box will be added into the push pin.


    Edit Media

    To make the application as elegant as possible, I implement the function to edit the media inside the push pin. For image and text media, a in-place editor has been implemented.


    Surface project - part 19 (Major Demo - Rich Media Map - The Demo - Part 2)


  • Introduction
  • Import Media
  • Create Media
  • Edit Media
  • Layout Mode
  • Tagging System
  • Searching System
  • Deleting Mode



  • Import Media

    For user who plays with the interactive map, he would need to import some media through the local file system. The button, which arounded with a red border shows below, will let user import file through integrated file browser.


    The integrated file browser support several features. It can let user preview image, recognise video extension as well as text file.


    Let's see the effect of image preview.


    User can select any supported image file from the file browser, and the browser will automatically import the media into the push pin.



    With the same procedure, user can import video media.


    With the same procedure, user can import text media.

    Surface project - part 18 (Major Demo - Rich Media Map - The Demo - Part 1)

    Ok, finally, I have accomplished the major demo application. And fulfil all the functionalities and user interface requirements. As the features of the application are rich, thus I have to separate this post into several parts.

    • Introduction
    • Import Media
    • Create Media
    • Edit Media
    • Layout Mode
    • Tagging System
    • Searching System
    • Deleting Mode

    Introduction

    When users open the application, they will find a map with Groote Island at the centre. There is a operation bar near the bottom area.


    Each button are the operation bar has its meaning. For left to right, the first button is to clear the screen, when users open too many push pin then they can close all the push pin panel instead of doing it one by one.

    The second button is to save the push pins into a XML file, which persists all the information I need to recreate all the push pin when the application runs again.

    The third button is to go to deleting mode, which disable all the other operation and allow users to remove push pin from the map ( we will see this action in later stage)

    The 4 buttons besides the deleting mode button is for searching purpose. user can search push pin based on 4 predefined tags, which can be assigned to individual push pin (we will see this action in later stage)


    User can long press any point on the map to put a pus pin onto the map.


    After putting a new push pin, user can click the push pin, and the push pin panel will be displayed. For a push pin contains no media, the only visible part is the push pin operation bar. For detailed description for the push pin operation bar, I will do it from next post.



    Saturday 31 March 2012

    Surface project - part 17 (Major Demo - Rich Media Map - Animation)

    It will be interesting to see some animation inside the demo. I have found one place to demonstrate it. When user searches the push pin based on tag information, an animating ellipse will be displayed around matched push pins. WPF provide three animation class, DoubleAnimation, ColorAnimation and PointAnimation.

    To implement the animation on the ellipse is not quite straightforward, as if you attach the color animation on the ellipse to change color of the stroke, then you will receive exception says that color cannot apply to brush.

    The solution is to attach the color animation on the stroke of the ellipse.

    Following code shows how to attach color animation on ellipse's stroke.

    ColorAnimation cAnimation = new ColorAnimation();
                cAnimation.From = Colors.Red;
                cAnimation.To = Colors.Green;
                cAnimation.Duration = new Duration(TimeSpan.FromSeconds(1));
     
                cAnimation.AutoReverse = true;
                cAnimation.RepeatBehavior = RepeatBehavior.Forever;
                string name = GetUniqueName();
                Mediator.TheWindow.RegisterName(name, ellipseborder);
                myStoryboard.Children.Add(cAnimation);
                Storyboard.SetTargetName(cAnimation, name);
                Storyboard.SetTargetProperty(cAnimation, new PropertyPath(SolidColorBrush.ColorProperty));

    Following snaps show the effect of the ellipse around the push pin.



    The color of the stroke of the ellipse is changing from red to green.

    Surface project - part 16 (Major Demo - Rich Media Map - Dynamic Panel Size)

    As I have said in the major demo introduction, there is no existing solution to resize the scatter view item only in one direction. (at least, i haven't found it!)

    To make the demo application as elegant as it could, I decide to programmatically control the size of the push pin panel.

    The width of the push pin panel

    As I use a list box to contain all the element, and it is necessary to display the vertical scroll bar when the content exceeds the scroll viewer. However, if you make the scroll bar visible or auto, the space for the scroll bar will always be displayed no matter if the scroll bar is showing. So I need to change the width of the push pin panel when the scroll bar is unneeded.

    The surface list box contains a scroll view which in term contains the scroll bar. There is a property called ScrollableHeight, which can help us to decide if the scroll bar is necessary. And the LayoutUpdated event will be fired when that property has changed. Through debugging, I have found the path to access the scroll view.

    Grid grid = VisualTreeHelper.GetChild(TheSurfaceListBox, 0) as Grid;
    ScrollViewer scroll = grid.Children[0] as ScrollViewer;

    We can then change the width of the push pin panel so that it will clip the area for the scroll bar when it is not needed.


    The height of the push pin panel

    WPF layout system is complicated, you will encounter a lot of null exception as the width or height on a particaliar control is not set. However, sometimes we cannot set a predefined size as the size will be vary in different situation. I found the min height on each element is suitable for decide the height of the push pin panel which contains all the elements.

    Surface project - part 15 (Major Demo - Rich Media Map - Drag&Drop)

    For this major demo, the supervisors want to let user rearrange the index of the media in the push pin panel. Thus user can drag and drop the media on top of another media so that the position of the media changed. Although drag and drop is not a fancy stuff in today's programming world, there are still some notable aspects which make the implementation in Surface not that easy.

    Problem 1

    As I put the media in a list box, the touch down event would be fired by the list box at first, then the event will route up and down. However, if I implement to capture the touch down event to start a drag action then I will encounter an annoying but unavoidable problem, the user cannot scroll or touch any part inside each media component. There is an solution provided by MSDN, although it is not perfect, but still acceptable. The application will let user to start a drag action when user put at least 2 fingers on the element. After that they can user one finger instead.

    Problem 2

    The supervisors want the dragging element under finger will have the same appearance compare to the original element. As the media components are composited with mangy controls, thus it will cost a lot to duplicate the whole media component. WPF provides RenderTargetBitmap as an solution to achieve the same result by displaying an image which snaps the visual tree.

     public Image DragSourceCopy()
            {
                Size dpi = new Size(96, 96);
                RenderTargetBitmap bmp =
                  new RenderTargetBitmap((int)(this.DesiredSize.Width), (int)(this.DesiredSize.Height),
                    dpi.Width, dpi.Height, PixelFormats.Pbgra32);

                bmp.Render(this);
                JpegBitmapEncoder encoder = new JpegBitmapEncoder();
                encoder.Frames.Add(BitmapFrame.Create(bmp));
                Image image = new Image();
                image.Source = encoder.Frames[0];
                return image;
            }

    By using the above piece of code, you can make a copy image of the whole component, no matter it is a video, image or text media. After tests, the DesiredSize records the correct width and height of the component shown on the screen.

    Following snaps show the drag drop action to rearrange the media in push pin panel



    After drop, the index of the element changed.

    Surface project - part 14 (Major Demo - Rich Media Map - Image Component)

    For supervisors, they want the image component can be editable, which means user can add their own drawing on to existing image, or create a new image instead. There is an sdk example which does the same thing. In the example, user can draw, erase and undo stroke, however, user cannot choose the thickness of the stroke nor save the image. As this application, the media map, is all about interactive with push pin, but not a painting application, thus I remove the undo function and implement the function to choose the stroke's thickness and also the ability to save the drawing.

    Following code shows how to change the thickness of the stroke

         private void StrokeSizeSlider_ValueChange (object sender, RoutedPropertyChangedEventArgs<double> e)
        {
          DrawingPadCanvas.DefaultDrawingAttributes.Width = StrokeSizeSlider.Value;
          DrawingPadCanvas.DefaultDrawingAttributes.Height = StrokeSizeSlider.Value;
        }

    Following snaps show how to create a new image.





    Following snaps show how to draw on an existing image



    Following code shows how to implement saving function.

     private void SaveImage_TouchDown(object sender, TouchEventArgs e)
        {
          string path = "SavedImage.jpg";
          MemoryStream ms = new MemoryStream();
          FileStream fs = new FileStream(path, FileMode.Create);
          RenderTargetBitmap rtb = new RenderTargetBitmap((int)DrawingPadCanvas.Width, (int)DrawingPadCanvas.Height, 96d, 96d, PixelFormats.Default);
          rtb.Render(DrawingPadCanvas);
          JpegBitmapEncoder encoder = new JpegBitmapEncoder();
          encoder.Frames.Add(BitmapFrame.Create(rtb));
          encoder.Save(fs);
          fs.Close();
        }


    After saving the image, it will be stored in local computer

    Friday 30 March 2012

    Surface project - part 13 (Major Demo - Rich Media Map - Introduction)

    A major demo application has been required by the supervisor. As the supervisor is on a two-week trip, thus I will have two weeks to accomplish it. The major demo requires to integrate the demos I built previously for an interactive map, and this time they need a more impressive look and feel. Before he left, we briefly discussed the user case for the application and the user interface as well. Below is the picture scripted by the supervisor.

    As this time, the scale of the application is large, and the requirements are challenged, thus a detailed application specification and a few backbone applications are necessary to build. This post will record the user case and user interface specification.



    Major Demo - Rich Media Map - User case
    • user can place a push pin on to the map
    • user can view the content inside the push pin by click the pin
    • user can import image files into the push pin
    • user can import video files into the push pin
    • user can import text files into the push pin
    • user can add drawing on to the image, which is imported into the push pin
    • user can modify the text information, which is imported into the push pin
    • user can create a drawing and the drawing should be imported into the push pin automatically after finishing drawing.
    • user can create a text information and the text information should be imported into the push pin automatically after finishing typing.
    • user can create a video from web camera and the video should be imported into the push pin automatically after finishing recording.
    • user can remove any media from the push pin.
    • user can assign 5 tags for the push pin, including culture, education, environment, health and other.
    • user can set a main tag for the push pin from culture, education, environment, health and other, the appearance of the push pin will be changed.
    • user can search the push pins by choosing one of the 4 tags (culture, education, environment, health).
    • user can save the push pin as XML file, thus the push pin will be persisted and reusable for other applications.
    • user can delete push pins from the map.
    Major Demo - Rich Media Map - User interface
    • a main panel should displayed when user opens a push pin.
    • there should be a sub panel, which contains all the operation buttons, at the bottom of the main panel.
    • a file browser should be displayed when user chooses to import a media file.
    • the media elements inside the push pin should be layouted as a list.
    • user can rearrange the index of any media element by drag and drop the media inside the main panel.
    • The switch between view and import mode for the push pin should be an in-place switch.
    • The switch between view and drawing mode for the image media should be an in-place switch.
    • The switch between view and typing mode for the text media should be an in-place switch.
    • The switch between view and recording mode for the video media should be an in-place switch.
    Major Demo - Rich Media Map - File browser functionality
    • user can go into a directory.
    • user can go up to parent directory.
    • when user choose to import a media file to the push pin, he can select a media file from the browser.
    • file browser is integrated into the push pin panel.
    Major Demo - Rich Media Map - Image component functionality
    • user can change the view mode into drawing mode.
    • in drawing mode, user can select the color of the brush.
    • in drawing mode, user can select the thickness of the brush.
    • in drawing mode, user can draw on to a canvas by moving fingers.
    • in drawing mode, user can erase the drawing.
    • in drawing mode, user can save the drawing.
    Major Demo - Rich Media Map - Text component functionality
    • user can change the view mode into editing mode.
    • in editing mode, user can modify the content of the text.
    • in editing mode, user can save the modification.
    Major Demo - Rich Media Map - Video component functionality
    • user can play the video.
    • user can pause the video.
    Major Demo - Rich Media Map - Video recorder component functionality
    • user can preview the video stream from the camera.
    • user can start recording the live video.
    • user can finish recording the live video.

    Additional user interface compromise and solution

    compromise: As there is no existing solution to resize the WPF scatter view item in only vertical direction (the default behaviour is scale in both vertically and horizonly), user cannot change the size of the push pin panel.

    solution: To achieve an elegant layout, the height of the push pin panel will be controlled programmatically. There will be a fixed width and a maximum height, so that no empty area will be show when the push pin panel only contains small number of media. A scroll bar will appear when the total height of the content exceeds the panel's maximum height.

    Saturday 24 March 2012

    Surface project - part 12 (demo - Map Media List)

    As the supervisors want to store multiple media file into single push pin, thus I build a powerful demo to demonstrate this capability. The following demo allows the user to add image and video file onto the push pin. And the application treats different media file differently. For image file, users can import text file as the description for the image. See snap shoot below.




    The push pin can store video file as well.



    The following steps demonstrate the complete procedure

    1. long press a point on the map will add a new push pin onto the map.



    2. click the pushpin will pop up an empty panel(if you haven't add any media into it)


    3. Open the custom file browser and select the media file




    4. click the import button and choose a txt file as a description for the image.



    5. User can add a video media file as well, choose the video you want to add.



    6. Save the push pin to an XML file, looks like below.

    <?xml version="1.0"?>
    <MediaPinList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
      <ListOfMediaPin>
        <MediaPin>
          <Longitude>136.470245160833</Longitude>
          <Latitude>-13.954014816082406</Latitude>
          <ListOfElement>
            <PushElement>
              <FilePath>C:\Users\Public\Pictures\Sample Pictures\Koala.jpg</FilePath>
              <Description />
            </PushElement>
            <PushElement>
              <FilePath>C:\Users\Public\Pictures\Sample Pictures\Penguins.jpg</FilePath>
              <Description></Description>
            </PushElement>
            <PushElement>
              <FilePath>C:\Users\Public\Pictures\Sample Pictures\Chrysanthemum.jpg</FilePath>
              <Description />
            </PushElement>
            <PushElement>
              <FilePath>C:\Users\Public\Videos\Sample Videos\Wildlife.wmv</FilePath>
              <Description />
            </PushElement>
            <PushElement>
              <FilePath>C:\Users\Public\Pictures\Sample Pictures\Lighthouse.jpg</FilePath>
              <Description />
            </PushElement>
          </ListOfElement>
        </MediaPin>
        <MediaPin>
          <Longitude>136.59384822539985</Longitude>
          <Latitude>-13.992668132658892</Latitude>
          <ListOfElement>
            <PushElement>
              <FilePath>C:\Users\Public\Pictures\Sample Pictures\Koala.jpg</FilePath>
              <Description></Description>
            </PushElement>
            <PushElement>
              <FilePath>C:\Users\Public\Videos\Sample Videos\Wildlife.wmv</FilePath>
              <Description />
            </PushElement>
          </ListOfElement>
        </MediaPin>
      </ListOfMediaPin>
    </MediaPinList>

    Surface project - part 11(C# Object to XML and vice versa)

    The task in this week is to extend the custom media tag browser here to add multiple media file into one push pin. The idea is fairly easy, as the surface list box is general enough to composite any framework element as an item, the remain problem is how to persist the pushpin into local file. The supersivors preferred to store the pushpin into XML file. After look into the MSDN, I found that .net already provides a very flexible and powerful mechanism to serialize object into XML file and vice versa.

    As the object, which represents the push pin, only contains string and double fields. To mark the object as [Serializable] is enough.

    Following examples demonstrate the basic coding. The full example can be found here.

    How to serialize object to XML

    string path = "MySettings.xml";
    XmlSerializer x = new XmlSerializer(settings.GetType());
    StreamWriter writer = new StreamWriter(path);
    x.Serialize(writer, settings);

    How to deserialize XML back to object

    MySettings settings = new MySettings();
    string path = "MySettings.xml";
    XmlSerializer x = new XmlSerializer(typeof(MySettings));
    StreamReader reader = new StreamReader(path);
    settings = (TVSettings)x.Deserialize(reader);

    Saturday 17 March 2012

    Surface project - part 10 (demo - mobile device interaction)

    User case:

    A mobile device explorer, user can explore and download images, which are stored on mobile device, through Surface screen. A case study is here, the conclusion is to use wifi instead of using bluetooth to establish the communication.

    Solution:

    This application is consisted with a master application on Surface and a slave application on my android device. The slave application is built using adobe air (Flex mobile project), so that it can be installed on android, ios, blackberry os and many other mobile systems

    The application is using basic socket technology, the protocol for encode and decode data is self-made.


    User can explore the file in mobile device through a custom file browser




    When a file type is available to download, a download button will display beside the file



    When a file is downloading, a progressbar will be displayed on the bottom of the browser



    The downloaded file will be displayed in a scatter view