Integrating BITS with Amazon S3 (Part 1)

August 10, 2014 1 comment

Recently, I was working on a group project where we had a simple requirement to download prerequisites for the application we were building. That is, the application needed and expected certain components to be installed on the system (i.e. SQL Server Express, Crystal Reports, device drivers, etc.). After some research, it was decided to leverage the Background Intelligent Transfer Service (BITS) that is built into the Windows machine. BITS allows you to transfer files between client and server and it has built-in features like transfer throttling based on available bandwidth and persistence of transfer state even if network disconnect occurs. We also decided to use Amazon S3 in order to store our files to be downloaded as this has great reliability and scalibility. Below is a diagram of the different components.


The application we were bulding is a WPF application written in C#. We tried to leverage as many libraries available in order to expedite development and so for communicating with BITS we leveraged SharpBITS, a .NET wrapper of the BITS API. Using SharpBITS is outside the scope of this post but if you’d like more information on using this library, there is a nice post that is very educational here. So, using public accessible urls to the files stored in Amazon S3, things worked beautifully. BITS is a workhorse and it did its job well until we decided to lock down those resources and added security into the mix. The following describes hurdles and obstacles that were encountered in the process of integrating these two components and the solution that was built in order to address certain limitations that will become apparent as we delve into intricacies of Amazon S3 and BITS.

The Problem
A great way to lock down files in Amazon S3 and to access them is to generate presigned urls. The urls are only valid for a time period and the credentials are embedded into the signature process and sent along via the query string parameters in the url. Here’s a sample presigned url:

The process of signing and generating these urls is rather involved but one of the key ingredients involves including the HTTP verb (i.e. GET, POST) in the url signature process. So a request for a file named test.txt would generate 2 completely different signatures for a GET and for a HEAD request, for example. And this little detail is the root cause of all evil especially when using BITS to download files. Here’s why.
Whenever a download is scheduled for a file url with BITS, it first makes a HEAD request in order to get information about the file like the file size. It then makes one or more GET requests in order to download the file itself. However, we can only give BITS a single url for a file to be downloaded so which do you give to BITS? Regardless of which one you use, you will succeed for one but get a 403 access denied response for the other.

In part 2 of this post, we will look at the solution to this problem and how we can circumnavigate the way BITS works by basically being a middle man in the communications process.

Categories: General

Integrating BITS with Amazon S3 (Part 2)

August 9, 2014 1 comment

In part 1 of this post series we talked about the pain points of integrating BITS with Amazon S3. In this part of the series we will look at how to overcome the issue described in part 1.

The Solution

So basically, at this point, we cannot change the way BITS works obviously. Also, we cannot change the way Amazon S3 works either when security is added to resources. The one thing we can do, however, is be a middle man in the process. In other words, instead of downloading a file from Amazon S3, why can’t we download it from another service (i.e. a proxy service that acts on behalf of Amazon S3 for us) that knows how to deal with these intricacies? The following illustrates all the components taking the proxy service into account.


So now, instead of giving a bits a file url that points to S3, we give a url that points to our proxy service.


This is the approach we took and it worked out really well. You still have to take care of generating both a presigned url for a HEAD and a GET request but when BITS makes either request to our proxy service, we can respond appropriately.

  • When BITS make a HEAD request to the proxy service, we make a GET request to Amazon S3 and return the obtained response back to BITS.
  • When a GET request comes in, we just do a redirect (302) to the actual Amazon S3 presigned url for a GET and it will respond to BITS directly.

Initially, we hosted our proxy service within the process of the client application. This presented some drawbacks like what happens if the user decided to kill the client application while we were downloading? BITS would encounter a transient error and continue to try to reestablish communications but it tries to do this for a limited amount of time only. What if the application was down for a longer period of time? We decided to host the proxy service in a Windows Service application on the local machine. That way, it was always up and it would automatically start on system reboot.

I hope you found this post helpful. When we encountered this issue, we searched for a solution and there was very little information that was available except for a post here or there that eventually led us in the right direction. Although making a basic wcf service that implements this functionality is rather simple, I will be posting the code for the proxy service in the near future so please check back if not already posted.

Categories: General

Creating gradients in Microsoft Paint.

February 11, 2011 Leave a comment

There are times when you need to create a gradient in order to fill the background of some div or other page element. For some people with Photoshop skills, this can be a relatively quick and simple task to do. However, some of us out there are not graphic designers nor typically have the time to learn the ins and outs of a fancy program like Photoshop or other tool. In this post I’m gonna show you how you can create a nice looking gradient that you can build in Microsoft paint in less than 1 minute.So here we go.

Let’s say you need a simple gradient (for a navigation menu or a div, button, etc.) that is 25 pixels in height.

1. Open up Paint and resize the default blank image to 100 pixels by 25 pixels as so.

2. Zoom in to make your drawing surface easier to see, select your first gradient color, followed by the line tool. You are going to draw a diagonal straight line from the bottom left corner to the top right corner( you can hold down [Ctrl] as you draw to ensure the line is straight).

3. Select the paint bucket and fill the bottom( or top) side of the line. Just make sure to only fill one side.

4. Resize the image by skewing the horizontal down to 1% only.

You will be left with a 1 pixel wide sliver of gradient that is 25 pixels in height. You can then use this to style an element’s background with CSS like so.

background-image: url('../coolGrad.png');
background-repeat: repeat-x;
height: 25px;

The following image shows a div with the class above applied to it.

Obviously you can play around with the colors and height/width of the image in Paint to achieve unique results. But, in a nutshell, that’s it folks!

Categories: General

Creating a Custom Silverlight Task Dashboard Control (Part 2)

December 15, 2010 1 comment

In part 1 we looked at adding some basic properties as well as creating the layout and structure for a custom Silverlight task dashboard control. In this post we will continue that discussion and we’ll look at how the wiring of the elements defined in the template get wired up. At this point we’ve created a custom control template with support for custom properties; however, there is no plumbing in order to make it come to life.
In order to do this, we need the help of a specific method defined in the base Control class, OnApplyTemplate(). This method gets called when the control is initialized and it allows you to find elements defined in the template. What you do at this point is specific to the control you are building and the logic you intend to have. For example, if the template contains a button that needs to execute some code when clicked, this is the spot where you get a reference to that button and wire up a click event handler or change its properties. You may not need to do anything except just get a reference to the element( so that you can access it later on and do something with it). The bottom line is, because there is no implicit wiring of an element defined in the xaml to its reference in the code behind class, you have to explicitly do it yourself and this is where you do it. Here’s the implementation for this control.

 public override void OnApplyTemplate()

      rootElement = base.GetTemplateChild("RootElement") as Grid;
      if (rootElement != null)
          // Launch storyboards
          Storyboard completedSB = (Storyboard)base.GetTemplateChild("CompletedStoryboard") as Storyboard;

          Storyboard inProgressSB = (Storyboard)base.GetTemplateChild("InProgressStoryboard") as Storyboard;

          Storyboard notStartedSB = (Storyboard)base.GetTemplateChild("NotStartedStoryboard") as Storyboard;

    // Set the tooltips for the progress bars
    completedPB = base.GetTemplateChild("ProgressBarCompleted") as ProgressBar;
    ToolTipService.SetToolTip(completedPB, CompletedProgressValue + "%");

    inProgressPB = base.GetTemplateChild("ProgressBarInProgress") as ProgressBar;
    ToolTipService.SetToolTip(inProgressPB, InProgressProgressValue + "%");

    notStartedPB = base.GetTemplateChild("ProgressBarNotStarted") as ProgressBar;
    ToolTipService.SetToolTip(notStartedPB, NotStartedProgressValue + "%");

    // Set percent TextBlock
    TextBlock TextBlockCompletedPercent = base.GetTemplateChild("TextBlockCompletedPercent") as TextBlock;
    SetPercentageText(TextBlockCompletedPercent, CompletedProgressValue);

    TextBlock TextBlockInProgressPercent = base.GetTemplateChild("TextBlockInProgressPercent") as TextBlock;
    SetPercentageText(TextBlockInProgressPercent, InProgressProgressValue);

    TextBlock TextBlockNotStartedPercent = base.GetTemplateChild("TextBlockNotStartedPercent") as TextBlock;
    SetPercentageText(TextBlockNotStartedPercent, NotStartedProgressValue);

    // Set Click handlers for labels
    TextBlock TextBlockCompleted = base.GetTemplateChild("TextBlockCompleted") as TextBlock;

    TextBlock TextBlockInProgress = base.GetTemplateChild("TextBlockInProgress") as TextBlock;

    TextBlock TextBlockNotStarted = base.GetTemplateChild("TextBlockNotStarted") as TextBlock;

As you can see, this code basically overrides the OnApplyTemplate() method and pulls out elements from the template by calling the GetTemplateChild() method. It is important to note that I had declared private variables for each of the elements I wanted to work with in code and here is where I instantiate them. At this point, I can call methods, set properties, set event handlers, etc. Inspecting the code will reveal that this is where I start the animations for the progress bars and also where I set click handlers for some textblocks. Feel free to download the source code and as always, you can contact me if you have any questions.


Download Source Code

Creating a Custom Silverlight Task Dashboard Control (Part 1)

September 11, 2010 1 comment

Recently I was tasked with working on a Silverlight application where we were pulling data from SharePoint. One of the things we were pulling was tasks from a SharePoint tasks list. I wanted to display a simple dashboard giving a breakdown of tasks by status (i.e. In Progress, Not Started, Completed, etc.) with visual progress bar indicators of the percentage for each like shown below:

I got the idea from a white paper I read once about creating dashboard web parts using XSLT in SharePoint Designer a while back. Anyhow, I digress, but basically I needed to create a custom control that I can plug into any Silverlight app, set some properties, and viola! Since this was my first attempt at creating a custom Silverlight control, I thought why not write a blog about it.

I went ahead and started by creating a Silverlight control library in Visual Studio. I added a class which I called TaskDashboard and derived it from the base Control class. At this point all I have is this:

public class TaskDashboard: Control


Now, I need to expose properties so that the control user can set the values for each of the task statuses.  Although there are 5 different statuses for a SharePoint task, I only care about 3; Completed, In Progress, and Not Started. To expose these, I  created 3 dependency properties. Now, they don’t have to be dependency properties but, they come in handy when you want to add things like data binding, styles,  and animations (as you’ll see later). And, since most of the properties exposed by Silverlight elements are dependency properties, why go against the current. This is what that looks like:

public static readonly DependencyProperty CompletedProgressValueProperty = DependencyProperty.Register("CompletedProgressValue", typeof(double), typeof(TaskDashboard), null);
public double CompletedProgressValue
    get { return (double)base.GetValue(CompletedProgressValueProperty); }
    set { base.SetValue(CompletedProgressValueProperty, value); }
public static readonly DependencyProperty InProgressProgressValueProperty = DependencyProperty.Register("InProgressProgressValue", typeof(double), typeof(TaskDashboard), null);
public double InProgressProgressValue
    get { return (double)base.GetValue(InProgressProgressValueProperty); }
    set { base.SetValue(InProgressProgressValueProperty, value); }
public static readonly DependencyProperty NotStartedProgressValueProperty = DependencyProperty.Register("NotStartedProgressValue", typeof(double), typeof(TaskDashboard), null);
public double NotStartedProgressValue
    get { return (double)base.GetValue(NotStartedProgressValueProperty); }
    set { base.SetValue(NotStartedProgressValueProperty, value); }

At this point I need to create the default control template that will define how the control will look. For ASP.NET developers, this is similar to defining your UI markup in an .ascx file when creating a custom web user control. However, unlike ASP.NET, the wiring is not implicit and there are certain rules to follow:

  1. The control template definition must be placed in a file named generic.xaml. If you’re creating more than one control, all of their templates must be defined there.
  2. Also, the generic.xaml file itself must reside in a folder named Themes; this is because Silverlight is very closely related with WPF and that’s the way the cookie crumbles for lack of a better term.

In generic.xaml, you need to have a resource dictionary with a style for each control in the control library, if you’re only working with one control, just declare one style. In the style, you set the Template property for your control as follows:

    <Style TargetType="local:TaskDashboard">
        <Setter Property="Template">
                <ControlTemplate TargetType="local:TaskDashboard">
                    <!-- Define the template by adding elements here -->                  

Notice the mapping of the namespace prefix which has the namespace and the assembly name. Also, notice the TargetType property of the Style tag; it specifies the namespace prefix which you use to bring in your assembly (“local” in my case), colon, and the class name for the control. One other part that’s needed is to set the control’s DefaultStyleKey property in the constructor as follows:

// Constructor
public TaskDashboard()
 DefaultStyleKey = typeof(TaskDashboard);

So now that I have my basic pieces, I need to define what goes inside the ControlTemplate; this is what gives the control the basic layout and structure. In this case, it consists of a Border with a Grid that contains the TextBlocks and ProgressBars that I need in order to make this work.

Defining the Layout

The layout consists of a Border which has a Grid. The Grid in turn contains all of the child elements such as TextBlocks and ProgressBars that I need to display the content of the custom control.

<Border BorderBrush="{TemplateBinding BorderBrush}" 
      BorderThickness="TemplateBinding BorderThickness}"
      Background="{TemplateBinding Background}"
      CornerRadius="{TemplateBinding CornerRadius}"
      Width="TemplateBinding Width}">
    <Grid x:Name="RootElement">
            <Storyboard x:Name="CompletedStoryboard">
                <DoubleAnimation Storyboard.TargetName="ProgressBarCompleted" 
                            Storyboard.TargetProperty="Value" Duration="0:0:1"  
                            From="0" To="{TemplateBinding CompletedProgressValue}">
            <Storyboard x:Name="InProgressStoryboard">
                <DoubleAnimation Storyboard.TargetName="ProgressBarInProgress" 
                            Storyboard.TargetProperty="Value" Duration="0:0:1" 
                            From="0" To="{TemplateBinding InProgressProgressValue}">
            <Storyboard x:Name="NotStartedStoryboard">
                <DoubleAnimation Storyboard.TargetName="ProgressBarNotStarted"
                            Storyboard.TargetProperty="Value" Duration="0:0:1"   
                            From="0" To="{TemplateBinding NotStartedProgressValue}">
            <RowDefinition Height="Auto"></RowDefinition>
            <RowDefinition Height="Auto"></RowDefinition>
            <RowDefinition Height="Auto"></RowDefinition>
            <RowDefinition Height="Auto"></RowDefinition>
            <RowDefinition Height="Auto"></RowDefinition>
            <RowDefinition Height="20"></RowDefinition>
            <ColumnDefinition Width="Auto"></ColumnDefinition>
            <ColumnDefinition Width="Auto"></ColumnDefinition>
            <ColumnDefinition Width="Auto"></ColumnDefinition>
        <!-- Child controls -->

I used the TemplateBinding expressions in order to set some of the properties of the Border. You’re probably wondering where do these properties such as BorderBrush, Background, and Width come from. Recall that the control’s class inherits from the base Control class and that’s where they’re either defined or inherited from a higher level class like FrameworkElement. One thing to note though; the base Control class does not have a property for CornerRadius, in this case, I just created a dependency property in the TaskDashboard class and I bind it here to the Border’s CornerRadius property. This gives me the nice rounded effect around the control.
I also wanted to give this control a simple animation so that when the control loads, the ProgressBars start filling up with their assigned values. To achieve this, I simply added Storyboards as resources under the Grid, one for each ProgressBar. Nothing fancy here,  I’m just animating the Value property of the respective ProgressBar for 1 second from 0 to whatever value the control user sets for each of the task status dependency properties defined earlier.

The Child Controls
These are the elements that make up the control’s default template. You may need to access these elements in order to set properties, handlers, states or whatever but in order to do that, you must first wire these up so that they can be accessed from the control’s code behind(see next section).

<TextBlock x:Name="TextBlockTitle" Grid.Row="0" Grid.ColumnSpan="3" Margin="10,10,10,0"
       Text="{TemplateBinding Title}" Foreground="{TemplateBinding TextLabelForeground}"
       FontWeight="Bold" FontSize="12" />
<Border Grid.Row="1" Grid.ColumnSpan="3" BorderBrush="{TemplateBinding BorderBrush}"
   BorderThickness="0.5"  Width="{TemplateBinding TitleUnderlineWidth}"  
   Margin="10,0" />                               
<!-- Completed Tasks -->
<TextBlock x:Name="TextBlockCompleted" Tag="Completed" Grid.Row="2" Grid.Column="0"
       Margin="10,10" Text="Completed:" FontWeight="Bold"
       Foreground="{TemplateBinding TextLabelForeground}" />
<ProgressBar x:Name="ProgressBarCompleted" Margin="10,10"
         Width="{TemplateBinding ProgressBarWidth}" Height="20" HorizontalAlignment="Stretch"
         IsEnabled="True" Grid.Row="2" Grid.Column="1"
         Background="{TemplateBinding CompletedProgressBackground}"
         Foreground="{TemplateBinding CompletedProgressForeground}" Maximum="100" Minimum="0"
         Value="{TemplateBinding CompletedProgressValue}"  />
<TextBlock x:Name="TextBlockCompletedPercent" Margin="10,10" Grid.Row="2" Grid.Column="2"
       Text="{TemplateBinding CompletedProgressValue}" FontWeight="Bold"
       Foreground="{TemplateBinding TextLabelForeground}" />
<!-- In Progress Tasks -->
<TextBlock x:Name="TextBlockInProgress" Tag="InProgress" Grid.Row="3" Grid.Column="0"
       Margin="10,10" Text="In Progress:" FontWeight="Bold"
       Foreground="{TemplateBinding TextLabelForeground}" />
<ProgressBar x:Name="ProgressBarInProgress" Margin="10,10"
        Width="{TemplateBinding ProgressBarWidth}" Height="20" IsEnabled="True" Grid.Row="3"
        Grid.Column="1" Background="{TemplateBinding InProgressProgressBackground}"
        Foreground="{TemplateBinding InProgressProgressForeground}" Maximum="100"
        Minimum="0" Value="{TemplateBinding InProgressProgressValue}"  />
<TextBlock x:Name="TextBlockInProgressPercent" Margin="10,10" Grid.Row="3" Grid.Column="2" Text=""
       FontWeight="Bold" Foreground="{TemplateBinding TextLabelForeground}" />
<!-- Not Started Tasks -->
<TextBlock x:Name="TextBlockNotStarted" Tag="NotStarted" Grid.Row="4" Grid.Column="0"
       Margin="10,10" Text="Not Started:" FontWeight="Bold"
       Foreground="{TemplateBinding TextLabelForeground}" />
<ProgressBar x:Name="ProgressBarNotStarted" Margin="10,10"
        Width="{TemplateBinding ProgressBarWidth}" Height="20" IsEnabled="True" Grid.Row="4"
        Grid.Column="1" Background="{TemplateBinding NotStartedProgressBackground}"
        Foreground="{TemplateBinding NotStartedProgressForeground}"
        Maximum="100" Minimum="0" Value="{TemplateBinding NotStartedProgressValue}"  />
<TextBlock x:Name="TextBlockNotStartedPercent" Margin="10,10" Grid.Row="4" Grid.Column="2" Text=""
       FontWeight="Bold" Foreground="{TemplateBinding TextLabelForeground}" />

This is it for now. In (Part 2) of this post I will go into detail of how the wiring of these elements get done with the control’s code behind. If anyone has any questions, feel free to email me or post any comments.

Go to Part 2