Wednesday, June 25, 2008

Running Batch Jobs In An ASP.NET Web Application Using Application State

Download Example Files:

Sample Run Job Web Application

 

Recently while doing work for a client, I came across the need to run a job (batch) within the application while giving the user that ability to monitor the progress of the job and also being able to track how much time was remaining until the completion of the job.

Along with the aforementioned specs, the requirements were as follows:  they needed a web page that they could go to in order to start the job, they needed the website to be able to alert them if another job was being run, they needed to be able to cancel the job at any time.

I assume that this is a common need for businesses and would like to share a simple example that illustrates how one might go about solving such a problem.

image

How I Did It

Since I didn't want to create a database or something similar to handle such a task due to the overhead that it takes to create a database, I decided to store information about my 'job' in the Application State object (which is essentially a HashTable) due to the fact that Application State object could be accessed and modified across the entire application.  This way, anyone, regardless of who started a job, would be able to access the information regarding the job.

The meat of my solution lies in two things: The BatchRun class that I created (Figure 1), and the use of a special property that accesses the Applicaiton State object within my page (Figure 2).

 


 

    [Serializable]

    public class BatchRun

    {

        #region Constructors

 

        public BatchRun()

        {

        }

 

        public BatchRun(int totalNumberOfItems)

        {

            TotalNumberOfItems = totalNumberOfItems;

        }

 

        #endregion

 

        #region Properties

 

        public DateTime? LastUpdatedTime { get; set; }

        public DateTime? StartTime { get; set; }

        public int TotalNumberOfItems { get; set; }

        public int ItemsCompleted { get; private set; }

        public bool ShouldStop { get; set; }

 

        public bool HasNotBegun

        {

            get

            {

                return StartTime == null;

            }

        }

 

        public bool IsCompletedOrExpired

        {

            get

            {

                return PercentDone == 100 || ShouldStop || (LastUpdatedTime != null && LastUpdatedTime.Value < DateTime.Now.AddMinutes(-Settings.Default.StalledMinuteWait));

            }

        }

 

        public int PercentDone

        {

            get

            {

                if (TotalNumberOfItems == 0)

                    return 0;

                return (int)(100 * ((double)ItemsCompleted / (double)TotalNumberOfItems));

            }

        }

 

        private TimeSpan? TotalTime

        {

            get

            {

                if (StartTime == null)

                    return null;

                return DateTime.Now - StartTime.Value;

            }

        }

 

        public TimeSpan EstimatedTimeRemaining

        {

            get

            {

                if (ItemsCompleted == 0 || TotalTime == null)

                    return default(TimeSpan);

                return TimeSpan.FromSeconds(

                (int)((TotalTime.Value.TotalSeconds / ItemsCompleted) * (TotalNumberOfItems - ItemsCompleted)));

            }

        }

        #endregion

 

        #region Public Methods

        public void Start()

        {

            if (TotalNumberOfItems == 0)

            {

                throw new ArgumentException("Total Number of Items not set!", "TotalNumberOfItems");

            }

            StartTime = DateTime.Now;

            LastUpdatedTime = DateTime.Now;

        }

 

        public bool IncrementItemsCompleted()

        {

            if (ItemsCompleted < TotalNumberOfItems)

            {

                LastUpdatedTime = DateTime.Now;

                return (++ItemsCompleted == TotalNumberOfItems);

            }

            return true;

        }

        #endregion

    }

Figure 1

 

 


        protected BatchRun CurrentBatchRun

        {

            get { return (BatchRun)this.Application["CurrentBatchRun"]; }

            set { this.Application["CurrentBatchRun"] = value; }

        }

Figure 2

 

The BatchRun class keeps all of the information that you need to derive what percent of the job is complete as well as how much time is left.  As you can see, I've included some intelligent properties within the BatchRun class to make these calculations easier and uniform.

The special application state property allows you to just access and modify the information about the job without having to be thinking about where you are storing this data.

The rest of the solution is just hooking into this information to let the user know what the status of the job is.

Here's the entire code behind file for the batch run example web page (please note that not everything I've done here is optimal, but is merely to give an example of how one would accomplish such a task):

 


using System;

using System.Threading;

 

namespace RunJobWebsite

{

    public partial class _Default : System.Web.UI.Page

    {

        protected BatchRun CurrentBatchRun

        {

            get { return (BatchRun)this.Application["CurrentBatchRun"]; }

            set { this.Application["CurrentBatchRun"] = value; }

        }

 

        protected void Page_Load(object sender, EventArgs e)

        {

            if (!IsPostBack)

            {

                this.txtStartDate.Text = DateTime.Today.AddMonths(-1).ToShortDateString();

                this.txtEndDate.Text = DateTime.Today.ToShortDateString();

            }

        }

 

        protected void btnRunBatch_Click(object sender, EventArgs e)

        {

            SetBatchParameterTableVisibility(false);

 

            if (!CurrentBatchInProgress())

            {

                this.lblPercentage.Text = "0";

                this.lblTimeRemaining.Text = "Unknown";

                CurrentBatchRun = new BatchRun();

                var ts = new ThreadStart(RunBatch);

                var thread = new Thread(ts);

                thread.Start();

            }

            else

            {

                ShowAlert("Statement Batch Already In Progress");

            }

            timerBatchRun.Enabled = true;

        }

 

        private void RunBatch()

        {

            DateTime startDate = DateTime.Parse(this.txtStartDate.Text);

            DateTime endDate = DateTime.Parse(this.txtEndDate.Text);

 

            var tempDate = startDate;

            int i = 0;

            while (tempDate < endDate)

            {

                i++;

                tempDate = tempDate.AddHours(1);

            }

 

            CurrentBatchRun.TotalNumberOfItems = i;

            CurrentBatchRun.Start();

 

            tempDate = startDate;

            while (tempDate < endDate)

            {

                DoSomething(new Random().Next(1, 5));

                tempDate = tempDate.AddHours(1);

                if (CurrentBatchRun == null || CurrentBatchRun.ShouldStop)

                {

                    break;

                }

                CurrentBatchRun.IncrementItemsCompleted();

            }

        }

 

        private void DoSomething(int seconds)

        {

            Thread.Sleep(seconds * 1000);

        }

 

        private void SetBatchParameterTableVisibility(bool visible)

        {

            this.tblBatchParameters.Visible = visible;

            this.tblBatchProgress.Visible = !visible;

        }

 

        private bool CurrentBatchInProgress()

        {

            var batchRun = CurrentBatchRun;

            if (batchRun == null || batchRun.HasNotBegun)

            {

                return false;

            }

            return !batchRun.IsCompletedOrExpired;

        }

 

        protected void timerBatchRun_Tick(object sender, EventArgs e)

        {

            BatchRun currentBatch = CurrentBatchRun;

            if (currentBatch != null && currentBatch.HasNotBegun)

            {

                return;

            }

            if (currentBatch == null)

            {

                SetBatchParameterTableVisibility(true);

                return;

            }

            if (currentBatch.IsCompletedOrExpired)

            {

                if (currentBatch.ShouldStop)

                {

                    ShowAlert(String.Format("Statement Run Completed at {0}, but was cancelled.", currentBatch.LastUpdatedTime));

                }

                else

                {

                    ShowAlert(String.Format("Statement Run Completed at {0}.", currentBatch.LastUpdatedTime));

                }

                timerBatchRun.Enabled = false;

                SetBatchParameterTableVisibility(true);

            }

            else

            {

                this.lblPercentage.Text = currentBatch.PercentDone.ToString();

                this.lblTimeRemaining.Text = String.Format("{0:f2}", currentBatch.EstimatedTimeRemaining.TotalMinutes);

            }

        }

 

        private void ShowAlert(string alert)

        {

            Response.Write("<script>alert('" + alert + "')</script>");

        }

        protected void btnStop_Click(object sender, EventArgs e)

        {

            ShowAlert("Batch Run Stopped");

            if (CurrentBatchRun != null)

            {

                CurrentBatchRun.ShouldStop = true;

                this.timerBatchRun.Enabled = false;

                SetBatchParameterTableVisibility(true);

            }

        }

 

 

    }

}

 

Download Example Files:

Sample Run Job Web Application

Thursday, June 19, 2008

Hacking The Zune Podcast Feature To Give You Bookmarks For Your Audio Books

I got a Zune (www.zune.net) about a year and a half ago and have been quite happy with it. One of the major features that was lacking was the ability to save your place if you were listening to an audio book. I spend a lot of my time listening to self-help audio books and religious audio books. In order to keep track of my place in the audio book I realized I had two choices: just listen to one audio book and nothing else since the Zune remembers your last listening location, or have my audio books have many small tracks and try to remember the track that I was on.

As you can imagine, both of these methods have serious follies. I looked online to see if there was some sort of hack that could provide some sort of bookmark functionality, but didn't find anything of value. I did, however, stumble upon a forum post where someone suggested that you put your audio files into the Podcast section of your Zune. I realized that the Zune keeps track of your last listening location in each of your Podcast episodes. I immediately tried to drag files into the Podcast section of the Zune software, but it didn't take.

After a lot of investigation and trial and error, I found that marking the mp3 as an "ITUNESPODCAST" (iTunes podcast), if you drag it into your Zune software under music, it would immediately recognize it as a Podcast.

Here's how I did it:

First, download Mp3Tag (http://www.mp3tag.de/en/download.html), which is software you are going to want regardless. Mp3Tag makes it SOOO much easier to tag your mp3s with proper information about the album, artist, song, etc.


Second, you're going to want to change a couple settings to make Mp3Tag a little easier to use.

Set Mp3Tag so it doesn't annoy you every time you save mp3 tag information:

Under Tools-> Options-> Messages

image

Now make it so Mp3Tag will auto-save everything you change so you don't have to manually save every time you make a change:

Under Tools-> Options-> Tags

image


Third, open the mp3 you will want to add by either dragging it into Mp3Tag or using the open dialog.


Fourth, add the ITUNESPODCAST=1 extended tag to the mp3 by right-clicking on the mp3 in Mp3Tag and click on "Extended Tags..."

image

image

You can also set up your other tag info (you will mainly be concerned about the Title and Album information) for your Podcast file.


Fifth, just go to the 'Music' section of your Zune software and drag your quasi Podcast file into your Zune software, or put it where Zune will pick it up, and you now have a Podcast file that will save your place when you listen to it!


Merging / Joining / Combining Mp3s

I am a big audio book person and have downloaded and ripped a lot of audio books that were in many small audio files. I prefer having just one mp3 per audio book to keep my music library clean.

After trying out a few "try before you buy" programs that help you merge audio files, I stumbled upon MergeMp3, a free application that Aleksandr Gekht, a very nice person, wrote and wanted to let everyone use for free.

Check it out:

http://www.komkon.org/~shchuka/software/mergemp3/#download 

Tuesday, June 17, 2008

Using External Config Files For .NET Applications

A co-worker of mine came across a blog post about using external configuration files: http://blog.andreloker.de/post/2008/06/Keep-your-config-clean-with-external-config-files.aspx.

I realized that this can be incredibly useful, as we have had issues with config files at my current job.  Currently, we are managing four different configuration files: one for development, integration, staging and production.  Each of these config files have almost the exact same content, but differ in just a few ways.  I anticipate just having one main config file, but keeping a set of four external config files for those pieces of the configuration files that change.  We'll see how it goes.

I did a little experiment using these external config files on a "Cleanup" console app that I was working on.  I followed the example of the mentioned blog post, but ran into a couple blocks.

One error I got stated that 'The attribute 'configSource' cannot be specified because its name starts with the reserved prefix 'config'.' This happened because I was trying to specify a configSource for the System.ServiceModel section of my app.config.  I was trying to do this because of the fact that the URL's to WCF services in the client section might change.  It turns out that you can only specify a configSource for the child elements of the System.ServiceModel section.  So I changed the section that had a configSource to the 'client' element, and it fixed it.

Once I got that fixed, I ran into another issue stating 'Unable to open configSource file <Config File Name>.', which I had obviously created.  I couldn't figure out why this was happening, but then I realized that the application, due to the fact that it was an executable, was looking in the BIN/Debug folder, not the root of my application.  To remedy this, I made the custom .config files so they would 'Copy Always,' and then my app worked!  You can refer to the screen shot below:

image

Thursday, June 5, 2008

Piece By Piece Query Building With Linq

In numerous projects in the past, I've found myself needing to write code and build sql queries that filtered results based on one or more criteria and any combination of criteria. I've also needed to write code that returns extra parts of an entity (typically involving the joining of tables, but only under certain conditions.

In this post, I'm assuming that you know at least the very basics about Linq - how to create a DataContext class from a the Linq to SQL designer, select from it's tables and filter the results.

In the past, I have done a few things to solve this type of problem. I would create queries for each scenario, which resulted in a maintenance headache due to the numerous code changes every time a small change was introduced, and of course, more places to mess up. I also tried formulating my stored procedures to return different result sets depending on boolean parameters passed in. This, of course proved to be non-intuitive and error prone due to the variation in the type of data that was coming back. As a result, I felt I always had to choose between a lot of code, or non-intuitive code.

Linq to SQL - Building Queries Piece By Piece

Thanks to expression building capabilities of Linq, we can now build our queries one piece at a time without having to do tricky dynamic sql building while avoiding the caveats of the aforementioned approaches.

An example of a case when you would want to do this sort of thing would be (using an annoying Northwind database example) if you wanted to search for Orders based on any combination of criteria that you allow (e.g. searching by OrderDate, Freight, ShipCountry etc.) and possibly populating the order details of each order.

Traditionally, this would prove to be a bit of a challenge due to the number of queries or complexity of the query you would write to accomplish this, not to mention all of the CRUD code you would have to write. Thanks to Linq, we can now accomplish this in a matter of a handful of lines.

Writing Your Query Piece By Piece

So how is this done? As you create your queries, instead of returning result sets, you are returning an expression tree. An 'expression tree' can be thought of as a query builder in the sql sense. When you are writing your 'select statements' in your code, Linq is storing what it needs to know to query the database when you finally are ready to get the results back.

This means, using the Northwind orders example above, we can just keep defining our Order query piece by piece in a very natural way and we can know that when we are hitting the database, we are getting back exactly what we want.

Here's an example of how you might query Orders in a Northwind database using this 'piece by piece' query approach:





using System;

using System.Collections.Generic;

using System.ComponentModel;

using System.Data;

using System.Drawing;

using System.Linq;

using System.Text;

using System.Windows.Forms;

using System.Data.Linq;

 

namespace NorthwindLinqExample

{

    public partial class NorthwindQuery : Form

    {

        public NorthwindQuery()

        {

            InitializeComponent();

        }

 

        private void btnSearch_Click(object sender, EventArgs e)

        {

 

            using (NorthwindDataContext database = new NorthwindDataContext())

            {

                // get the query for all orders

                var orderQuery = from o in database.Orders select o;

 

                // checks to see if you want to filter by order date

                if (this.chkUseOrderDate.Checked)

                {

                    orderQuery = filterOrderQueryByOrderDate(orderQuery);

                }

 

                // checks to see if you want to filter by freight

                if (this.txtFreight.Text != String.Empty)

                {

                    orderQuery = filterOrderQueryByFreight(orderQuery);

                }

 

                // checks to see if you want to filter by shipping country

                if (this.cmbShipCountry.Text != String.Empty)

                {

                    orderQuery = filterOrderQueryByShipCountry(orderQuery);

                }

 

                // checks to see if you want to load order details also

                if (this.chkLoadOrderDetails.Checked)

                {

                    setQueryToLoadOrderDetails(database);

                }

 

                // sets the results of the query as the data source on our data grid

                this.gvResults.DataSource = orderQuery.ToList();

            }

        }

 

 

        private void setQueryToLoadOrderDetails(NorthwindDataContext database)

        {

            var loadOptions = new DataLoadOptions();

            loadOptions.LoadWith<Order>(o => o.Order_Details);

            database.LoadOptions = loadOptions;

        }

 

        private IQueryable<Order> filterOrderQueryByOrderDate(IQueryable<Order> orderQuery)

        {

            var filterValue = this.dateTimeOrderDate.Value.Date;

 

            return from o in orderQuery

                   where o.OrderDate >= filterValue &&

                   o.OrderDate <= filterValue.AddDays(1)

                   select o;

        }

 

        private IQueryable<Order> filterOrderQueryByFreight(IQueryable<Order> orderQuery)

        {

            var filterValue = this.txtFreight.Text;

 

            return from o in orderQuery

                   where o.Freight == filterValue

                   select o;

        }

 

        private IQueryable<Order> filterOrderQueryByShipCountry(IQueryable<Order> orderQuery)

        {

            var filterValue = this.cmbShipCountry.Text;

 

            return from o in orderQuery

                   where o.ShipCountry == filterValue

                   select o;

        }

    }

}