Monday, April 20, 2009

How to work with progress bars, part 1



Have you ever seen a progress bar that goes from empty to full then starts over at the beginning again? I really hate those.

I had always assumed that progress bars were pretty uncontroversial. It wasn't until I worked at Intelerad for about a year that I found out that many people don't actually understand what progress bars are for, how they should work or how to write code to implement them.

When you present a progress bar to the user you're telling them:

1) That the computer is doing something that'll take a while.
2) That the computer hasn't frozen or otherwise become unresponsive.
3) Approximately how long the computer will be busy.


It's important to remember that a progress bar is not to display the progress some arbitrary chunk of code. If you think of a progress bar showing the progress through some function or chunk of work, you're likely to show multiple progress bars in a row where each filling of the progress bars will represent the progress through a particular task. Don't do this.

What the user actually wants to know is how long the computer will remain busy doing its thing. Alternatively, how long it will be until the computer returns a result. When you've filled up a progress bar and restarted it, you're messing with the user's head.


(Don't mess with the user's head)


A common worry when creating code that has a progress bar is that if you show progress bar for the entire time the computer is busy, and not for individual subtasks, then that code will not be reusable. It won't be reusable because you'll have to hard code the values you're setting the progressbar to in that function. As a result you won't be able to use that code with, say, another progressbar because the values that you need to set the progress bar to in that context will be different. 80% done for this progress bar might be only 40% done in a different context.

It's actually very easy to create little subroutines that only worry about the progress though their portion of the task and then wrap those progress meters into meta-progress meters to show the progress through any larger task. Here's how you do it.

Progress bar-fu:
(n.: the ancient Japanese art of making progress bars that don't jerk the end-user around)




Think of the progress through the overall task as being the sum of the progress through each individual task. Looking at the problem this way it should become apparent that each sub task can look at its own progress as going from 0 to 100% with this value being a smaller proportion of the progress of the overall task.

Let's look at an example. Let's say we have an overall task composed of three subtasks. The first sub task is 40% of the overall task. The second sub task is 50% of the overall task. The last task is 10% of the overall task. Drawing this out your great little chart like the following:



Here we can see that the first subtask's progress, as a value that goes from 0% to 100%, is equal to the overall task progress going from 0% to 40%. All we need to do to convert the sub task progress to the progress through the overall task is to multiply it by .0.4 (or 40%).

This strategy composes nicely.

Let's say that our first sub task is composed of two subtasks. The first sub sub task goes from 0 to 30%.



We can calculate the value of the subtasks progress by multiplying the sub-subtask progress by 0.3. We can then see the sub sub task's contribution to the overall task by multiplying it by 0.3 then 0.4 (or 0.4 * 0.3 = 0.12). So when the sub sub task gets to 100% we will have completed 12% of the overall task.

Here' how you do this in code. I'm going to use java because lots of people use it, understand it and it's what I know.

First you need an interface like this:

public interface Progress {
/**
* @param progress number between 0 an 1 that signifies the progress through a task.
*/
void advance( double progress );
}

You pass an object of this type to any function you want to track progress for. Like this:

/**
* Tells a progress bar to go from 0 to 1 (complete) in steps
*
* @param progress
* to set
* @param steps
* to take.. More takes longer.
*/
private static void pretendToDoSomething(Progress progress, int steps) {
progress.advance(0);
for (int i = 0; i < steps; i++) {
sleepAWhile();
progress.advance((double) i / steps);
}
progress.advance(1);
}

Oh course with real code you'd be something useful, but you get the idea.

Next you'll want an object to represent the overall progress. Here's what the interface would look like:

/**
* Allows for a progress bar made up of multiple sub progress bars.
*/
public class CompositeProgressBar {
public CompositeProgressBar(Progress progressBar) {
}

/**
* Builds a new sub progress bar. A sub progress bar is a progress bar whose
* full length maps to the param subProgressBarSize of the
* {@link CompositeProgressBar#masterProgress}.
*
* THis method will also advance the previous sub progress bar to 100%.
*
* @param subProgressBarSize
* @return a {@link Progress} that represents param
*/
public Progress buildSubProgressBar(final double subProgressBarSize) {
}
}

The idea is you'd pass the Progress object representing the overall progress and you'd call buildSubProgressBar to, well, build the sub-progress Progress objects.

Here's a typical example:

/**
* Demonstrates how the progress bar can be used recursively.
*
* @param progress
* - could be any progress - goes from 0 to 1
* @param label
* for text output
*/
private static void moreSubTasks(Progress progress, Sayable sayable) {
CompositeProgressBar compositeProgressBar = new CompositeProgressBar(progress);
sayable.say("part 1");
pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.2), 100);

sayable.say("part 2");
pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.2), 102);

sayable.say("part 3");
pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.6), 103);
}

Here's the implementation of CompositeProgressBar.pretendToDoSomething(Progress progress, int steps);

/**
* Builds a new sub progress bar. A sub progress bar is a progress bar whose
* full length maps to the param subProgressBarSize of the
* {@link CompositeProgressBar#masterProgress}.
*
* THis method will also advance the previous sub progress bar to 100%.
*
* @param subProgressBarSize
* @return a {@link Progress} that represents param
*/
public Progress buildSubProgressBar(final double subProgressBarSize) {
progresSoFar += currentSubProgressBarSize;
currentSubProgressBarSize = subProgressBarSize;
return new Progress() {
public void advance(double progress) {
if (progress < 0 || progress > 1)
throw new IllegalAccessError("\"progress\" should be less "
+ "between 0 and 1 but was: " + progress);
masterProgress.advance(subProgressBarSize * progress + progresSoFar);
}
};
}

For context here' what the entire object looks like:

/**
* Allows for a progress bar made up of multiple sub progress bars.
*/
public class CompositeProgressBar {
/** The master progress bar we're splitting into sub progress bars. */
private final Progress masterProgress;

/**
* Amount of progress we've gone though so far, not counting the amount is
* the latest sub progress bar
*/
private double progresSoFar = 0;

/**
* The length of the current sub progress bar.
*/
private double currentSubProgressBarSize = 0;

public CompositeProgressBar(Progress progressBar) {
this.masterProgress = progressBar;
}

/**
* Builds a new sub progress bar. A sub progress bar is a progress bar whose
* full length maps to the param subProgressBarSize of the
* {@link CompositeProgressBar#masterProgress}.
*
* THis method will also advance the previous sub progress bar to 100%.
*
* @param subProgressBarSize
* @return a {@link Progress} that represents param
*/
public Progress buildSubProgressBar(final double subProgressBarSize) {
progresSoFar += currentSubProgressBarSize;
currentSubProgressBarSize = subProgressBarSize;
return new Progress() {
public void advance(double progress) {
if (progress < 0 || progress > 1)
throw new IllegalAccessError("\"progress\" should be less "
+ "between 0 and 1 but was: " + progress);
masterProgress.advance(subProgressBarSize * progress + progresSoFar);
}
};
}
}

.. and, as I've just mentioned, you use the object like this:


/**
* Demonstrates how the progress bar can be used recursively.
*
* @param progress
* - could be any progress - goes from 0 to 1
* @param label
* for text output
*/
private static void moreSubTasks(Progress progress, Sayable sayable) {
CompositeProgressBar compositeProgressBar = new CompositeProgressBar(progress);
sayable.say("part 1");
pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.2), 100);

sayable.say("part 2");
pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.2), 102);

sayable.say("part 3");
pretendToDoSomething(compositeProgressBar.buildSubProgressBar(0.6), 103);
}



All we have to do now is look up this composite progress bar to some sort of GUI component. To do this we have to create a progress object that wraps a JProgress instance. Here's how you this:


private static Progress convertToProgress(final JProgressBar progressBar) {
return new Progress() {
public void advance(double progress) {
progressBar.setValue((int) (PROGRESS_MAX * progress));
progressBar.setString((int) (100 * progress) + "%");
progressBar.repaint();
}
};
}


You can then place this JProgress into a JFrame and send the Progress object to the CompositeProgressBar constructor and you're all set.

Congratulations. You are now masters of the first level of progress bar Fu. You can write a progress bar that accurately reflects the progress of the overall task, even when the overall task is made out of little, tiny pluggable pieces of code. What is more, those little tiny pluggable pieces of code can now be reused in different contexts, with different progress bars. This is truly a great day for the user.

The remaining question is, how do you get to level 2 of progress bar Fu? Ah, that is a good question young grasshopper.

What if, you have a task for which it is completely impossible to gauge how long the task will take?

What if, you have a task for which you have a rough estimate for how long it will take but, you have no way to increment the progress bar because your code is blocked doing something else. For example, it may be doing some I/O in a different thread. Alternatively, maybe running a third-party process and there's no way for you to get any feedback about what that process is doing. Now would you do?

For the answers to those questions you'll have to click here to go to part II


Part II

7 comments:

kevininspace said...

Hmmm, interesting post. Very useful too. I'd like to see what you have to say in Part 2, because to me it looks like whatever you pass to buildSubProgressBar() is a guess. And I assume that, since we're smart coders, we can actually resort to some fancy handwaving to estimate what that chunck of code should represent in the overall scheme of things. In other words, "How can I guess how much of the overall processing time will this sub-task consume?" Or am I just not getting this?

Andrew Trumper (AdderTheBlack) said...

The short answer is you need to profile. Essentially you run the code with some real data and measure how long each part takes. You can use this information to figure out what numbers to pass to buildSubProgressBar().

kevininspace said...

Yup, that would work.

Green Tractor said...

Progress modeling makes an assumption (my system and my systems conditions are the same as yours) that may be badly flawed.

Network data access speeds vary over a very wide range. Local File I/O performance varies - especially if file cache effects participate. GPU (if involved) capabilities really vary. CPU speeds vary to a lesser extent.

Kay said...

The thing with progress bars is that you need some kind of general information about the process you are trying to describe.
When talking about network access, which might suffer from congestion, you could use a floating average always assuming you know the total size you are downloading. That naturally extends to other things that might take varying amounts of time, especially if they are accessing resources you cannot control.

For subprocesses implementing algorithms you could also use the Big-O of that algorithm, if you know it. Obviously well-known algorithms are great fits for this.
Making a best guess is often good enough - if the user knows the step might take a long time to complete it's ok to say something imprecise.
It's even ok to err on the good side of things:
For example a sorting step using quicksort might be faster than what the Big-O for the expected runtime says it is (accounting for the constant factors in the calculation, of course!) but that's ok. It's a best guess.

But I wholeheartedly agree with your main point: Progress bars should never jump backwards, simply because that suggests that the computer made it all worse ;)

faqface said...

Andrew,
Computer people (programmers et al) often think/say that users are idiots.

Programmers are often idiots too.

Too often.

dalore said...

Let's look at an example. Let's say we have an overall task composed of three subtasks. The first sub task is 40% of the overall task. The second sub task is 50% of the overall task. The last task is 10% of the overall task.

The bit I don't get is where you came up with the figures of 40/50/10 ? How do you know those percentages?

Wouldn't the best/easiest way just to say each subtask is 1/3 of 100%?