Wednesday, August 27, 2014

Scala course on Coursera by Martin Odersky

Martin Odersky, the creator of Scala, is doing an online course on Coursera starting September 15th of this year (2014). It's an advanced course ment for those who already know a programming language like Java or C# although knowing languages like C/C++, Python, Javascript or Ruby will also work. Since I am actually in the process of reading Martin Ordersky excellent Scala book, this works out nicely for me. The first edition of that book is available for online reading too.

Scala is an attempt to create a language for the JVM that is completely compatible with existing Java code but pushes further than Java. Scala adds things like lambdas and function programming concepts while trying to address criticisms of Java like its verbosity. Scala is a language that has been picking up steam recently and it seems to be where the cool JVM people are hanging out (assuming it's possible to be cool with Java).





Ok, one more ridiculous video: is that a Mac Plus? Pff, Java doesn't run on that.

Tuesday, August 26, 2014

final Keyword in Java

I like the "final" keyword in Java. However, I'd like it more if every reference were final by default and "muttable" was a the keyword to create a mutable a reference. This way around is better because if you see "mutable" in a reference declaration then you know that the author took the time to do that because they are mutating the reference. Mutation is the case you want to watch out for and discourage. Today, if a reference isn't marked as final you don't know if that means it's mutated or if the programmer isn't using final. In my experience mutability is the rarity and final is the common case.

Unlike C++, Java doesn't have a way of making both the reference and object constant. In java the declaration.

public class Foo {
    private final Date someDate = new Date();

// ....

Doesn't stop you from mutating the object like this:

    someDate.setTime(1234); 

It just stops you from changing the reference like this:

    someDate = new Date(); // not allowed, referenced is final

Some programmers use the final keyword to mean that not only shouldn't the reference change but the object shouldn't either. Restricting final in this way is not a good idea. Basically, that's not what final means and if you do that you're not helping anything.

I am generally against using language keywords like final to mean something other than what the compiler can assert from them. Marking a reference final doesn't guarantee that the object won't change. You might as well add a comment to the declaration like this:

public class Foo {
    // someDate is not mutated
    private final Date someDate = new Date();

// ....

Because it has about the same chance of either being honoured or going out of date.

Additionally, knowing that the reference is final (not the object) is still very useful on its own. This is why C++ has the ability to make both the object and reference constant independently. By using final to mean the object and reference shouldn't change you confuse maintainers with an idiosyncratic style and destroy the utility of final-for-references-only.

If you want to create an object that doesn't change then create an immutable object. (see the link)

I have to say that I personally don't make function parameters final because it's too much bother. I've found it looks like noise in the code and other developers resent it. Instead I have the compiler warn me if there's any assignment to a method parameter, which accomplishes the same thing.

If you are using eclipse you can turn that on by going to Window -> Preferences -> Java -> Compiler -> Errors/Warnings and turning on the "Code Style -> Parameter Assignment" warning. Go ahead, make it an "Error" if you dare.

Marking an object's members as final is much more useful. I've always though of object members as mini-global variables. Global variables are bad but global constants are less so. When followed religiously, it allows the maintainer to see at a glance which references are being mutated or ideally that non are. When combined with extensive use of immutable objects it also allows you to quickly see what is being mutated if anything.

While I don't bother marking method parameters final, I try and mark every class and object member I can as final. I've gone so far as to re-write code to mark more members final. I find it helps me understand my code and code of others faster than if I didn't and avoid errors too.

Friday, August 22, 2014

Evangelism in Software

"Don't worry about people stealing your ideas. If your ideas are any good, you'll have to ram them down people's throats." - Howard H. Aiken

Imagine that one day you're sitting in your office  coding away and suddenly you hit upon this idea for a glorious new framework that allows you to not only solve the problem but also problems you know that others are working on. You write up your framework, make sure it's unit tested and put it in the libraries project for all to use and you're done.

Well, no you're not. While the framework might be available no one knows about it yet. So, no problem, you send an email to the developer's mailing list and for whatever reason, people still aren't using it. No problem, you've done enough code reviews to know that people need a bit of background knowledge when using the library so you create a presentation that explains why it's the library is awesome and present it to everyone. It seems to go well, but even then, no one uses it. What is going on? Everyone keeps doing it the long way. What is going on?

Insight is special because it must be earned with experience; long hours getting burnt doing silly things over and over again until you see the light of a better way. The more revolutionary the insight the more likely that people won't understand or appreciate it.

For example, back in the late 1950s, John Backus and a small team at IBM developed Fortan. The first high level language capable for being used on the limited computers at the time. It was designed to make the act of programming quicker and easier. It was an incredible technical accomplishment. It was also treated dismissively by many in the industry who though it didn't make programming easier at all or that they didn't need it because they already found programming easy. They preferred to stick to assembly.



This pattern repeats with structured programming, object oriented programming, revision control, garbage collectors, lambdas and even today with functional programming.

The worst thing about any revolutionary technology is it makes your knowledge obsolete. There's a cost to throwing out what you know and the more you know the more you need to throw out. If you take a look at the list of things above, they were all very hyped as silver bullets when they came out. Not all of them lived up to the hype... and these were the ideas that succeed. Writing frameworks is hard and very few are done well. It's hardly surprising people are sceptical.

In order to get any change accepted you need to:

  1. Buy in from those who will be affected. Change isn't always good and you need people to believe that your change is a change for the better.
  2. It must be understandable. The stepper the learning curve the more you have to work to convince people it will be worth the payoff.
  3. Must have a significant payoff. The advantages of your new framework should be easy to articulate; work on your elevator statement.

Once you've made your first converts things get easier because they will evangelize on your behalf. At least they should. If you can't even get the people who work with you in the same domain to take an interest you might not have the revolutionary new framework you think you do. In this case listen to the feedback they are giving. They might have some insight that will make your framework better.

I can tell you, creating a new API that other people actually use is very difficult. It's as much a political challenge as it is a technical one. That said, sometimes the results of all that effort can be revolutionary.

Wednesday, August 20, 2014

Coding Standards - not blank!

In answer to the question: "Where did the blog post on coding standards go?". Much like the one ring it's still secret and it's still safe; I haven't finished writing it yet. I'm not sure why that blank post showed up but I think that I accidentally clicked the publish button before I started writing. It does work on symbolic level, though. The worst coding standard is the blank coding standard.

Every large team should have a coding standard that defines how that code should be formatted. These coding standards make code quicker and easier to read because the style is consistent and once you learn it your brain can work via pattern recognition - that is reading code by processing its shape not its content. It's the same sort of thing as the brain does when it comes to reading normal text. That's what makes "italics" and ALL CAPS more difficult to read. Your brain has to work harder because it can't just match shapes, it has to do some decoding first.

I don't agree with the theory that one coding standard is any better than the other. The most readable coding standard is the most familiar. K&R, BSD, or Gnome coding standard is easy to read because it's familiar. The only point I'd make is it would be nice if everyone used the same coding standard. Come to think of it, why aren't C-like languages defined in such a way that limits the different conventions possible.. Like the position of the {}? I suppose the original reason was that C was intended to provide a flexible toolkit for creating as many arguments as possible. Using C and its derivatives, it's possible to create lengthy discussions that go on for hours without conclusion. It's also possible to create bitter team divisions as the brain's legacy limbic system switches on and forms tribes around different ideals. Hallelujah! and Amen.

:-)


Thursday, August 14, 2014

Using Multiple Monitors

I've been an advocate of multiple monitors even since I realized that productivity wasn't decadence. I have three monitors attached to my computer all of them are Dell U2211-H panels. The U2211-H panel has a 16:9 aspect ration which makes them a little too wide. Text is best read in long, thin columns so the extra width is overkill for that. Also, if you have three 16:9 screens you need a really wide desk. 4:3 is the perfect ratio for multiple computer screens. If you're working on text all day, a portrait oriented 4:3 is even better since you can fit more text on the screen that way. It's also close to the ratio of a standard sheet of 8.5 X 11.

The main problem with 4:3 screens is that they are very expensive new. The rumour is that most computer monitors these days are simply a repackaging of HDTVs. Even the 16:10 aspect ratio is getting hard to find and 16:10 is better than 16:9.

My three 16:9 monitors are laid out with the left most one in portrait mode and the other two in landscape. I put my code editor and email client on the portrait screen and use the two landscape monitors for everything else. Having a 16:9 monitor in portrait mode is comically tall but works really well for code and emails. Using a 16:9 monitor in portrait mode allows me to see 95 lines of code on the screen at once (with a nice big font!) without having to scroll. To get the same effect from a larger 16:9 monitor you'd have to almost double the display size.


I use my computer for both gaming and coding. Having the leftmost monitor in portrait mode is a comprise. It means that whenever I code I move my mouse and keyboard over to the left so that this screen becomes my primary screen. Then, once the day is over, I slide back over to use my centre, landscape monitor for games and web browsing. My monitors are actually on a pivot so I should try just pivoting the centre monitor while I'm working then pivoting it back when I'm done. Maybe I should put all three monitors in portrait mode, since the bulk of what I do involves reading code or text. I'll have to try that over the next few days and get back to you.

Recent Window management features added in Windows 7 make super large displays a second option to the multi-monitor setup. Windows 7 has the ability to snap a window to the left or right half of the screen. This makes managing two windows on the screen at once far easier. The keyboard shortcut for this is windows key-left or right arrow.

That's really the only difference between multiple monitors or one massive monitor - software support. With multiple monitors the OS knows you want to use each monitor as a separate region so that works already. If you're using one giant monitor the OS has no idea what to do. Do you want one big window or a bunch of regions or what? Back when I bought my three monitors I didn't know about the Windows 7 snap features so, I chose multiple monitors rather than rely on software that might not be there. Oh well.. I still love my three monitors.


Wednesday, August 13, 2014

Software Development Estimates

Programmer estimates are notoriously bad. Having been in the position to give estimates and to be blocked by another's faulty estimate, the whole process is deeply frustrating for everyone.

I've been trying to figure out what is going on with task estimation for a while. There are plenty of theories on the internet. Many say programmers are just too optimistic. That's undoubtedly part of it but in terms of a theory with predictive and explanatory power it's like saying that programmers underestimate tasks because their estimates are always too short.

A group of psychologists first proposed a theoretical basis for.. I feel I should say "optimism".. in a 1979 paper called "Intuitive prediction: biases and corrective procedures". This pattern of optimist task estimation has shown up in tax form completion, school work, origami and a bunch of other things. From this we can conclude that programmers don't suck at task estimation, human beings suck at task estimation. A condemnation of humanity but an encouraging result for those who still cling to the ridiculous notion that programmers are people. The phenomenon is called the "Planning Fallacy" and Wikipedia has a great summary if you're interested. I estimate it will take you 5 seconds to read so go ahead.



Optimistic estimates are bad enough, but organizations will often add estimate distortions of their own.

  1. Fabricate a schedule without consulting anyone who will be taking part of the project.
  2. Ask developers for estimates and then cut them down because they are too high or those wacky developers are "padding" their estimates.
  3. Ask for estimates from developers without there being a detailed plan to estimate.
  4. Use estimates for a plan that's not being used anymore. Sometimes this happens because of additional features were added. Sometimes it's because the team you are integrating with hit a technical wall and had to change everything. Sometimes people are still using the pre-project estimates when the task estimates are available.
  5. Ask a lead developer for an estimate then give the job to an intern.

I've been on more projects than I can count and it didn't seem to matter who was on the project or how many times they've been through the process because at least one of these things happened. The last project I was on had the last three estimation pathologies show up at one point or another.

Once a project is behind schedule and you have a deadline to meet, all the options start to suck.


There have been many attempts to fix this estimate mess some more successful than others. The current best practice is to use some variant of agile/scrum. I say "some variant" because agile programming has many forms and not all of them are understood properly.

Agile software development turns the estimate problem on its head by admitting that estimates are likely to be wrong and that features will be added or removed so why not deal with it. The first thing agile does is to try and compute a fudge factor for the task estimates. The assumption is that the estimates are relatively accurate they are just off by some constant factor X. If you can figure out this factor you can multiply all estimates by X and get a more accurate number. In practice this helps but ins't a panacea.

The second thing Agile does is it seeks to minimize the problems bad estimates cause. This may seem like giving up because it is. As far as I can tell, no one in the software development industry has managed to solve this problem outside of some very special cases. The best strategy is to plan for bad estimates.

With agile, programming task estimates are re-calculated on a weekly basis so that estimates can include the latest information as its discovered. The focus is on keeping estimates up to date and communicating them instead of realizing a feature is too large when it's impossible to do anything about it. Additionally, features are done in order of importance; there's an ordered product backlog with the most critical things at the top of the pile. This way, developers know what is important and what isn't and can work with that in mind. When the deadline comes around you're guaranteed to have the important things done instead of whatever was fun to program.

It's way too hard to give a good summary of Agile here so I'm going to point to some resources:

  • Wikipedia's Agile Software Development page is a good starting point.
  • You can also look at Scrum. Most industry standard agile best practices are some variant of Scrum.
  • Joel Spolsky advocates his version which he calls Evidence Based Scheduling. I wouldn't normally include this but the page has a good explanation of where typical task estimation goes wrong.
  • There are quite a few consulting groups that can help too.


Developers need a feedback loop for their estimates. At some point after the feature has been implemented (this includes debugging!), developers should get feedback as to the original estimate and the actual time taken. Most agile tools will try to capture that but it might miss important aspects like design time or the time it took to gather requirements. In any case, this information should be explicitly presented to everyone on the team (including managers). Developers rarely consider how long - in real time - things are taking or how it relates to their estimates. Compiling and presenting the numbers to the team at the end of a project, when they are likely to be receptive, is enough to start this thinking process. It also communicates that the organization cares about the accuracy of estimates. If you're curious about why something took so long, this is a good place to have that discussion.

Out of the many projects I have been on, none of them have outright failed despite the usual optimistic estimates and estimation pathologies. This is because I have always worked within organizations that that were flexible enough to work with bad estimates. I am completely convinced that protecting your project against bad estimates is a realistic approach to managing estimate risk. That said, better estimates are always welcome so watch out for organizational estimation pathologies and make sure the developers realize how long their tasks are actually taking vs their estimates.


Thursday, August 7, 2014

Garbage Collectors

Hurray for garbage collectors!

There's a been quite a lot of work put into garbage collectors these last 10 years and they have gotten a great deal better. They now pause less and are more efficient then their forbearer and a good thing to. Common wisdom has it that there's a 2X productivity different between managing memory yourself and using a garbage collector system so it's a good thing they're more usable now.

I've been watching a video about modern garbage collectors online. It's called "Understanding Java Garbage Collection and what you can do about it". Take a look.