Software and craftsmanship

We are in the age of craftsmanship when it comes to software development.  I recently gave a presentation at STPCon where I compared the production of software to the production of automobiles, all of that leading up to a discussion on Lean Manufacturing and how that might apply to software.  At first I thought that automation (replaceable parts made possible by tool and die making) was the answer to quality software, but now I am beginning to wonder.

My presentation focused on Formula One race cars and how performance testing software was a lot like testing a Formula One car.  Those cars are state-of-the-art when it comes to wheeled vehicle design, and they are also hand crafted.  It occurred to me that craftsmanship is a desirable thing.  Would you rather have a car that comes off an automated assembly line or would you rather have a hand-crafted one?  Software and automobiles are not the same class of objects.   Ford invented the assembly line in order to product large numbers of dependable automobiles at a reasonable cost.  The assembly line churns out copy after copy of a car design, and they are all identical.  With software we make it once and copy it over and over.  In other words, we have no need to create the software from scratch over and over again.  Once we've made it, we don't have to make it again.  It's a different beast.

With automation, everything we make is an exact copy of the one we made before.  With craftsmanship, everything we make is different from the one we made before.  There will be similarities, but they will not be identical.  With automobiles, all the dependency is on the designer.  A good designer can turn out an automobile that is a work of art, or a total lemon.  We can start out with a good design and end up with a total lemon if our assembly process is flawed.  Herein lies the key to successful software creation.  We need to begin with a good design and marry it to a repeatable assembly process.

I recall watching a news clip of the pilot of one of the joint strike fighter competitors right after he stepped out of the cockpit after the first test flight.  He said, "It flew just like it did in the simulator."  That statement struck me.  They designed the plane using Computer Aided Design software, and then they built a simulator to test the design before they even started on the prototype.  Imagine if we could do that with software.


A new political party

If you wanted to start a new political party, how would you do it?  Well, you would need basic principles:

  • All decisions we make must be in the best interests of Canada.
  • All debate will occur in public, and only in public.
  • Debate will continue on a decision until everyone agrees.

Let's examine the principles.  First, acting in the best interest of Canada makes perfect sense.  This is our home, and we should look after it.

The second principle comes from some thinking I've been doing about the art of speech.  (I'll blog about that later.)  In the art of speech there can be only one kind of debate, and that is friendly debate.  Debate that occurs behind closed doors smacks of collusion or espionage or gossip and is unacceptable in an enlightened community.

The third principle has the effect of forcing us to really take our time with decisions.  Debate on a decision can go on for years, and in fact, some debates may never end.  I'm making a value judgement here in saying that taking our time with decisions that affect every part of our community is a good thing.

Would that be enough?  Would you join?


The end of money

Canada has decided to stop making the penny.  That's sort of like saying we are going to stop using the number one.  Let's think about this for awhile.  Let's suppose that everyone likes not having pennies in their pockets, and the government saves a bunch of money (not pennies).  The results are so positive that they decide to get rid of the nickel and the dime.  Why not?  They're pretty useless too.  We'll keep quarters and loonies and toonies for vending machines and coin operated devices.

At this point store owners may say, "Ya know what?  Handling coins is expensive and time consuming, and we would rather not do that anymore.  From now on we only accept plastic."  Not a problem, say we, the people, we like only having to carry a small plastic card around with us.  Heck, even the coin operated devices will be "swipeable/chipable" soon.  Get rid of all the money!  Our lives will be simpler.  Prosperity for everyone!

Hm...sounds wonderful.  What could possibly go wrong?  Let's say that the banks and the insurance companies merge.  They're doing that now, so it's not much of a stretch.  (Gambling and money lending have always gone hand in hand.)  Now let's say that these new "credit authorities" discover that injuries from skateboards are costing them lots of money in insurance claims.  They could say, stop using skateboards, but people would just laugh at them.  They could try to make skateboards illegal, wait, we did that once already, didn't we?  They could make all the skateboarders wear helmets and wrap themselves in foam, yeah, that'll work!

But, what if you couldn't buy a skateboard?  What if your little plastic card wouldn't let you buy something that was unsafe, that was undesirable?  If you have money in your hands, you can do whatever you like with it.  If all you have is plastic, you can do only what the computer at the bank will let you do.  Think about it.


To hear the cold

warm
inside
outside
wood soft
snow
the rhythms of snow
slow
graduated
gradual
just finished that
in the warm
in the city
look
to the cold
coursing through
magic magic magic
listen to the cold
the visitor
who comes
who dreams
to give
to take
to hear
to hear the cold coming through
to sing the cold that comes
to sing


Live Programming System

It's been awhile.  I am pulling the strings on my little Android startup.  It was a fun experiment, but it will never be a money maker.  The code is now open source, so have fun with it.

Meanwhile, my little gray cells have been busy.  Awhile back I blogged about a 3D IDE, and now that I have nothing else to do I've decided to flesh out the design.  Idle hands, as the saying goes.  This idea could change the world of programming as we know it.

Perhaps you have heard of network visualization.  A good example of this technology in action can be found at Linkedin Maps.  Network visualization will form the heart of the live programming system.  Here's a simple context diagram showing the components of the system.

Let's review the components.  As you can see from the diagram, the code repository is an external component.  Ideally, LPS should be able to connect to any compatible code repository.  The core function of the LPS system will be to pull data from the code repository, like Github for example, and present the network visualization of the repository.  What will that look like?

Well, each node in the visualization will represent a project in the repository.  The size of the node will reflect the size of the code base.  The more lines of code, the larger the node.  The color of the node will represent the level of activity on the project.  The color white will represent no activity, or a project that has gone dormant.  Bright red, for example, would represent a project that has a lot of activity.  The nodes will represent one more dimension of information by blinking.  Blinking will indicate that a programmer is currently coding on a module in that node and is available for live programming.  Live what, you say?!  That's right, live programming in real time.

Once the user has chosen the project they wish to browse, they will launch the programming browser system from the node.  This will open an Eclipse-like work space with a number of windows.  The main window will show the current file being edited by the live programmer.  There will be a windows showing who else is viewing the programmer, and a window for chatting.  The chat window will have the ability to do syntax highlighting so that viewers can suggest code snippets or alternative approaches to the problem being solved by the programmer.  Think of as a live webcast where the main window contains source code instead of a slide deck.

From the programmer's perspective, they will have a plugin on their normal IDE that will connect them to the LPS server.  This will make them available for viewing.  They can present themselves in a couple of different modes.  In private mode, they don't see who's connected, who's typing code samples, etc.  They become like the main attraction in a spectator sport with fans looking on from outside the glass house.  In collaborative mode they can switch perspectives and participate in a kind of paired programming context, but without all the sweaty armpits.

On a programming project, you might require all your programmers to be connected to the in house LPS.  That way architects and senior programmers can keep a watchful eye on the more junior members.  Another way it might be used is when a programmer gets stuck he could ask a coworker to "connect up" and help him with the problem.

I think this will be incredibly popular.  I mean, who wouldn't want to watch Linus Torvalds, for example, code the Linux kernel in real time?  I bet people would line up and even pay money to see that.

Anyway, that's my design.  I'm going to poke at it for awhile unless someone offers me a job.  Then it will go into the "would be cool" pile.

 

 


Startup Story

What a busy couple of months it has been.  Last year I started working on an idea for the tablet market.  I wanted to create an application that would help coaches teach their players the fundamentals of formations, positions, and movement.  The tablet market seemed like the perfect market for such an application, so I started doing research.  I toyed around with developing for the Blackberry and Nokia markets, but once I tried Android there was no turning back.  I downloaded and explored the Android SDK, and after a couple of months I had a pretty good idea of where I wanted to go.

Since I was contracting full time last year, things were moving along very slowly.  I know myself well enough now to know that when I am working on these things alone I tend to solve all the fun problems and then lose interest.  I knew if I wanted to keep this idea going I would have to get a team together, and that fit in well with my dream of having a software development company.  I posted a developer position on craigslist for the Winnipeg market and got zero resumes.  That was a bit of a shock.  Either the market was very tight, or there were no Android skills in my area.

I put the idea on the back burner for a month or so, and then started thinking that if I couldn't find a team perhaps I needed to grow one.  When I came off my last contract in December I had time to focus on this full time.  I called my contact at the University of Manitoba Computer Science Co-op program, and it turned out they had one student left who hadn't found a placement.  I quickly dropped into the local incubator project (The Eureka Project) at SmartPark, and they had space available.  Things were starting to come together.

Now, I have interviewed a lot of IT people, but it didn't take me long to realize why my co-op candidate hadn't found a placement.  One-word answers were about all I could get out of him, but I know that being a good developer, or software quality specialist, doesn't mean you have to have great soft skills.  I finally gave up trying to get him to talk and just stood up at the white board and started describing the design.  His eyes lit up, and all he said was, "That sounds cool!" Good enough for me.  We started developing on January 10th, and it turned out that my developer has the right stuff because we went live with version 1.0 on February 17th.

If you are into soccer or basketball and you have a relatively new Android based device you can find the application here: SportsBoards2D. We have a couple more sprints ahead of us where we will add a few more features.


It's all about the channel

Why do we have radios in our cars? So we can pipe advertising to a captive audience. Why do we have TVs in our homes? So we can pipe advertising to a captive audience.  Why, when you go to the movie theater now, do you have to sit and watch commercials for 5 minutes before the movies starts? Captive audience.  And, why do we have our computers plugged into the Internet?  If you had asked that question fifteen years ago the answer would have been to share information, and that's still partly the answer.  But, things are changing.

What do radio, TV and the Internet all have in common?  They are all channels for advertising.  These channels can be categorized in three ways: one-way, semi-one-way and two-way.  One-way channels are channels where the communication flows only in one direction.  Radio and TV over the air waves are one-way channels.  They broadcast out, you pick up the signal, and there is nothing sent back.

Semi-one-way channels are channels where the communication flows mainly in one direction, but some data is returned in the other direction.  Satellite radio, and cable and satellite TV are semi-one-way channels.  Most of the communication flows to you, but only your usage information, such as which stations you listen to, and how often you change stations, flows back.

Two-way channels are channels where communication flows equally in both directions.  The Internet is a two-way channel.  Anyone can put up their own website these days.  Blogging is a good example.  I can send my thoughts to you, and you can send your thoughts to me.  You are in a sense on equal footing with the big players.

Content providers and advertisers like one-way and semi-one-way channels because it's very easy to capture the audience. If you want to watch Gilligan's Island over air or cable TV, you have to watch the commercials; you can't turn them off.  You can try to avoid them by leaving the room, but you might miss the part where they get off the island, so you sit through them.

Advertisers like semi-one-way channels even more because not only can they capture the audience, they can place them in a glass box and study their behavior.  They can figure out what's popular and what's not and tailor their content creation and advertising techniques to be more effective.

With a two-way channel like the Internet, it's a bit more difficult to capture the audience.  There are millions of websites to visit around the web, and you are free to visit any one of them.  Efforts were made in the early days of the internet to capture the audience, but they failed miserably.  Internet service providers would provide software to make it easy for their customers to connect, and this software would provide a custom browser and override the user's home page preferences.  Early attempts to provide Internet via the TV also flopped.  With these systems the customer was provided with a keyboard that would control a custom browser running on the set-top box.  These were early attempts to force the Internet viewing audience (the old, captured TV audience that managed to escape) back into their seats.

Today, people are no longer sitting in front of their TVs as much as they used to; they sit in front of their computers.  The audience has escaped the old channels and entered into a new channel where they are no longer captive, and this has caused a problem for advertisers.

In the early days of the Internet one of the big challenges was to figure out how many "eyeballs" your website was getting, or what was the "stickiness" of your web site.  These terms referred to how many visitors your web site received and how long they stayed in your site before browsing away.  The challenge was trying to figure out how much to charge for an advertisement on a web page.  How do you know people are seeing the ad, how many are clicking on the ad, and how many are buying the product after the click on the ad?  These are the kinds of questions advertisers were struggling with.

Nowadays, companies like Doubleclick (owned by Google) have this mastered.  They know exactly how many users visit a page, how many times an ad is displayed, and how many click the ad.  But savvy Internet users have figured out a way around advertising on the web.  Browsers can be fitted with plugins like Adblock which prevent advertisements from displaying on web pages.  Once again, the Internet audience has proven to be difficult to capture.

A strange thing has happened over the last couple of years; the Internet has become branded.  You know the brands: Twitter, Facebook, Google, Amazon, Youtube, etc.  There are dozens of social networking sites, dozens of search engines, countless numbers of places to buy things, but for some reason everyone is talking about these sites like they are the only ones that exist.  How did this happen?  When did it happen?

At first it was a bit of a puzzler, but then it hit me.  Smartphones.  We are all buying smartphones now, and every smartphone comes with these brands installed.  On many smartphones you can't uninstall them unless you jump through some technical hoops that the average person won't be able to handle.  This is a quiet throwback to the early days of custom browsers and Internet on the TV, and it reads like a second attempt to put the captive audience back into their seats.

One of the problems with smartphones is that they are too small.  Besides actually calling people with them (do people still use them for that?), you can't really browse the Internet or do much more than texting or "tweeting" with them.  You need something bigger, like a tablet.

The tablet revolution is happening now; last year they were a novelty, but this year they will become a necessity.  Why would you lug a laptop around when  you can do everything you want with a nice compact tablet?  It will fit in your pocket.  (Remember those cigarette pockets they used to put on golf shirts?  Clothes will be redesigned to accommodate the tablet.  Watch for those baggy cargo pants to come back in fashion.)  It will certainly fit in a purse given the current fashion for really big purses, and you won't even know it's in your backpack because they are so light and compact.  Everyone is going to want one.

And, the phone companies have brought out a wonderful new thing: you can share your data plan between your phone and your tablet.  One might be surprised by the seemingly generous move by the phone companies given their current track record for overcharging their customers ridiculous sums for using the Internet on their phones.  But, it's not so surprising when you discover that the same Internet brands come installed on your tablet.

I bet if you go to the bar where the old advertising guys sit around talking of the glory days you would hear them reminisce about the days when you could only get four stations on the TV set.  Those were the days of the really big brands because absolutely everyone saw the same commercials.  Those were the days when brand names were household words.  In those days, if you had a runny nose you reached for a Kleenex, not a tissue paper.  Today, when you want to talk to someone you don't pay them a visit or even call them on the phone, you "Tweet them" or "Facebook them".  Those phrases have become new household words in the last couple of years.

Now, personally, I don't like being forced into a captive audience.  I never listen to the radio, ever.  I'm one of those people who switches the channel whenever a commercial comes on the TV, and I spend most of my watching time on the commercial-free stations.  (I'd pay money for a commercial-free science fiction station.)  I have a smartphone, but I don't have a data plan.  I like having the freedom to choose.

You might argue that if tablets really do take over that people will just hack them and run their own un-locked applications on them.  I know I will.  I run Linux on all my computers for exactly this reason.  I get to choose what I see.  Now there's a strange concept.  A Clockwork Orange flashes to mind, with that scene where they force open Alex's eyes so he can't stop watching.  I grabbed the book off the shelf to check my references and found this wonderfully appropriate quote on the first page, "things changing so skorry these days and everybody very quick to forget, newspapers not being read much neither".  Anthony Burgess, 1962.

Put all this next to the current move by the channel providers to meter bandwidth and block certain types of traffic, and you've got a bit of a pattern emerging.  There are data plans out there now where your Facebook usage doesn't increase your bandwidth consumption.  Why block, or make prohibitively expensive, some traffic but make others free and easy?  If everyone is using up all the bandwidth with Facebook and Twitter, wouldn't those be the obvious targets for metering?  Not if that's where you want everyone to go.


Wealth and Software Quality

I was reading (and re-reading) Paul Graham's essay on wealth creation, and it made me think in a new way about software testing. It's a fascinating article that provides valuable insight no matter what field you are in. First, let me provide a little context.

At the last WOPR, Goranka Bjedov was relating some thoughts from her presentation at StarWest where she questioned the future of testing. Basically, she was saying companies are picking speed and price, not quality, and that testers are going to disappear. That got me thinking about what exactly I do when it comes to software testing, and I blogged about it here. The way I see it, software quality as a goal will never disappear, but our roles in the software process need to evolve to stay current.

Paul's essay got me thinking even more about our roles as software quality specialists. Basically, he says don't confuse wealth with money. Wealth is what people want. Money is just the medium for moving wealth around. When you create software, you create wealth.  What kinds of wealth do software testers create? Who are we creating wealth for? Do they really want what we create? Interesting questions.

Testing creates wealth in the form of a service. Nobody is going to say, "Here's a buck. I'll take one of those yummy tests you are making." If fact, the things that testers create are often discarded, tossed out at the end of the project. We're a bit like waiters; we don't actually make the software -- we just help bring it to the customer. We're also a bit like the annoying violin guy in that the customer could be completely satisfied if we weren't there.

Our customers, the people we create wealth for, are the development teams. Do they want what we create? Most of the time they don't. We are like auditors, in a way, a necessary evil. We're there because there's something wrong with the way they create wealth. We're like a safety net. We're also a bit like leeches.  We confiscate their code and say, "You can't have it back until we are done testing it."  In the end, when the software is sold, we benefit from the wealth the development team creates. It's no surprise that development teams sometime look down at us, or see us as a roadblock or bottleneck.

One might argue that the customer wants our wealth indirectly because our wealth contributes to the overall wealth in the software. I think that is a pretty weak argument. If the software came out perfect and did exactly what the user wanted, would anyone care if it was never tested? Does the end user say, "I only want software that is tested?"  They just want software that works.

As Paul indicated, money is the medium of exchange for wealth; its a means for moving wealth around.  What is the medium for exchange for testing? In other words, what is the medium by which we move the wealth of testing around?  If your team is following a methodology like the Rational Unified Process (RUP), then you will be familiar with project artifacts like test strategies, test plans, test cases, and test results.  These artifacts are the medium of exchange.  They are the means by which we move our testing wealth into the software.  But, how effective are they?

The test strategy lays out the connection between requirements and testing.  Given a set of requirements, the team comes to agreement on a plan of attack.  Very little testing wealth is transferred through the test strategy.  In fact, one could argue that no wealth is transferred.  If the goal of the project is to produce quality software, then all we are really doing in the test strategy is saying, "Yes, we will take steps to create better quality software."  Once the software is complete, the strategy gets discarded or tossed into the archives, never to be seen again.

Given an agreed upon plan of attack, the test plan lays out the details on how the team will execute.  It says we need these people in these roles using these tools following a high level schedule to produce a set of results.  Again, it's hard to find any wealth being transferred by this artifact.  Like the strategy, the test plan gets discarded and the software has not gotten any better.

Test cases are a curious artifact.  Some projects spend hundreds of hours laboriously listing all possible test cases and mapping them back to the requirements.  Sometimes that is a useful exercise, especially when the teams are inexperienced, but can you test without first documenting all possible test cases?  Of course you can, but again, let's ask the question, "Have we transferred any wealth to the software?"  Not yet.

Software testing transfers wealth to the customer through finding and fixing bugs.  That's where the real wealth is found.  Every bug that is found and fixed makes the software a little better, a little less likely to break.  This is what the user wants.

Where does that leave us? We often find ourselves in a situation where we are forced on the software development teams, where we leech off their wealth, and where our customers couldn't care less about us. If we are following a process like RUP we may not be creating wealth at all but merely creating money with the hope that we might somehow, magically transfer wealth to the software we are creating.  Not a good place to be. How do we get ourselves out of this rabbit hole?

We need to shift our thinking. What we do is part of software development, and our wealth is built into the software we create. We need to stop thinking of our roles as separate from the software development process. We need to focus on the tools and technology that help drive quality into software. Things like Test Driven Development, build automation, code quality analysis, continuous integration and version control. Being able to monitor software as it runs in production, creating ways for software to notify us when it behaves badly, ensuring software can be updated seamlessly without interrupting the customer, and being able to design software that facilitates all that. We have to start thinking like software developers who specialize in quality.

That means we are not writing test strategies, test plans and test cases all the time. We're not gatekeepers who get in the way of wealth creation. We're part of the software development process.  We focus on creating the continuous integration systems.  We install and configure the code quality analysis tools.  We create the coding standards and API documentation.  We implement the version control systems.  We do everything we can to make sure code quality is built into the team, the processes, and the tools on a software project.

So, take a look at what you are doing when you get back to work, and ask yourself the question: am I creating wealth, or am I just creating money?  I think you'll find the answer quite enlightening.


Utility Model

I recently read an article by Richard Stallman on the dangers of Software as a Service (SaaS).  You can read it here.  In it he warns about how SaaS takes away your freedom in much the same way as closed source, proprietary software does.  He makes some good arguments, but I believe he misses where the real danger lies.

First a little background on SaaS.  With SaaS, you don't own the software or the hardware it runs on.  A good example of SaaS is on-line tax software.  With on-line tax software you use it once to create your tax file and you pay a flat rate for that service.  You get a tax file returned to you and perhaps they will do the submission to the government on your behalf.  You don't buy the tax software, you don't install it, and you don't have to worry about it again until next year.

The utility model is a little bit different but follows the same principles.  With the utility model you don't own the hardware or even the software; you simply pay for computing cycles as you need them the same way you pay for electricity from the electrical utility.  Unlike the tax software example, instead of paying a flat rate you pay for how many computing cycles you use.

Now, think back to how the electrical utilities were born.  In the early days of electricity, each city or factory would have its own electrical generator.  They would build it big enough to support future growth, but eventually they would grow out of their current generator.  They could either build another generator and hook it up the the old generator, in the form of a grid, or they could buy a neighboring generator to hook into.  The idea here is that you sell us your generator, we'll manage it for you, and you get the benefit of selling your excess capacity to us and that will help keep the cost of your electricity down for years to come.

The same idea can be applied to computers.  Companies would sell their computer power to the vendors who would in turn sell computing cycles back to them on a pay per use basis.  Now, companies don't need to buy computers anymore.  No need to worry about capacity; no need to worry about upgrading hardware and software.  Just install your applications on the utility, and pay for whatever cycles you use to run them.

Now, here's the problem.  Computing technology is a strategic resource, just like oil, refined uranium, spaceflight, etc.  Countries that have computers have an advantage over countries that don't have computers.  Depending on where you live, there are countries to whom you cannot give computing technology.

Let's say that all the major businesses in Canada decide to go with the utility model.  One of the side effects of this move is that all their old computer hardware would eventually disappear.  Remember, they don't have to worry about replacing old hardware, and it doesn't matter where the computing cycles come from as long as they can run their applications.  Eventually, the vendor would get rid of all the old junk sitting on the floors of the computer rooms and replace it would shiny new computing cycles.  These new cycles might be coming from inside the company's own computer room, or they could be coming from halfway around the world.  It doesn't matter, as long as the applications run smooth and the response time is acceptable.

Let's also say that suppose Canada decides they're not going to sell the US any more of our good Canadian beer because, well, they just drink too much of it, and there may be a shortage if we're not careful.  Then the US says, okay, then we're not going to sell you anymore computing cycles.  Oops.  We just transferred control of our ability to run out applications to another entity, and once that happens the entity can willingly or unwillingly take that control away from us.

As a software quality specialist, I am looking forward to working with software as a service and cloud technology.  I think it will provide some interesting challenges and there's nothing I like more than wrapping my little gray cells around a tough problem.  The danger with SaaS is not in the technology itself.  The danger is in not being in control of it, and not knowing where your computing cycles are coming from.  Open source software solves that problem by putting control into the hands of the user.  I'm not saying SaaS should all be open source.  But if SaaS offers advantages over the way we've been doing software till now, then let's explore it.  Let's kick the tires and see what SaaS can do, but let's make sure we have some tires to kick.

Companies should be building their own clouds.  They would provide huge advantages for development teams.  Think about how quickly you could turn up and down test machines.  Or think about how it would solve the problem of availability.  Instead of buy big iron boxes to eliminate single points of failure, design for the cloud and who cares if one node goes down.

There are projects out there now where you can get under the covers of clouds and software as a service.  Look into qemu-kvm which is the heart of virtualization.  Or check out Silver Lining, an open source cloud controller project.  These are just a couple, but if you dig you will find more.  And if you don't find them, get out there and start building them.


Welcome to my new blog

This is my new blog.  It's shiny! I'm starting to like blogging, and my old blog service was just not doing it for me.  'Nuf said.  Let's see where this goes.