Details

Created: 07 March 2016

Test Driven Development (TDD) is method from Extreme Programming (XP) that reverses the traditional order of coding and then testing. It works like this: Before you add a feature to a module, you devise a test that will demonstrate that the module properly supports that feature. You run the test, and it fails, since you haven't implemented the feature yet. You code up the feature, then run the test again, and this time it works. Then you go onto the next feature, repeating the same procedure, but running the previous test as well as the new test. You continue this cycle, building up a collection of tests as you add features to the module.

Now, why would anyone want work in this backwards fashion? After all, how do you even know what to test for before you have written the code?

That last question is a clue that something is wrong, and helps explain the reason for TDD.

In Agile development, each user story is required to have acceptance criteria, which are a means to answer the question "Did this coding change satisfy what the story asked for?" For example, consider a story that says "Users signing on to the system should be authenticated." There are all manner of solutions that could meet this criterion, and not all of them would satisfy what the author of the story wanted. On the other hand, consider a story with these acceptance criteria:

• Sign onto the system, using a user ID and password that are in the user database. The signon should be successful.
• Sign onto the system, passing in an OAuth 2.0 bearer token from Facebook. The signon should be successful.
• Attempt to sign onto the system with an invalid user ID or password. The signon should fail.
• Attempt to sign onto the system with an invalid OAuth 2.0 bearer token. The signon should fail.
• Perform three failed signon attempts, then attempt to sign on with a valid user ID and password. The attempt should fail until 30 minutes have passed.

These acceptance criteria give us a much clearer target to code for. TDD takes this a step further, by developing acceptance criteria for each step of the development, in the form of tests. You add tests and features in very small steps. Sometimes they feel unnaturally small, but they keep you thinking about edge conditions, as each test suggests the next one. Suppose, for example, you are writing a routine to format money. You start with a test that passes in a number, and when you implement the change, the output looks good. "Hmmm," you say. "What if the number is negative?" So you write a test to that, and the result does not look so good. So you change the code to do something sane with negative numbers, like displaying an error message, and you run the test again. "But what if the number is really big?" you think, so you write another test, and so on. Each time you do this, you write the simplest code that will pass the test, then you refactor it, if necessary, to make it cleaner and to avoid duplicate code. You can do this safely, because with the suite of tests you build up, you can be assured that the refactoring has not altered the behavior of the module.

One of the best resources I have found for starting out with TDD is Test-Driven Development by Example, by Kent Beck. The Part I of the book, chapters 1 through 17, take a fictitious application for managing bond portfolios and add to it the ability to handle multiple currencies. The author leads us through the test-code-test-refactor-test cycle little by little, showing how one test suggests another, and how it is possible to vary the size of our steps from baby steps, when things are confusing, to large steps when things are more clear.

Part II of the book, chapters 18 through 24, uses TDD to develop an xUnit harness in the Python language. xUnits are test harnesses that make TDD go much faster by keeping track of the tests for each module, running them, comparing their output with the expected output, and indicating which tests, if any, failed. This makes TDD much more painless than if one had to run each test manually and visually verify the results. xUnit is a generic term for TDD testing harnesses. Examples of more specific ones are JUnit for Java, CppUnit for C++, NUnit for .Net, and UUnit for Unity. This section of the book, in addition to giving a practical demonstration of TDD, also imparts a good understanding of how xUnit harnesses work, and is invaluable for anyone who needs to write one for a language that is not currently supported.

Part III of the book, chapters 25 through 32, deals with design patterns for tests, and for getting tests working. This includes answers to fundamental questions such as:

• What do we mean by testing?
• When do we test?
• How do we choose what logic to test?
• How do we choose what data to test?

It also includes special testing considerations, such as mocks (programs that simulate slow or hard-to-set-up things such as database calls, in order that the tests can run quickly), and fakes (a method used in some complicated cases where you start by hard-coding the result you want, which tests the test itself, before refactoring the code to do the desired processing).

The book is a good introduction to TDD, and does a good job of explaining the rationale, demonstrating how it is done, and giving you tools to help get you started.

My favorite quote from the book is "I taught Bethany, my oldest daughter, TDD as her first programming style when she was about age 12. She thinks you can't type in code unless there is a broken test. The rest of us have to muddle through reminding ourselves to write the tests."

The book is available from Amazon here.

Details

Created: 28 January 2016

"The quality goes in before the name goes on"--Zenith Electronics slogan.

"Quality is job one"--Ford Motor Company slogan.

When I worked on an automotive assembly line in the early 1970s, they had a Quality Control department. It acted like a sort of quality high-pass filter: incoming parts and assembled trucks were inspected; if they were high enough quality, they were used or shipped; if not, they were scrapped or sent to the rework department.

Over the years, Quality Control departments transformed themselves into Quality Assurance departments, the idea being that rather than just filtering out low-quality products, they would endeavor to make sure the products got built right in the first place.

The Toyota Production System refined this idea with poka-yoke (error proofing) and the andon light (where anyone on the line could signal a problem, and the line would stop while they figured out how to keep the problem from happening again).

Sadly, many software development organizations, particularly in mature companies, are still stuck in the era of Quality Control. The QA department finds bugs and reports them, the bugs are fixed, and charts are distributed to management so they know which groups to beat up about having too many bugs.

What is missing is the big picture of quality: it is not a department that acts as a gate, or even worse, gets rated on how many bugs they find. (I once worked with a QA department that sent very vague bug reports to development, things like "This message may be wrong." Their explanation was that they got rated on how many bugs they found, so their boss didn't want them to waste any time doing research on whether or not something really was a bug.)

Since we're talking advertising slogans, remember the Maytag repairman, who never had any work because Maytag washing machines were so good? That's kind of what we are shooting for, but we don't have a QA department that is sitting idle. Instead, we have a QA presence on each Scrum team (as required in order to have cross-functional teams), and in addition to figuring out how best to test things, he or she also helps the team design for quality and for testing.

Poka-yoke applies to software in many ways. Modules should be small, and should adhere to the Single-Responsibility Principle. (Robert C. Martin: "Functions should do one thing. They should do it well. They should do it only.) Their APIs should be well documented, preferably in extractable comments, like Doxygen or Javadoc, so they are more likely to stay in sync with the code. Their actions should make sense; in other words, when a programmer makes an assumption about what the module is going to do, he or she should usually be right.

Program source tends to get sloppy over time, particularly when you have new programmers who are accustomed to different bracketing styles. This can cause errors when a bracket isn't where it is expected and is overlooked. (Keeping modules small makes this a lot less of a problem, though.) Program source can be standardized to a common bracketing style using tools like Artistic Style.

Following the Single-Responsibility Principle makes it easier to unit-test modules. If a bunch small modules that have been thoroughly unit tested are assembled to make a product, the amount of testing required for the complete product is reduced. We still have to test to make sure we didn't have misunderstandings in the design or assembly, but if we know the modules are well behaved, we no longer have to throw everything but the kitchen sink at it.

With small, simple modules, static analysis tools like Sonarqube can help you analyze your test coverage, and make sure modules are not too complicated to adequately test. You also might look at the techniques that teams who write safety-critical software use, like The Power of 10: Rules for Developing Safety-Critical Code. Some of their rules, like no dynamic memory allocation, are too stringent for a lot of products, but it is worth looking at what they do.

Another area where design can improve testing is separating data from its presentation, as in the Model-View-Controller, Model-View-Presenter, or Model-View-ViewModel design patterns. Rather than having to test via screen-scraping tools that work with the GUI, you can test at the Model API, which makes it a lot easier to automate tests. You still have to test the View and Controller, Presenter, or ViewModel to make sure they accurately reflect the Model, but the effort is a lot less.

Debugging flags can be helpful too. I mentioned them here in the context of shipping with partially-implemented features, but they are also useful for displaying information that the customer normally does not want to see, but can be useful for debugging and testing. When the customer calls in with a problem, you tell them to turn on this debugging flag and send you the output, which makes it a lot easier to triage the problem remotely. And during testing, you can turn them on to see intermediate results that help you know the product is operating properly. And since you have followed the Single-Responsibility Principle, you can be pretty confident that turning on the debugging flags will not suddenly make the product act different.

Finally, automated test runners, as have been made popular by Test-Driven Development, let you run tests automatically whenever a module changes. Some examples of these are Jenkins (formerly known as Hudson) and TeamCity.

The Quality Control Inspector of the 1970s has given way to the Quality Assurance Designer of the new millennium, who is less concerned with finding bugs, and more concerned with making sure they do not happen.

Details

Created: 16 February 2016

Details

Created: 03 March 2015

When I was a young programmer, a colleague was writing an engine-test system to run on a minicomputer. The only available high-level language was FORTRAN II.

Edsger Dijkstra had just published his "Go-to statement considered harmful" paper, and there was much talk about how you could not code in FORTRAN without GOTOs, since it was not a block-structured language. (Modern FORTRAN is a completely different story. As they say, "It's not your grandfather's FORTRAN.")

Unconcerned, my colleague designed his programs using Nassi-Shneiderman structured flowcharts, and coded them using GOTOs only to build the structured-programming constructs he needed. Higher-level languages had block-structured constructs built in, which made structured programming easier, but given sufficient desire and self-discipline, it was possible to achieve similar results. (I used this approach for many years in coding assembly language.)

Object-oriented languages brought us encapsulation, where you can instantiate a class and call its methods, without ever being able to see its internal data. In 1967, George Dodd, of General Motors Research Labs, developed a package called APL ("Associative Programming Language") for processing computer-graphics data. (Within GM, when someone mentioned APL, the listener immediately responded "Dodd's or Iverson's?") Upon first calling APL, you were given a handle, which you passed in on subsequent calls. The data structures APL used were pointed to by the handle, but they were opaque to the caller. This was encapsulation, a dozen years before Bjarne Stroustrup started his work on C++.

A few years ago I wrote some new code for an assembly-language system, and I wanted to use Test-Driven Development. There was nothing like JUnit or NUnit or UUnit available for assembly language, so I wrote my own simple test harness, and structured my internal routines to facilitate independent testing. I was amazed by the number of small errors I caught during the red-green-refactor process and was happy to discover them now, rather than waiting for a customer to find them. The benefits were definitely worth the effort, even though I would have preferred to use a higher-level language.

In all these cases, it was possible to get by without modern tools. No doubt, tools can make things easier. When you code in an object-oriented language, your focus stays on the problem being solved and the structure of the solution, without getting distracted by which hardware instruction to use (something that modern compilers are more qualified to judge than we are). Test runners like JUnit, NUnit and UUnit take the drudgery out of unit testing, making TDD much smoother. But they are just tools.

Sometimes I think about the cartoons I watched as a child. Today, computers can take a starting and ending pose and fill in all the frames in-between. Back then, each frame, 24 per second, was drawn by hand. It is hard to imagine anything more tedious. But with all that drudgery, they were still able to tell stories.

Tools are good, because they help eliminate mistakes and remove drudgery. Any time you can remove drudgery, you reduce the risk that you will say "I'm really busy, I'll just skip the tests this one time." But you don't always need them.

This applies to agile development tools as well as engineering tools. I get unhappy whenever I hear a Scrum team say "We can't do this because Version One, or Rally, or RTC, doesn't support it." The tool's job is to make our work easier and better, if we have to bend up what we do to suit the tool, the tool is not doing it's job, and we need to find, or make, a different tool.

Remember, it's just a tool.