I’ve been up to my neck in code the last little while — mostly poking around in the graph drawing classes. I have noticed a lot of drawing bugs in the graph, and I decided it was time to put my refactoring hat on while I was in there.

I write a lot of non-object-oriented C code, and rarely worked in a role where my primary responsibility involved writing up class hierarchies. Most of my OO knowledge comes from these side-projects that I work with, and projects I did in school.

So, object-oriented design doesn’t come very naturally for me. As a result, I tend to start out with a procedural style, but wrapped up inside classes. Then, the design evolves.

FuzzMeasure 1.0 was (mostly) one single class, with no document-based structure to speak of. It consisted of about 20 or so source files, with a lot of weight on the main window controller. FuzzMeasure 1.2 was almost a complete rewrite of 1.0, because I had to shift the operation into a group of classes that better utilized the document-based app organization.

Now, 2.0 carries much of the same code that 1.2 had, but I totally reorganized a lot of my signal processing code and graph drawing code. These two areas alone represent a lot of arduous design and development work.

Unfortunately, I gave up too early on my graph drawing classes when I first ported the code over from 1.2, because they worked fine. However, now I’m adding new drawing types to the graph, and the design’s not holding up very well. So, taking a page from Agile methods, I’m iterating over the design to fix it up now that I’ve encountered a problem with it.

So, as you can see, the iterative method works pretty well. I started out with a project that came naturally to me, and kept improving it over time. I released my product nice and early, and started gaining users (and, more importantly, money). Had I started out with an insane design, I would have taken much longer to release my 1.0, or I might have given up on it completely out of frustration.

Back to coding for me!

Over the last few days, while taking a bit of a break from FuzzMeasure development, I started studying up on some web development frameworks to help me with modifications I plan to make to the site. I made the standard tour around Ruby on Rails, django, TurboGears, and a few others. I watched both the Rails and TurboGears videos, and saw a good sample of TextMate in action.

I originally stumbled on TextMate a few months ago (or, possibly over a year ago — the last few years have just flown by for me). When I originally found it, I thought it’d make a great non-vi editor to use for my web development, scripting work, and, maybe even Cocoa coding. I played with it for a little while, and it didn’t really meet all my needs at the time. Also, I didn’t want to replace my perfectly-fine vi with something that would cost me $50.

So, this time around, when I finally decided to give TurboGears a whirl for my new site, I decided to give TextMate another shot. After about 20 minutes of use, I fired Allan my registration fee.

Here are the things I like about TextMate so far:

• Well-done Subversion integration
• Ability to build in Xcode, and provide clickable build output
• Live web view while editing HTML, which updates as you type
• Great built-in editing shortcuts for all the languages I use
• Automatic TODO list building (although it’s a little slow for my FuzzMeasure directory)
• An automatic terminal shortcut out-of-the-box
• The ability to just open up a directory from the terminal — no need to make projects if you don’t feel like it

Aside from the list, my favourite overall thing about TextMate is its depth. It seems that every hour I spend with it, I learn something new to help speed me up. I wanted to summarize the things I liked about it before I spent too much time with it, or else it’d likely take too long for me to scratch up this quick post about it.

Of course, it’s not without its faults. There are a few nits (partially due to bugs, and partially TextMate’s behavior that conflicts with some of my bad typing habits) that get to me, but I’ve adapted to all of them very quickly.

So, for the time being, TextMate will replace vim as long as I’m coding on my PowerBook. We’ll see how long I last…

Introduction

Because I have limited time to work on FuzzMeasure, I make sure that my development time is well-spent. Automation is a common technique that indie developers will suggest to help deal with the “one person, many jobs” struggle we all face. Much of the suggestions for automation seem to revolve around support efforts (retrieving lost license keys, building customer databases, etc).

Indie developers often go an extra step, and add automation to the process of software release. Gus and I both have automated build scripts that pull a tagged release branch from our subversion repository, build it, strip it, and then construct a disk image. If I chose to do these tasks manually, I could possibly slip up and forget one of the details. Furthermore, since releases aren’t always so close together (as much as I’d love to “release early, release often”), remembering exactly what needs to get done to build a release image gets challenging.

In general, when you automate common tasks, you get time savings and repeatability — both very important benefits to indie developers. Time savings means more time to spend working on important code changes and support issues. Repeatability provides peace-of-mind to the developer, and helps build a solid reputation externally.

So, how could we possibly save ourselves even more time? What else could we automate as developers?

What if we automate the task of writing code? I can understand if some of you don’t recognize this concept, because not everyone gets into situations where you can take advantage of automated code-generation. As usual, I’d like to use an example to help explain where this comes in handy.

A Recent Example

In FuzzMeasure, I need to add various filters for octave bands, third-octave bands, and A/C-weighting. When I started this task, I thought it might be a great idea to simply add the logic to FuzzMeasure that designs the filters based on various parameters. One of those parameters is sampling rate.

Well, writing filter design code is highly non-trivial. It involves a lot of theory I’m not willing to learn right now. Even if I did decide to spend the time writing filter design code, then I’d be wasting time on making FuzzMeasure unnecessarily flexible.

The way I got around this was to use Octave to design all my filters. The obvious benefit of this approach is that there are widely-known methods available for designing high-quality instrumentation-grade filters using Octave. I would use Octave to dump filter parameters, and write all the code to do filtering manually.

Employing Automation

After writing all the filter parameters into my code and testing if it’d work, I remembered that different sample rates warrant different filter parameters. I was half way through adding support for one other sampling rate when I realized how much time I was wasting, and how easily I could introduce mistakes.

I then spent an hour or so writing a quick set of Octave scripts that write my code. Run the script, and out comes a .m and .h file that contain my filter parameters. At the top of the script is a list of sample rates that will get dumped.

Had I decided to hand-write the code with the filter parameters, I could have easily lost about 4 or 5 hours (that’s a good, solid day of work I often get to spend on a Saturday at home). After writing the scripts to dump the filter parameters in the same way that I was hand-writing them, and testing the code, I learned that I had a bug in the way I was hand-entering the values all along!

Had I written the hundreds of lines of code by hand (about 1500), and then had to re-modify them all, it would have taken forever! The fix turned out to be a quick change to my code-writing script in Octave, and then the hundreds of correct lines came out all correctly in no time at all (ok, maybe about 20 seconds, because filter design is fairly math-intensive).

At the time I first did this (a month ago), I thought that the scripts only saved me a bit of time in one shot. When I brought home my MOTU Traveler, I realized that I had missed a potential sampling rate (176400) in my list of filter parameters. Last night I added another entry in my list of sampling rates, and my filters were all ready! It took only a minute to update everything.

Conclusion

If you find yourself in the middle of writing a bunch of code that seems repetitive, take a step back and see if you could automate the process of writing that code.

It won’t work in all cases (how I’d love a “write FuzzMeasure” script — Step 1 – Come up with app idea, Step 2 – run script, Step 3 – Profit), but there are a myriad of situations where you can save yourself a lot of time.

For example, you could easily write a script, or use a tool to generate a stubbed out .m file using only a skeleton .h file. I have tried to write some vim scripts bound to keys that try to do it, but in the end I tend to use Accessorizer. I also still tend to write a lot of that manually, because I simply haven’t gotten into the groove of using helper tools. That’s a whole other issue, though.

Introduction

We all know unit tests suck, right? However, I think they suck in the same way that writing design documents, requirements, or test plans suck. They’re all annoying to write. However, every one of them is valuable, no matter how you produce them (OmniOutliner, Excel, TextEdit, a napkin, your arm). It’s not glamorous to think of every possible execution path — but if you fail to plan, you’re planning for failure. Here I will give you a quick overview of how (and why) I write unit tests, and how to use gcov to make your unit testing more effective. Updated on 2005-11-03. See bottom of article for a list of changes.

Motivation

Many of us independent developers work in our spare time. We have to treat that spare time with respect, and fire on all cylinders during our coding sessions. I know that I, for one, do not want to waste my time monkey-testing my code after making changes to underlying frameworks. So, I decided to spend some time building a better monkey to do the job for me.

Perhaps I should rephrase that last sentence. I decided to find as many pre-built monkey parts to make the most efficient monkey possible. This monkey would make the perfect companion to my development efforts. Also, this monkey would be commanded to do his (or her — I hear she-monkeys make great testers, too) dirty work every time I build my code. The pre-built monkey parts are OCUnit (now bundled with the Xcode developer tools), and gcov (bundled with Xcode 2.1 and later).

Now, because we only have our spare time to build our robo-monkey, we have to accept that the monkey will never be perfect. We cannot expect that our monkey will find every flaw in our application, and we cannot expect that the monkey will guarantee 100% defect-free code. However, I guarantee you that the monkey will do a very good job of watching your back during the development process.

Frameworks

In FuzzMeasure, I decided to start working with frameworks for the next release. I relocated my math routines to SMUGFoundation.framework, so that I could run a plethora of sanity tests after every build. For FuzzMeasure 1.3, I took care of this using separate command-line applications, but that got very annoying to maintain over time. Also, when tests are hard to add, then we simply don’t add them.

I suggest heading over to Wolf’s site, where you can watch his awesome tutorial to help you embed your own frameworks into your Xcode project. As a starting point, rip out any code from your application that you could imagine re-using in another application, should you ever finish this one…

OCUnit

We already know how to write unit tests with OCUnit. I put this section in so that you could catch up if you haven’t already.

Note that the process is streamlined in Xcode 2.x. All you have to do is add a new Target to your framework project, and select Unit Test Bundle which you’ll find listed under the Cocoa targets. I’ll let you tinker to get this part running first. It’s pretty easy.

I’d like to provide some advice about deciding what tests to write. Surely, it’s important that you catch the edge cases (test your minimums, maximums, and outside your bounds). However, it’s important that you also exercise your knowledge about what you’re writing.

For example, how does one test Fast Fourier Transforms? I know how the FFT works, so I should be sure to test some standard behavioral edge cases. So, a unit impulse results in a flat spectrum, and a flat spectrum results in a unit impulse. Note that you have to use STAssertEqualsWithAccuracy for these sorts of tests with floating-point values.

Another suggestion is to try and write test cases that behave like your UI will. If your UI contains, say, a library with a selection of books, and you allow users to delete one, multiple consecutive, and multiple random entries, then you want to ensure your model classes can handle those deletions. This really isn’t rocket science, so I don’t need to speak too much about this.

gcov

Don’t start here until you have working unit tests, or you really know what you’re doing and feel like skipping ahead.

Create a new build configuration called “Coverage” to sit alongside your “Debug” and “Release” configurations, if you have them. I suggest copying the “Debug” target, as it has optimizations turned off already.

Now, double-click the framework target (mine’s called SMUGFoundation) and select the Build tab. Select the “Coverage” configuration from the “Configuration” popup button to make sure we don’t trash the other configurations.

Do the following:

• Check “Generate Test Coverage Files”
• Check “Instrument Program Flow”
• Add “-lgcov” to “Other Linker Flags”

At this point, your have instrumented your framework to support the auto-generation of code coverage data. When you run your unit test using the framework built in this manner, it will generate a dump of what lines of been hit (and how many times).

We can now move on to modifying the test target to actually give you the coverage reports. I chose to have gcov dump its output to the build window, just after the unit test output. Also, the results of gcov will be written to sourcefile.m.gcov in a coverage subdirectory alongside your build subdirectory, both which lie in your project directory.

If you expand the unit test target’s disclosure triangle, you should see the build phases. Double-click the Run Script build phase, which currently runs your unit tests. It should contain the following text:

 # Run the unit tests in this test bundle. "${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests" 

Modify it to include the following text, changing SMUGFoundation with your own framework’s name. This’ll look ugly, but it should copy/paste okay from Safari.

 # Run gcov on the framework getting tested if [ "${CONFIGURATION}" = 'Coverage' ]; then     FRAMEWORK_NAME=SMUGFoundation     FRAMEWORK_OBJ_DIR=${OBJROOT}/${FRAMEWORK_NAME}.build/${CONFIGURATION}/${FRAMEWORK_NAME}.build/Objects-normal/${NATIVE_ARCH}     mkdir -p coverage     pushd coverage     find ${OBJROOT} -name *.gcda -exec gcov -o ${FRAMEWORK_OBJ_DIR} {} \;     popd fi 

And that’s it! Build your unit test target with the “Coverage” configuration, and it should now output the overview of the coverage in each individual file. Yes, it’s likely to include a few shocks. I also have a few files getting 0% coverage right now, too. The monkey’s only as effective as the instructions you give him, so it’s up to you to increase testing.

If you have problems (missing .gcno files, maybe), try doing a clean and build on the framework using the “Coverage” configuration. If you’re reading this, you’re a developer and can probably figure out what’s going wrong. If you still have trouble, drop me a line.

To figure out where to increase coverage, open the coverage subdirectory in your project directory. Open one of the files using your favourite text editor (or, feel free to transcribe them to your arm — I’ll wait). The number listed in the left column indicates how many times that line of code got executed. If you see hashes, then that means the line has never been hit.

By increasing code coverage, you can have increased confidence in your automated testing. More than anything else, you will have a method for measuring how well your unit tests cover your framework code. While increasing your code coverage numbers, you will become more proficient at writing good unit tests. As you’ll learn quickly, you can never get to 100%. For example, the return nil; in an if ( ![super init] ) block may never execute.

Conclusion

I think that without code coverage, writing unit tests can be a little bit unrewarding. Without any sort of feedback about what you’re doing, you may never know how good your tests are.

Of course, code coverage isn’t perfect. If you design things incorrectly to start with, you can’t save yourself with code coverage and unit tests. However, upping your code coverage will save you from the nasty little bugs that will inevitably come back to haunt you later. For me, code coverage and unit testing is worth the extra effort.

Update (2005-11-03)

Removed the check for a specific “Library Search Paths”, as it doesn’t take into account multiple architectures and/or compilers. Just keep in mind that you need gcc4 and later to get -lgcov.

Update (2005-11-01)

Changed ${SYMROOT} to ${OBJROOT} in the script, because although they’re the same on my setup, they’re not the same for everyone. “The SYMROOT and OBJROOT build settings identify the build locations for build products and intermediate build files, respectively,” according to this link. In the case of code coverage, all your .gcno files sit alongside the intermediate .ob build files.

I love vim. I love gVim even more. I am very, *very*, fast in vim. In fact, I write all these entries in vim.

Now look at Xcode. Xcode is slick. I can do a lot of fancy IDE things in Xcode. I don’t work quickly in the Xcode editor. I probably write/modify code in Xcode at about 20–40% of my peak speed. I depend on Xcode for debugging, organizing all the UI stuff, and so on.

How does one marry the two in a way that does not suck? I would like to know where I can find helfpul hints about workflow when using Xcode and vim (even if it’s purely anecdotal). I may have to just break down and set vim to run xcodebuild at the command prompt with some keystroke, which should work well enough.