Lack of Testing: An Example

In my previous post I was saying how it's hard to test my code independently of the GUI Framework, since my application is heavily dependant upon the services it provides. Here's an example, exporting an image (which is represented in my program as a buffer that is drawn to)


Here's the basics:
- prompt user for location to export to.
- display a filedialog containing a file filter for image types
- check whether the given filetype is acceptable
- if there is no filetype present, append the currently selected file filter file extension
- save the data

    def on_export(self, event=None):
        """Exports the current sheet as an image, or all as a PDF."""
        filename = self.export_prompt()
        if filename:
            self.util.export(filename)


    def export_prompt(self):
        """Returns the filename to save to"""
        val = None  # return value
        wc =  ("PNG (*.png)|*.png|JPEG (*.jpg, *.jpeg)|*.jpeg;*.jpg|" +
               "BMP (*.bmp)|*.bmp|TIFF (*.tiff)|*.tiff")

        dlg = wx.FileDialog(self, _("Export data to..."), style=wx.SAVE |
                             wx.OVERWRITE_PROMPT, wildcard=wc)
        if dlg.ShowModal() == wx.ID_OK:
            filename = dlg.GetPath()
            _name = os.path.splitext(filename)[1].replace(".", "")
            types = {0: "png", 1: "jpg", 2: "bmp", 3: "tiff"}

            if not os.path.splitext(filename)[1]:  # no extension
                _name = types[dlg.GetFilterIndex()]
                filename += "." + _name
                val = filename
            if not _name in meta.types[2:]:
                wx.MessageBox(_("Invalid filetype to export as:")+" .%s" % _name,
                              _("Invalid filetype"))
            else:
                val = filename

        dlg.Destroy()
        return val


#------ the util.export() method:


    def export(self, filename):
        const = get_wx_image_type(filename)
        self.gui.board.deselect()
        self.gui.board.redraw_all()

        context = wx.MemoryDC(self.gui.board.buffer)
        memory = wx.MemoryDC()
        x, y = self.gui.board.buffer.GetSize()
        bitmap = wx.EmptyBitmap(x, y, -1)
        memory.SelectObject(bitmap)
        memory.Blit(0, 0, x, y, context, 0, 0)
        memory.SelectObject(wx.NullBitmap)
        bitmap.SaveFile(filename, const)  # write to disk


def get_wx_image_type(filename):
    """
    Returns the wx.BITMAP_TYPE_X for a given filename
    """
    _name = os.path.splitext(filename)[1].replace(".", "").lower()

    types = {"png": wx.BITMAP_TYPE_PNG, "jpg": wx.BITMAP_TYPE_JPEG, "jpeg":
             wx.BITMAP_TYPE_JPEG, "bmp": wx.BITMAP_TYPE_BMP, "tiff":
             wx.BITMAP_TYPE_TIF, "pcx": wx.BITMAP_TYPE_PCX }

    return types[_name]  # grab the right image type from dict. above

meta.types is a Python list of valid filetypes, e.g. ['jpg', 'png', 'bmp'...]

Now, sure, I could go in an test that given a particular filetype, the MessageBox is displayed (or not), or that entering no file name appends the correct extension.

The easiest way for me to test this is to draw a bunch of lines, and then try exporting it. I verify the file dialog appears, the file type filter works. I try selecting PNG, and typing in test. I see a file named test.png is created, I open it, it matches my image. I repeat the process, typing test2.png. File test2.png is created.

Writing mock GUI dialogs to return fake values and doing this in a TDD manner just seems like it wouldn't have helped much. I know there's a bit of code repeat (in get_wx_image_type) and this may not be the best example.

On testing, and attempting test-driven development

I want to unit test. I really do. I find it pretty hard to start with adding a bunch of tests to existing code that you know is working, to some extent. My application relies heavily on the GUI, and the GUI itself cannot be tested, and has to be mocked with fake classes returning mock data.

This of course has its downsides. I've found myself just writing my program's code in an ad-hoc manner, with some thought as to what I want to achieve, but not necessarily how I'm going to get there. I'll try some things that seem to work, and begin building around this. I notice common code and refactor it into more readable, less copy/pasted methods.

But, I still run into issues from time to time. My code's main problem is its tight object coupling -- what should be my "Model"s know about their Controller/View, and explicitly call methods on them. I'm slowly changing this by using the Publish/Subscribe pattern, where a message is broadcast through my system and is handled by any interested listeners, instead of calling methods directly on the controllers.

But, this is going to take a long time to do. I want to try test-driven development by building up a feature of code with unit tests proving that the code does what it should. However, this comes with complexity - if I want to edit the code that saves the program's data to disk, then it's hard to test if valid data is being created. Or, if I wish to test that my new drawing tool actually draws correctly.

How can a unit test verify this? The simplest way (for me) is to try drawing with the tool. I added a bug reporter to Whyteboard in October 2009 and most of the reports I've received are issues that I would have never caught with unit tests -- broken/misconfigured wxPython installations, Mac-specific issues, Unicode errors, problems with the Media player.

These all relate to environment, and as a sole developer on a limited budget (I can't test on a Mac!) there is pretty much nothing I can do but to speculate what may be causing errors based on limited data I have.

Oh well, I've totally gone off the subject of this post. I will give TDD a shot, just to see how beneficial it is. But, my program needs a change to become more testable. Making these changes will undoubtedly introduce regressions that will go undetected due to the lack of current tests!