Article on Whyteboard in Dec 2010 Linux Journal

An article has been published in Issue 200, December 2010 of the Linux Journal magazine, in the "new projects" section. Praise was given for its simple UI, PDF annotating abilities, the history replay view and being able to embed multimedia into a sheet.

Having a simple website

The Whyteboard home page is simple. Very simple. I was going to create a "modern" site, with some nice javascript goodness and whatnot, but I have trouble creating a website from blank. I'm not much of a design guy - I do a little CSS at work, but not in a few months, and it's always been making changes to existing sites.

So, I've gone the complete opposite and done the most basic site I could. It's pretty much one page with some summarised information about the program.

Whyteboard 0.41 released!

Hurrah - finally got a release by the end of the month as I was hoping to do.

This version should finally kill off all unicode related bugs - in total, over 15 bugs have been fixed. I'm embarrassed that they slipped through my radar - but at least they're now fixed.

What's new in this release?

- Program now reads in your system language at startup and sets the default locale based on that. No more setting preferences!
- Many enhancements to the "Shape Viewer" dialog
- Misc. UI improvements such as little * shown in the program title when there's unsaved changes.

- Massive code overhauls and changes to make the program easier to develop on



Wow! Looking back it seems like there's not much new...

Download at http://code.google.com/p/whyteboard/downloads/list

Packaging her up

I've decided to try and package my classes into more appropriate locations instead of containing everything into one directory. I've just committed a repackaging of my GUI classes. I'd like to go further and split each file into smaller modules, with one classes each, especially for my "Tools".

However doing this will definitely break older save files and I'll have to monkey patch around it. I think I should move away from using Pickle as my save format, and perhaps use JSON instead. I just need to find a way to make a save converter for existing files...

Refactoring, refactoring

I've spent some time refactoring the program's code lately. The GUI class is getting really bloated with features and functionality that should be handled by a separate class. Thus, I've created a Menu class to create, bind and query the menu. Pretty good...

Also in the utility class, my save method was growing to over 100 lines, with many temporary variables being used. I managed to extract this into its own class, and split up the method into several smaller ones, passing parameters between them as needed. My temporary variables from the giant class became instance variables in the extracted class.

The end result is more code, but ultimately an easier to read, more abstract higher level save method - you just read the method calls it makes, as opposed to having to iterate over objects, save values etc inside the method.

I need to apply this to many more places in the program. I want to refactor out tab control to its own class, so that it can be shared by the Notes, thumbs and gui classes; these all share common data but it's duplicated per-class. Also the code is really hard to test - refactoring it out will definitely help.

Understanding Python decorators

I've finally come to grasp with Python's decorators, and have managed to create somewhat nicer code. Here's what I had before:

    def on_top(self, event):
        index, item = self.find_shape()
        self.shapes.pop(index)
        self.shapes.append(item)
        self.populate()
        self.list.Select(0)


    def on_bottom(self, event):
        index, item = self.find_shape()
        self.shapes.pop(index)
        self.shapes.insert(0, item)
        self.populate()
        self.list.Select(len(self.shapes) - 1)


    def on_up(self, event):
        index, item = self.find_shape()
        self.shapes.pop(index)
        self.shapes.insert(index + 1, item)
        x = self.list.GetFirstSelected() - 1
        self.populate()
        self.list.Select(x)


    def on_down(self, event):
        index, item = self.find_shape()
        self.shapes.pop(index)
        self.shapes.insert(index - 1, item)
        x = self.list.GetFirstSelected() + 1
        self.populate()
        self.list.Select(x)

As you can see - a lot of repeated code, with only different things happening in the middle of each function. This is ideal for a decorator. I struggled for a while to get it working as a decorator function inside a class, before reading how you define a wrapper function for creating a closure over the decorated function.

My code now reads,

    def move_shape(fn):
        """
        Passes the selected shape and its index to the decorated function, which
        handles moving the shape. function returns the list's index to select
        """
        def wrapper(self, event, index=None, item=None):
            index, item = self.find_shape()
            self.shapes.pop(index)
            x = fn(self, event, index, item)

            self.populate()
            self.list.Select(x)
        return wrapper

    @move_shape
    def on_top(self, event, index=None, item=None):
        self.shapes.append(item)
        return 0

    @move_shape
    def on_bottom(self, event, index=None, item=None):
        self.shapes.insert(0, item)
        return len(self.shapes) - 1

    @move_shape
    def on_up(self, event, index=None, item=None):
        self.shapes.insert(index + 1, item)
        return self.list.GetFirstSelected() - 1

    @move_shape
    def on_down(self, event, index=None, item=None):
        self.shapes.insert(index - 1, item)
        return self.list.GetFirstSelected() + 1

Unfortunately the event parameter is there because these are also wxPython event listeners, and I'd have to create delegate functions just to remove the one parameter, which seems pointless.

More pubsub calls

PubSub is a great Python package allowing you to decouple your objects from one another. I've changed Whyteboard to use more pubsub message passing instead of direct method invocation; it's allowed me to test the application easier and to also chain multiple actions from a single message (e.g. "change tab" updates the thumbnail, bolds its text and changes the notes' tree control selection

Eventually none of my objects will know about any of its interaction objects, except for the GUI, which will mediate method calls. Perhaps I could even put this mediator outside of the GUI so that the GUI only creates dialogs and other GUI components and handles menus' events by delegating it to another object


My main problem is the models sometimes need to know the state of the GUI or its sub-components: e.g. the canvas may wish to know if the drawing mode is set to transparent - which is stored in gui.util.

Eventually the canvas will have no reference to the GUI at all, so it needs to know a way to query the current state of the program. Hmm...

Note: test before release

Embarrassing...released version 0.40 with about 6 bugs that I found within a 10 minute play around with the program. For shame!

Anyway, that takes care of that, I hope. Version 0.40.1 released Had another bug report about damn unicode errors - thought I'd fixed those! oh dear

Whyteboard 0.4 released

New whyteboard released, bringing a whole new bunch of features.

New features:
- Highlighter tool
- Improvements to selection tool
- UI changes
- Better shape viewer
- PDF Cache viewer
- bug fixes and misc. changes/improvements - see changelog for further details

Enjoy. As always, report any bugs/issues/suggestions.

The Law of Demeter

I've been learning a lot about refactoring over the past few days, and have made an effort to put these practices into effect. One interesting concept is the Law of Demeter - classes should know as little about their interacting classes as possible. This means restricting access on instance variable, and method arguments.

Suppose we have an instance variable of type B inside A. Class B contains a reference to class C. The Law of Demeter states we should not access properties of C through B; instead we should tell B on how to interact with C.

Here's a before/after example of some code from Whyteboard I've modified, involving selecting a shape using the Select tool:

class Select:
    def check_for_hit(self, shape, x, y):
        """
        Sees if a shape is underneath the mouse coords, and allows the shape to
        be re-dragged to place
        """
        found = False
        handle = shape.handle_hit_test(x, y)  # test handle before area

        if handle:
            self.handle = handle
            found = True
        elif shape.hit_test(x, y):
            found = True

        if found:
            self.canvas.overlay = wx.Overlay()
            self.shape = shape
            self.dragging = True
            self.offset = self.shape.offset(x, y)

            if self.canvas.selected:
                self.canvas.deselect()
            self.canvas.selected = shape
            shape.selected = True

            pub.sendMessage('shape.selected', shape=shape)
        return found


class Gui:
    # bind a handler for shape.selected
    pub.subscribe(self.shape_selected, 'shape.selected')

    def shape_selected(self, shape):
        """
        Shape getting selected (by Select tool)
        """
        x = self.canvas.shapes.index(shape)
        self.canvas.shapes.pop(x)
        self.canvas.redraw_all()  # hide 'original'
        self.canvas.shapes.insert(x, shape)
        shape.draw(self.canvas.get_dc(), False)  # draw 'new'

        ctrl, menu = True, True
        if not shape.background == wx.TRANSPARENT:
            ctrl, menu = False, False

        self.control.transparent.SetValue(ctrl)
        self.menu.Check(ID_TRANSPARENT, menu)

In this code, we see the Select tool (a "model" is manipulating the canvas directly, when it really shouldn't be doing so. We can delegate this to the canvas itself, through the GUI.

class Select:
    def check_for_hit(self, shape, x, y):
        """
        Sees if a shape is underneath the mouse coords, and allows the shape to
        be re-dragged to place
        """
        found = False
        handle = shape.handle_hit_test(x, y)  # test handle before area

        if handle:
            self.handle = handle
            found = True
        elif shape.hit_test(x, y):
            found = True

        if found:
            self.shape = shape
            self.dragging = True
            self.offset = self.shape.offset(x, y)
            pub.sendMessage('shape.selected', shape=shape)
        return found


class Gui:
    # bind a handler for shape.selected
    pub.subscribe(self.shape_selected, 'shape.selected')

    def shape_selected(self, shape):
        """
        Shape getting selected (by Select tool)
        """
        self.canvas.select_shape(shape)

        ctrl, menu = True, True
        if not shape.background == wx.TRANSPARENT:
            ctrl, menu = False, False

        self.control.transparent.SetValue(ctrl)
        self.menu.Check(ID_TRANSPARENT, menu)


class Canvas:
    def select_shape(self, shape):
        """Selects the selected shape"""
        self.overlay = wx.Overlay()
        if self.selected:
            self.deselect_shape()

        self.selected = shape
        shape.selected = True
        x = self.shapes.index(shape)
        self.shapes.pop(x)
        self.redraw_all()  # hide 'original'
        self.shapes.insert(x, shape)
        shape.draw(self.get_dc(), False)  # draw 'new'

We can now see that the appropriate classes are performing their correct responsibilities. The model sends out a message to indicate something's changed, without caring how the event is handled, and the GUI is updating its controls and menus. The GUI delegates a method call to the canvas which updates itself.

There are no longer many "access levels" (this.that.other.do_something) as all operations are on variables within a small scope. This is better code that's easier to test.

New release almost ready

Since the last post, development has steadily resumed again. Over 30 commits have been made, fixing bugs and adding in a bunch of new features. One of the new things I've addded is when you are moving a shape using the Select tool, moving the shape close to the canvas' edge will scroll the canvas in that direction, so that you don't have to stop dragging the shape, scroll the canvas manually and resume dragging.

There's also a new feature that allows you to move selected shapes using the keyboard arrow keys. This integrates nicely with the above "canvas edge" auto-scroll. The shape viewer has gained a "delete" button to remove un-selectable shapes such as a pen/eraser.

The shape viewer has gained several improvements too, such as better synchronisation with the program as a whole. Undo/redo/deleting shapes/renaming a sheet all trigger the viewer to update itself and shapes report their properties more accurately.

A PDF Cache viewer allows you to see cached pdf/svg/ps files, and remove them from the cache, forcing Whyteboard to re-convert the item. The date a file is converted is also stored into the cache so you know when it was last converted.

Recently Closed Sheets is implemented as a sub-menu - as well as being able to re-open the last closed sheet with Ctrl+Shift+T, you can browse through the sub-menu and choose which sheet to close.

and various other small tweaks/improvements.
Stay tuned for a release - there's only a few bugs to iron out now. There's been over 80 commits between the last release and the current one - a lot has changed, making the program much better, overall.

Here's 2 HD videos showing some new stuff:

Improved Shape Viewer




Shapes Scrolling the canvas

Arrested Development

Development has practically stopped lately. Been busy with work - when I get home I really don't want to spend time programming.

I plan on getting a bit of work done this weekend - it's been too long since I coded!

Finally Employed

Well, it happened. Seven months after graduating university I've secured myself a web programming job using a variety of new technologies and development methodologies. I've only previously read about most of these, e.g. Agile, Groovy, Spring/Hibernate - which I told them in the interview, and was told that they weren't looking for someone with previous experience.

Now, I have 2 weeks until I start to learn as much about these things as I can just to get a nice head start into development, and make my first few weeks a little easier because I won't be starting from scratch.

Obviously this means that time spent to develop on Whyteboard will become limited; working full-time as a programmer, I may not *want* to program much in my spare time. But who knows, I do enjoy development and I'm definitely not putting the project on hold. Progress will continue, but slowly.

Goals: release Whyteboard 0.39.4 in the next 2 weeks. Learn Groovy, Grails and spend some time unit testing Whyteboard more.

Lack of Testing: An Example

In my previous post I was saying how it's hard to test my code independently of the GUI Framework, since my application is heavily dependant upon the services it provides. Here's an example, exporting an image (which is represented in my program as a buffer that is drawn to)


Here's the basics:
- prompt user for location to export to.
- display a filedialog containing a file filter for image types
- check whether the given filetype is acceptable
- if there is no filetype present, append the currently selected file filter file extension
- save the data

    def on_export(self, event=None):
        """Exports the current sheet as an image, or all as a PDF."""
        filename = self.export_prompt()
        if filename:
            self.util.export(filename)


    def export_prompt(self):
        """Returns the filename to save to"""
        val = None  # return value
        wc =  ("PNG (*.png)|*.png|JPEG (*.jpg, *.jpeg)|*.jpeg;*.jpg|" +
               "BMP (*.bmp)|*.bmp|TIFF (*.tiff)|*.tiff")

        dlg = wx.FileDialog(self, _("Export data to..."), style=wx.SAVE |
                             wx.OVERWRITE_PROMPT, wildcard=wc)
        if dlg.ShowModal() == wx.ID_OK:
            filename = dlg.GetPath()
            _name = os.path.splitext(filename)[1].replace(".", "")
            types = {0: "png", 1: "jpg", 2: "bmp", 3: "tiff"}

            if not os.path.splitext(filename)[1]:  # no extension
                _name = types[dlg.GetFilterIndex()]
                filename += "." + _name
                val = filename
            if not _name in meta.types[2:]:
                wx.MessageBox(_("Invalid filetype to export as:")+" .%s" % _name,
                              _("Invalid filetype"))
            else:
                val = filename

        dlg.Destroy()
        return val


#------ the util.export() method:


    def export(self, filename):
        const = get_wx_image_type(filename)
        self.gui.board.deselect()
        self.gui.board.redraw_all()

        context = wx.MemoryDC(self.gui.board.buffer)
        memory = wx.MemoryDC()
        x, y = self.gui.board.buffer.GetSize()
        bitmap = wx.EmptyBitmap(x, y, -1)
        memory.SelectObject(bitmap)
        memory.Blit(0, 0, x, y, context, 0, 0)
        memory.SelectObject(wx.NullBitmap)
        bitmap.SaveFile(filename, const)  # write to disk


def get_wx_image_type(filename):
    """
    Returns the wx.BITMAP_TYPE_X for a given filename
    """
    _name = os.path.splitext(filename)[1].replace(".", "").lower()

    types = {"png": wx.BITMAP_TYPE_PNG, "jpg": wx.BITMAP_TYPE_JPEG, "jpeg":
             wx.BITMAP_TYPE_JPEG, "bmp": wx.BITMAP_TYPE_BMP, "tiff":
             wx.BITMAP_TYPE_TIF, "pcx": wx.BITMAP_TYPE_PCX }

    return types[_name]  # grab the right image type from dict. above

meta.types is a Python list of valid filetypes, e.g. ['jpg', 'png', 'bmp'...]

Now, sure, I could go in an test that given a particular filetype, the MessageBox is displayed (or not), or that entering no file name appends the correct extension.

The easiest way for me to test this is to draw a bunch of lines, and then try exporting it. I verify the file dialog appears, the file type filter works. I try selecting PNG, and typing in test. I see a file named test.png is created, I open it, it matches my image. I repeat the process, typing test2.png. File test2.png is created.

Writing mock GUI dialogs to return fake values and doing this in a TDD manner just seems like it wouldn't have helped much. I know there's a bit of code repeat (in get_wx_image_type) and this may not be the best example.

On testing, and attempting test-driven development

I want to unit test. I really do. I find it pretty hard to start with adding a bunch of tests to existing code that you know is working, to some extent. My application relies heavily on the GUI, and the GUI itself cannot be tested, and has to be mocked with fake classes returning mock data.

This of course has its downsides. I've found myself just writing my program's code in an ad-hoc manner, with some thought as to what I want to achieve, but not necessarily how I'm going to get there. I'll try some things that seem to work, and begin building around this. I notice common code and refactor it into more readable, less copy/pasted methods.

But, I still run into issues from time to time. My code's main problem is its tight object coupling -- what should be my "Model"s know about their Controller/View, and explicitly call methods on them. I'm slowly changing this by using the Publish/Subscribe pattern, where a message is broadcast through my system and is handled by any interested listeners, instead of calling methods directly on the controllers.

But, this is going to take a long time to do. I want to try test-driven development by building up a feature of code with unit tests proving that the code does what it should. However, this comes with complexity - if I want to edit the code that saves the program's data to disk, then it's hard to test if valid data is being created. Or, if I wish to test that my new drawing tool actually draws correctly.

How can a unit test verify this? The simplest way (for me) is to try drawing with the tool. I added a bug reporter to Whyteboard in October 2009 and most of the reports I've received are issues that I would have never caught with unit tests -- broken/misconfigured wxPython installations, Mac-specific issues, Unicode errors, problems with the Media player.

These all relate to environment, and as a sole developer on a limited budget (I can't test on a Mac!) there is pretty much nothing I can do but to speculate what may be causing errors based on limited data I have.

Oh well, I've totally gone off the subject of this post. I will give TDD a shot, just to see how beneficial it is. But, my program needs a change to become more testable. Making these changes will undoubtedly introduce regressions that will go undetected due to the lack of current tests!