Constant logging

In the upcoming 1.0 release of Whyteboard, there is substantial logging across the program. This can help track down bugs when users report crashes from the program's internal error catching dialog. Whyteboard can now submit a log of the user's actions leading up to the crash as well as the stack trace. I needed the program to operate in several view states: it's always in debug mode but the debugging may not always be visible. I had to create a custom log handler to allow me to 'save' every log message in order to attach it to the error email.

from collections import deque
from logging import StreamHandler, DEBUG
from time import gmtime, strftime

class LogRemembererHandler(StreamHandler):
    '''
    A custom log handler that remembers the last x many logs sent to it
    for retrieving log messages (in this case, we want to fetch the logs 
    if the program crashes)
    '''
    _instance = None
    def __new__(cls, *args, **kwargs):
        if not cls._instance:
            cls._instance = super(LogRemembererHandler, cls).__new__(cls, *args, **kwargs)
            cls._instance.logs = deque(maxlen=101)            
        return cls._instance
    
    def emit(self, record):
        self.format(record)
        time = strftime("%d/%m/%Y %H:%M:%S", gmtime(record.created))
        self.logs.append(u"%s: %s" % (time, record.message))
        
    def get_logs(self):
        return self.logs

and to use:

        formatter = logging.Formatter('%(levelname)s %(asctime)s %(message)s')
        handler.setFormatter(formatter)
        logger.addHandler(handler)


...


        LogRemembererHandler().get_logs()

Just do it

I've started a new job as an ASP.NET developer. The development process here is much different to my previous job - no unit tests, all logic crammed into the aspx.vb files, huge methods, no ORM, ad-hoc release process and such.

Thankfully they use version control, however no bug database. I've only been working for 3 days and, while having done no coding I feel that I work better with issue tracking already, especially in a team. The current flow is bug tracking via email or verbally. I've been looking around and found that Fogbugz offers an On Demand service that lets you run Fogbugz on their servers for a free 45 day period.

In an attempt to win over my bosses to issue tracking I'm simply going to use it to track all my work. Any existing issues I come across on our sites will be logged and I may create a per-project milestone for tracking bits of code to refactor.

In regards to unit testing, I'll probably do something similar to a Visual Basic ASP.NET project at my old company where I extract our page logic into a separate DLL that can be unit tested independently, and the aspx.vb pages act as simple controllers that updates the view. I actually like this approach and don't mind doing much view-related programming in these controllers - I like the integration of control IDs to typed variables in the code-behind code.

Be wary of config settings and public source code

I forgot that a few weeks back I made a commit to Whyteboard's build procedure, that uploads a newly-created file via FTP a website, to allow Whyteboard to check if a new version has been released.

During this commit, I committed a file with my hosting username and password. Silly man!

Automating the build and release procedure

In the past I have been frustrated at the amount of time it takes for me to create a build, due to manual tasks that needed to be performed. I have finally taken a step towards an automated build process by creating a series of .bat and .sh scripts for Windows and Linux that perform the boring work for me.

There are a number of scripts, each performing different 'stages' of the build. e.g. on Windows there is build.bat, binaries.bat and release.bat.

Build: Creates the .exe from the Python source and includes all distributable resources etc that are needed into a folder.

Binaries: First, it asks for a version number. It then modifies the program's meta script that defines the version number to ensure it's correctly set.
It then calls the build script above, and then creates an installer file by launching InnoSetup as a GUI-less program. It then creates .zip files for the stand-alone exe and moves the files to a directory ready for release.

Release: It updates the file that is used by the program to look up the latest version number when doing an update check. It then FTPs this to the server.


I've added the option for Pylint to be run against all the program's codebase and output to a file; I'm figuring out how to get my unit tests running from the command line too. I'm looking into whether I can release the binaries to the several sites where I host the program via a REST API or over FTP. This would be a huge improvement.

All in all, this will help me deploy changes much quicker when releasing.

Logging - simple mistake

I recently ran into a problem where my logging started to duplicate each log message, but in a different format after one particular logging call.

I could not track down the error, until I actually read what I wrote:

import logging

logger = logging.getLogger("a.logger")

class Something:
    ...
    logging.debug("a message")

I'd accidentally called debug on the logging module, not my logger. Everything I was googling was pointing towards enabling the default logging config (e.g.

logging.basicConfig()

) but I hadn't done so.

Easy mistake to make, thought I'd briefly write about it.

Adding logging to Whyteboard

I've been undertaking quite a large change to the program: logging. Now, the program prints out a log of debug, information and warning logs to help see what's going on. I'm perhaps 25-30% complete in adding the logging, and it's already helping immensely. Just when loading a file for example, I can now see the steps the program takes in figuring out whether to convert a PDF, load a .wtbd file and then the step-by-step actions being performed there. It's great


I'm just not too sure about the best strategy for debugging things like events - e.g. in the mouse motion event handler, which gets called hundreds of times for the Pen tool, we don't want to flood the console.

Also, not too sure about what to do with logging the debug messages to file by default? One of the whole points to all of this is that I can view a user's debug log to see what they did before triggering some exception in an error report.

hmm.

Article on Whyteboard in Dec 2010 Linux Journal

An article has been published in Issue 200, December 2010 of the Linux Journal magazine, in the "new projects" section. Praise was given for its simple UI, PDF annotating abilities, the history replay view and being able to embed multimedia into a sheet.