tracSnap Evaluation Ideas

I'm working on tracSnap as a CSC495 credit this year. Taking a half course spread out over the whole year is a strange experience... I can never find a large block of time to work on this! I will probably get most of this work done over Christmas break, so that I can focus. So far, I've been working with Jon and Steve to come up with a plan on how to evaluate the tool. Here's what we've come up with:

1. File relation stuff: Using the file relations graph, try to find some interesting implicit file dependencies that maybe Hadley isn't aware of. And then we tell them, and ask if its useful information. This will be difficult, because there are a lot of confusing file names I will have to make sense of in order to determine which relations are the less obvious ones.
2. Social/Expertise stuff: Jon has identified tickets that seem to be defects in the code. So, if we look at the data that tracSnap had access to at the time the ticket was reported, would tracSnap have predicted which experts fixed the defect? (i.e.) Are the experts/social contacts suggested by tracSnap the same people that commented on the ticket, and helped to fix the defect?
   

I've been working on (2.) so far. I've had to rewrite a fair amount of the code in order to incorporate the necessary temporal component. (Before, tracSnap just used the entire repository, from revision #1 to the most recent revision, to make its suggestions. If we want to see if tracSnap "predicts" the right people to talk to, we can only look at the data that happened before the ticket was created.)

So I think I've got the code re-written as required, and so at the moment I'm waiting as tracSnap calculates all the file relations and experts on the Hadley data (this takes a while! 15000+ revisions!)