Wednesday, 23 April 2014

I'm looking for team members for the test lab #letstest2013

So I wanted to do a testlab session at the let's test conference...

The apps were released before the conference:

XBMC and Mumble

Although, mumble sounded good I wanted to have a crack at XBMC (basically because I didn't need headphones)

Before the conference I started thinking about how I might test the software...

Bizarrely, I started thinking about a circus...

Different circus formats

Different circus sizes (performers)
Different platforms (tents)
What's the experience like?
 - Show
 - Seating
 - Viewing
Customers,Types of customers

My first opportunity was thwarted by a rather good nature walk...

My second opportunity came by chance...Richard Robinson happened to be looking for team members to test in the lab....he knew Guy Mason who also knew that I wanted to test in the test lab... The end result...both of us were recruited to the team...yey!

And so I had my chance....

We were also joined by Ida Waller and Erik Brikarp. :-)

So the team comprised the following:
Erik Brikarp (
Ida Waller
Richard Robinson (
Guy Mason (

Rich did a great job of briefing the team, outlining the challenge but giving us the freedom to test what or how we wanted...kool!

Of course I had a lot of questions but soon realized I just needed to get on testing...

Oh and my circus pre-test lab ideas just seemed to go out the window....eek...

Armed with my printed copy of the little black book of test design just in case, I started...

Rich encouraged us to come up with some test ideas and then explore the ideas that we came up with...

I noticed XBMC had power options so started exploring those....

As I explored those I sketched down the power options as I went.
(this would become an important reference later)

I also seemed to want to talk to someone about the things I was discovering (Have you seen this? what do you think of this? Does this look right to you? Do you think it should do this?) 

As you can see there are a few power options, I started exploring those one by one and then I selected 'shutdown'. What did it do? Yes It powered off my laptop!

Now that might seem obvious, I'm in 'power options' and I selected 'shutdown' and I got shutdown...
I thought at least I might get an 'Are you sure you want to shutdown?' popup message with 'Yes' and 'No' options. So I thought bug....

By the time my laptop powered up again, and restarted the app it was nearly time for the end of the test lab session.......what?!!!!

So what do I do...I try and do the same thing I just did again.... select 'shutdown' (I would like to see if this is reproducible before I raise a bug)....but... I noticed when I had restarted the app the number of power options available to me had changed....hmmm...

What have I got......near the end of the session, no 'shutdown' bug raised, not reproducible and a potential new bug. I restart the app and my machine. Still the number of power options are still not the same as the first time I ran the app....hmmm..

2 potential bugs....and about 2 mins until the end of the session....

Dilemna 1, what to raise....

I quickly thought, there's two issues here:

  • One for no prompt when selecting shutdown (what if I select 'shutdown' by mistake...and have those really important test notes open)
  • The other, that the power options seemed to have changed once I had shutdown (also note this may not be a bug - there may be a reason for this, with the absense of being able to chat to a programmer/stakeholder etc I thought I would raise it anyway)

Dilemna 2, Quality of work dilema.

With almost no time left.

  1. Do I quickly raise a bug just to get it logged but not be happy with the level of detail of the bug?
  2. Or not raise a bug at all as I couldn't provide the right level of detail that I would like in the bug report.

I opted for option 1, at least get something raised, even if it's not the greatest bug report in the world (I'm really disappointed in what I reported..grrr).
I thought it would be better for someone knowing about a bug rather than not knowing...

I enjoyed the debrief amongst not only our team but also the all the teams that tested in the testlab.

Even though I didn't feel I tested that well in the session I thoroughly enjoyed the experience
Thank you Rich and thank you team, also thanks to Martin Jansson and James Lyndsay for running the test lab.

Awesome testers.....awesome people! I hope we could do it again!

I must become better at testing in the test lab.

I must become a better tester