You are here

Testing Questions

2 posts / 0 new
Last post
andysks
Testing Questions

Hi all. Starting new topic because it's quite a generic question and I'd like to get many replies. 

 

Every time I decided to give my campaign for testing, the method was that the people would reply by messages and I would reply back with a link to some patch or a workaround. Needless to say, although it started good, it ended up being confusing along the way. So I am trying to figure out what the best way to handle beta tests is.

 

First of all, I was calling all tests until now beta, but they were more like aplhas. This one that I want to conclude now is a campaign at its very final stages.

 

At the moment I have a couple of close friends who will test for me, but I can never have enough. I want this to be the last test. This is how I am thinking of doing it.

Create an online excel where the various testers will report, and a link to the patch hak download. That way the testers can always look if there is a latest  patch available before a session, and also not get duplicate issues.

Give an open link to a cloud service where the campaign is free to download. I know some of you guys have done such open tests. How was the response?

 

Do you see any problems to such an approach? What would you recommend so that this last test goes as smooth as possible?

  • up
    100%
  • down
    0%
Lance Botelle

Hi Andy,

As you are aware, I handled testing with a few testers (including yourself), and here are my experiences and thoughts on the matter:-

1) Good communication is absolute paramount.

That is the one and only golden rule, but as far as implementation, that will vary depending upon exactly what part of the module/campaign is being fixed.

For instance, if the fix can be handled by the "patching" system I devised and we used at the time of testing my own module, then all is well and good, and any testers can make use of the patches. However, if you make "module" changes (e.g. area design changes, or even conversation changes), then things start to become more difficult, expecially if you have testers who differ between "module" related differences.

EXAMPLE:

If you release test module 1 with various patches, and then release test module 2, which also had an area design change. And you have two testers who started testing from module 1, but another tester came along and started testing from test module 2, then you now have to support two lines of testing.

At one stage of testing my own module, I had about three module versions that needed accomodating. That is when things get difficult to manage (as you say).

So, the answer for ease would depend on how much you can rely on your "final" module build for testing. If you can manage all testing via one "latest" module build, then patching the rest during play would be the easiest way to manage it. And on that basis I would ONLY provide the *same* module version for any testers late to the testing rather than provide them with the latest version. i.e. If you adjust the module while testing, make sure you have an "unadjusted version" (the same as every other tester) to provide with for testing that has access to the *same* patches as every other tester.

Reported problems can be noted any way you like. Personally (as you know), I preferred to keep in touch via grouped emails that contained all the latest reports and patch links. I did it this way because at least I knew they would have "received" (in theory) the latest info, rather than risk they may have missed the latest update somehow.

By using group emails, you can also provide links that only they can refer to, so as to prevent the code falling into hands before it's ready for final release.

On a final note: I think we all need to say "alpha" when we first release something for testing. My own experience taught me that too. :) I was shocked at how many things I had "assumed" were done, to find out were not. Even NOW, a year or so on, I am only now saying I am finally fixing all those parts that went under the radar. e.g. Weather issue ... auto-inventory controls ... i.e. Things that testers may not have even realised were problems, but only the builder could perhaps see.

Cheers, Lance.

The World of Althéa Blog: http://worldofalthea.blogspot.co.uk/

  • up
    100%
  • down
    0%