Testing

Posted on September 8, 2015

With much effort, I have persuaded everything to build under yesod-1.4, and added enough configury that the web frontend, smtpd, and router can run independently of the live site. I’ve made port 25 work by binding an IPv6 address (I have a lot of those :-), and pointing MX records at it. The IPv4 world should be able to reach this address via the fallback MX records that point to my hosting provider’s relays. (Of course, that means that I’m not testing any IPv4 only senders, but I’m not too worried about that.)

So, how about testing this thing. Obviously I’ve sent a few messages manually from to and from another email provider. And I’m just putting back together the cabal unit tests. This now uses hspec-discover, which is a nice touch. I should write more unit tests too. But really I need a decent automated system test suite.

I did start working on an Email End-to-End tester a while ago. But apart from the fact that it hasn’t received any love for over a year, it depends on a POP or IMAP server at the receiving end.

All of which makes me think that I’m in the market, yet again, for a decent web testing solution. Approaches I’ve used in the past include JUnit, which is a) limited and b) JavaScript; and hand-coded scripts using WWW::Mechanize or similar. To be honest, something like the latter is pretty much inevitable, although my experience is that they’re tedious to set up, hard to interpret when they fail, and bitrot readily.

Wait a minute, what am I talking about? yesod-test precisely allows you to send requests and check responses. Without needing to set up an actual server, and with a test database. So potentially I can set up a db (maybe dump an actual database) and “click” round things.