How to Faster Find a Bug

UI Web Test in Golang - Part 2

After being very unhappy with browser based testing, I’ve switched to Go based HTML testing some time ago, and I’m very happy overall with the results. Tests are no longer flaky and are much faster.

Today I introduced a bug which was detected by my web tests. Hurray! BUT I’ve got pages of output from failed tests, because in web testing, if one test fails, all the other pages are also not rendered correctly, mostly because the context hasn’t been set correctly, data isn’t there from previous pages or URLs can’t be read from a page.

It was hard to find out what was going on and find the source of the problem. With some changes, I found the bug, fixed it and released a new version.

What changes did I make?

1. Fail early on first error / failed check

To create a page from a goquery doc I use a creator function.

func LoginPageFrom(
  t *testing.T,
  doc *goquery.Document,
) *LoginPage {
  hxPost, _ := doc.Find("form").Attr("hx-post")
  p := LoginPage{
    WebPage: WebPageFrom(doc),
    HxPost: hxPost,
  }
  return &p
}

and later I call a check function on the page to check if everything is correct

func (p *LoginPage) Check(t *testing.T) {
  assert.Equal(t, "Login | Inkmi", p.Title)
  assert.Equal(t, "/u/login", p.HxPost)
}

When one page in a flow fails, usually all the others fail too. So you get lots of output of failed tests, and it is difficult to find out what has happened. To have less output, it’s better to fail all tests if one test breaks and stop executing all the other tests. You can use t.FailNow() for this or panic as I’ve done. I’ve changed the code for Check() to stop all tests if one assert fails:

func (p *LoginPage) Check(t *testing.T) {
  ok := assert.Equal(t, "Login | Inkmi", p.Title)
  if !ok {
    panic("`Login | Inkmi` not in title")
  }
  ok = assert.Equal(t, "/u/login", p.HxPost)
  if !ok {
    panic("/u/login not HxPost URL")
  }
}

This way the first failure stops test execution, and it is much easier to find the cause of the failing tests.

2. Have asserts not only in checks but in construction

Still, this isn’t early enough. So I’ve changed the construction of the page to fail when it has the wrong data. If a page expects to find an hx-post and there is none, it now fails already with construction, not later with checking.

func LoginPageFrom(
  t *testing.T,
  doc *goquery.Document,
) *LoginPage {
  hxPost, _ := doc.Find("form").Attr("hx-post")
  ok := assert.NotEmpty(t, hxPost)
  if !ok {
    panic("hx-post does not exist.")
  }
  p := LoginPage{WebPage: WebPageFrom(doc),
    HxPost: hxPost,
  }
  return &p
}

3. Hand over meta-information so you can print it

To make it easier to understand the flow and context, I’ve added meta-information to the construction of the pages. In this case, the sourceUrl which came before this page.

func LoginPageFrom(
  t *testing.T,
  doc *goquery.Document,
  sourceUrl string,
) *LoginPage {
  hxPost, _ := doc.Find("form").Attr("hx-post")
  ok := assert.NotEmpty(t, hxPost)
  if !ok {
    Debug(doc, sourceUrl)
    panic("hx-post does not exist. Source: " + source)
  }
  p := LoginPage{WebPage: WebPageFrom(doc),
    HxPost: hxPost,
  }
  return &p
}

Adding a sourceUrl to the function call makes it easier to understand where the page came from (what was it’s predecessor).

4. Print formatted HTML for debugging

And I’ve also added a Debug call that prints source HTML with a-h/htmlformat to find the reason why hx-post is empty.

func LoginPageFrom(
  t *testing.T,
  doc *goquery.Document,
  source string,
) *LoginPage {
  hxPost, _ := doc.Find("form").Attr("hx-post")
  ok := assert.NotEmpty(t, hxPost)
  if !ok {
    Debug(doc, source)
    panic("hx-post does not exist. Source: " + source)
  }
  p := LoginPage{WebPage: WebPageFrom(doc),
    HxPost: hxPost,
  }
  return &p
}

Results

With this in place, it was a breeze to find the problem, fix it and release.