Filing useful bug reports

The below outlines what to consider when filing bug reports (especially in context of “Cloud Software” – where there are 10s of components, from user space to kernel space, interacting with each other.

This is written largely from the experience of debugging Linux virtualization sofware (KVM, QEMU, libvirt, libguestfs and related tools) and its interactions with management software (OpenStack), but the general guidance is applicable for any “sufficiently complex” software.

Structure of an effective bug report

Writing a good bug report is hard. Writing a coherent report with great reproducer details is even harder. Especially when dealing with complex software with involving several layers from kernel space to user space. So we should aim to be as meticulous as we can when filing bug reports, and help diagnose them. To that end this document outlines some, hopefully, useful suggestions when writing bug reports.

  • Summary. A concise one-line summary that describes the problem. Also spell out explicitly: if it a bug, an RFE, a “tracker bug” (that tracks mulitple bugs), and so forth.
  • Description. A clear description of the issue and its symptoms in chronological order.
  • Version details — of all software that’s relevant, from user space to kernel space.
  • Crystal clear details to reproduce the bug. (Bonus points for a providing a reproducer script.)
  • Environment details. (Don’t understimate the importance of this.)
    1. Most bugs in complex software are highly environment-specific. Clearer the details, fewer the round-trips between developers, test engineers, distribution packagers, bug triagers, etc.
    2. Call out if any special hardware is needed for testing, e.g. an exotic NAS, etc.
    3. If you altered anything – config files, code located in /usr/lib/pythonx.x/site-packages/, or any other confirationof low-level components (e.g. various libvirt config files), spell it out.
  • Actual results. Post the exact, unedited details of what happens when the bug in question is triggered.
  • Expected results. Precisely describe what is the desired behavior.
  • Additional investigative details. If you’ve done a lot of digging into an issue, writing a detailed summary (even better: a post-morten report) while its fresh in memory is very useful. Along with addition info like configuration settings, caveats, relevant log fragment, stdout or stderr of a script, or a command being executed, useful tracing details — all of which would be useful for archival purposes (and years later, it will help jog your memory).

Why?

The below might sound “obvious”, but it’s worth spelling them out:

  • Useful for new test engineers who do not have all the context of a bug.
  • Useful for documentation writers to help them write correct errata text/release notes.
  • Useful for non-technical folks reading the bugs/RFEs. Clear information saves a heck of a lot of time.
  • Useful for folks like product and program managers who are always not in the trenches.
  • Useful for downstream support organizations.
  • Should there be a regression years later, having all the information to test and reproduce makes your day.
  • Reduces needless round-trips of “NEEDINFO” between various parties.
  • Useful for new users referring to these bugs in a different context.

Overall, a fine bug diagnosing experience.

Further reading

Related reading material from other open source communities: