Visiting a crime scene and trying to figure out what happened is what a detective does. Our automation debugging also entails reflecting on the \u2018crime scene\u2019 trying to figure out what went wrong. All investigation work would have the same steps, gather data, plot the variables and evidence and deduce what might have happened, except off course I\u2019m pretty sure our automation scripts would not try to cover their tracks which makes our work infinitely easy!<\/p>\n

Picking up minute details which others might miss out and an in-depth understanding of how the world works helping with deductive analysis; These are perhaps the two most portrayed characteristics of our famous fictional character Sherlock Holmes. While automation failures \u2018investigation\u2019 might not be as difficult, but might need the same basics from a crime scene investigation.<\/p>\n

Debugging an issue would require<\/p>\n

1) Gather data on what was going on in the program when the error occurred,<\/p>\n

2) Deduce what might have caused it<\/p>\n

3) Replay the scene of crime, or as we call it \u2018reproduce the problem\u2019<\/p>\n

4) Fix the issue<\/p>\n

5) Test the fix<\/p>\n

Sherlock might have to battle his way through a lot of bad guys, we are fortunate not have such occupational hazards and therefore I will be skipping any reference to gun battles here.<\/p>\n

Gather data \u2013 the automation \u2018black box\u2019<\/h3>\n

The \u2018black box\u2019 simply put is a recording device including flight data recorder and the cockpit voice recorder on each aircraft. The most important piece of equipment in an air crash investigation is the black box, which is \u2018data\u2019. Sherlock too on every crime scene manages to pick at least one-minute detail which others might not have thought of, my point \u2013 Data is the most important aspect of an investigation.<\/p>\n

Report logs are generally used for just mentioning a passing or failing of a script, at max what step failed. Turn your automation logs \/ test report into a black box. Record all debug data in there, the way a safety critical device keeps logging all vital data at intervals and events.<\/p>\n

I stress on adding a lot of debug data within the logs – control flow details, specific data along important events like page load etc, important data being written \/ read and so on. The don\u2019t would include taking too many images, cluttering the log for other users and recording data which is not important.<\/p>\n

Deduction – Fitting in the variables<\/h3>\n

This part comes from depth of the system\u2019s understanding, and exposure to different types of issues fixed. This means it might take experience to become quick at debugging problems, however you can speed up the process. Certainly, this is the area where our detective and our automation engineer spend most of their time, putting in the pieces together.<\/p>\n

Knowing the control flow of the system is key. With that knowledge one can simply trace the main events and make sure they are coming one after another. Powered with your \u2018black box\u2019 like recording data around main events and knowledge (or detailed debug info) of what the control flow should be, isolating the problem becomes a piece of cake.<\/p>\n

In most cases folks don\u2019t have good recording mechanism (or don\u2019t want to read data available) therefore they don\u2019t start with data and skip this step altogether. Instead the move to the next step, re-running the script in debug mode. Taking a wild guess and placing a break point somewhere, then go back to multi-tasking doing something else during the debug script is running. On the break point look at a few variables and if it does not fit the hypothesis, another break point and re-run it. This is the WORST POSSIBLE way to debug IMHO.<\/p>\n

I read in Brian Tracy\u2019s book –<\/p>\n<\/div>