About Ali Khalid

This author has not yet filled in any details.
So far Ali Khalid has created 134 blog entries.

It’s automation not AutoMagic

By | May 10th, 2018|daily post|

Jim Hazen shares the story behind the term he is famous for “AutoMagic”

He was kind enough to spend some time talking about automation and it’s history

Instead of uploading the whole 2 hour long talk as one video, I’ll be creating topic wise smaller ones

If you are not introduced to Jim or the term “AutoMagic”, you would want to listen to this:

How to get around red tape processing

By | May 9th, 2018|daily post|

Stuck with processing red tape to get things done?

Here’s one way how I went around it

I was seeing a major lacking in our code coverage for a safety critical device

The only way to test it was through white box testing, having entrenched silos it was not an easy sell

Instead of selling upfront, we went covert to build a case first

Implement and generated results from the intrusive testing technique,

Once we had a business case opened it up to management and

The miracle happened, despite it seeming impossible to institutionalize, the evidence of its necessity was evident

And since then have tried this few more times with excellent results..

Give it a try next time, I’d be eager to know how it went..

Jenkins for CI like WordPress for Web development?

By | May 8th, 2018|daily post|

Remember Web front end development 15 years ago, and what has WordPress and other platforms done to it today?

The same might happen with CI processes through tools like Jenkins

The number of tools providing integrations to Jenkins is getting insane

Which is a really good thing, CI and Continuous testing is becoming way important

And getting hooks in there is not as smooth yet, but we can surely get there, probably in the near future

It might feel complex if setting up Jenkins is not approached the right way

Here’s a short guide to the very basic setup and brief description on it’s moving parts:

Jenkins – Installation Background

Most important assets when moving on from your current job?

By | May 7th, 2018|daily post|

IMHO – Lessons learned and the relationships you built.

We might have supported financial needs while being on that job

But the lessons we learned through our experiences are far more important

And even more are the relationships we built

On that note, I’d like to thank a team member of mine who just moved on..

Thank you @Arslan for your efforts and your contribution towards the team

I truly wish you all the success and hopefully

You had great experiences with us because I’m sure we build great relationships..

Seeing flakiness in your UI automation scripts?

By | May 6th, 2018|daily post|

One area I see neglected a lot is code complexity

I was fortunate to have worked for safety critical devices early on and learn some valuable lessons

One of which is code complexity (nested conditions)

When your code goes beyond a certain complexity level it becomes hard to have a deterministic result

More than often, I’ve seen a single area of code made too ccrammed and complex leading to problems all over

Spread out your conditions, instead of going deep vertically, try to go horizontal in your control flow

More on common UI problems causing flakiness:

To improve your life, what to change first

By | May 5th, 2018|daily post|

Try changing things around you or yourself?

The kind of life you are living is not because of the people around you

It’s because of how you feel about and react to life

The people, events, opportunities we see around us are a manifestation

Manifestion of our thoughts and expectations

So to change your surrounding, change your thoughts and manifest the change you want to see

Think and become the change you want to be

How to be a badass by Jensen Shero

Should we consider automation as programming

By | May 4th, 2018|daily post|

Here’s why this question comes up IMHO

For decades testers have known to be ‘Non technical’

They mostly never wanted to do much with the underlying technology

When automation came along (or when it became more popular) folks thought, well these guys know nothing of programming

So this automation thingy might not be real programming, just doing some record playback

In reality, automation is not just programming,

It IS hard core SOFTWARE DEVELOPMENT (when done right)


More here:

Does Automation Save Money?

By | May 3rd, 2018|Management|

Like lots of folks, I used to calculate automation ROI by measuring ‘hours saved’ if a person were to do these checks instead of a machine. Perhaps that’s how the market trend generally evolved and a way for vendors to sell their products / services. After working for years in the industry and listening to other thought leaders and folks sharing in the community, I feel the ‘cost cutting’ might be there, but not in the way most of us think about it which should change the way we think about automation.

To make that a bit obvious, what would you say is the ROI of a piano for an average user? It’s not easy to quantify the return on investment for a ‘tool’, but that does not mean it’s any less important depending on the circumstance.

The cost saving silver bullet

For years till date automation tools and services have been sold as a method of reducing cost. In theory it does sound logical, however after working in the industry for years, I don’t know of anyone who has really ‘seen’ these cost cuttings including myself. Let’s dissect the calculation of cost reduction in detail to try and pinpoint the discrepancies.

The Formula

The story goes something like this:

“Savings per test cycle= Tests/checks automated  x  Execution effort (man hours) per check”

And then we’d calculate the break-even point when the savings equal the initial investment in preparing that automation suite plus any other costs etc. For an accountant this would make perfect sense, except the “effort per check” cost does not exist! Let me explain.


Automated checking Vs Testing

The first problem is equating automated checks execution time to a tester’s man hours. The way a machine runs a script is not the same as how a person would test that feature. There is a lot of background to this concept if you are not familiar with methodologies like Rapid Software Testing and related concepts. For those who are not, let me try and summarize the required concept quickly.

The verb “Testing” is an act of “Thinking” and “Communicating” on how to test a specific feature. Once the tester decides what to test, then he / she executes the scenarios. A machine is incapable to “Test” since it cannot “Think” neither can it “Communicate” like a human. It can only “execute” what it’s instructed to check.

(Thanks to the RST community, James Bach, Michael Bolton and folks for articulating this clarity)


The missing effort

Let’s take an example of a candidate application which would hypothetically require around 1000 man hours to test the complete application (btw many products would fit this description). How many testers would be needed to regress over this application within 2 weeks? Around 13 full time testers. Do you think the team would have 13 testers on the team? Mostly not, they would have less than needed people and make do with whatever time they get.

Now, half the effort of “Testing” was the thinking part which a machine cannot do (Some would argue, including me, a lot more than half). The other half is supposed to be spent on “Execution”, where only a small percentage is actually being spent since the team size is ALWAYS smaller than needed.

That’s how there ‘might’ have been ‘some’ savings in terms of man hours but practically there are next to none because most teams are not operating under the assumptions followed while calculating the ROI.


Then why Automate?

Increased test coverage

From our example, we were not able to test the complete application. And from my practical experience, many products are ‘way’ less tested they should be. Adding a dozen more tester’s does not seem to be practical either.

To cover more ground, testers can program a dumb machine to do the basic ‘execution’ which they have to unwillingly do (since its boring doing the same thing again) every time a release is going out. This frees up their time to do intelligent work and get the repetitive checks done by a machine.


Testers focus on important areas

This might seem a repetition from the point above, but there is a slight and important difference. Tester’s don’t just free up their time, but they can now also leverage the dumb grunt by focusing just on the thinking part and delegate as much possible the ‘execution’ part. A high percentage might not be possible, but if automation is leveraged properly, the test quality can improve significantly since most time would be spent on ‘thinking’ than doing repetitive stuff. More on test scenarios that are ideal candidates for automation here.


Quick feedback – Find problems earlier

How many times has it happened after a bug fix an important feature stops working altogether, and this comes to light at the 11th hour when there isn’t enough time to regress the fix properly either.

There is a lot of value in getting feedback quickly. Different checks running at different stages of the development process can provide the needed feedback at that point. As an example, a possible plan could be run unit tests and high-level checks during development, complete regression in QA stage, user acceptance tests on production, or any process that suits your product and team.


Quick feedback – A big step towards CD

The companies to be most successful are the ones taking an idea from the drawing board into the consumers hand most quickly. This is where continuous delivery comes in. The race to minimize the ‘time to market’ can become a huge factor giving the first movers advantage. This video will give more detail on how automation facilitates that.


Increased confidence in the product

An inherent problem with exploratory testing is “it’s done by humans”, which is a good thing too but people tend to forget. A tester might not test the same function the exact same way every time or forget to test altogether. With automated checks, we can be certain of what features were tested and if they are working.

This makes the decision to ship a release much easier and allow for some quantitative measures to take decisions on. Although this alone cannot be enough to make the call, but coupled with decent exploratory testing, it makes a difference.

It’s not just the team, customers of the product also can have a sense of satisfaction of knowing certain checks being automated ensuring the functionality will most probably have gone through a checking process.


Commitment to quality

From our example and my experience, most teams do not have enough testing staff to completely regress the application every time a change is made. Some would argue it’s also impractical. Having automation in place shows the commitment towards ensuring maximum areas of the application get tested or checked before shipping to the customers.

This is where the phrase ‘Quality is a mindset’ comes in. When we hold ourselves and our product to a high standard, it will necessitate to indulge into some form of automation process, because most modern applications are not possible to test adequately with a cost-effective sized team.


There are saving, but not the way we calculate them

Equating man hours to machine hours of execution is not the correct formula for finding your return on investment for an automation project. The returns do not come in tester’s man hours saved, rather in different forms which are by no means less important, just less obvious.

The real value comes from increased test coverage, allowing testers to focus on what really matters while delegating grunt work to a machine, get quicker feedback on fundamental problems or features, a major milestone towards reducing time to market, an increased confidence of the customers and the team in the product’s quality which shows the commitment to quality having an impact on the end consumer.


Feel free to share what other benefits do you feel automation brings to the table.


Is this advice really free

By | May 2nd, 2018|daily post|

A common question I get when reaching out to people

It is human nature to expect something in return for spending time and effort

Over the years I’ve learned the trick in life is not to get, but to give

And the universe will respond back in many times more

But it takes faith and patience to keep going without return

In essence that’s what a business should do, solve people’s problems without trying to squeeze a dollar for every inch they contribute

I’ve always felt, living for something greater than yourself is a life worth living

Seth Godin, Susan Jeffers (and many more I cannot recall now)

Looking to add automation for a legacy product

By | May 1st, 2018|daily post|

Few things you should know before going in:

Legacy products need a different approach than most modern tech stack applications

The automation framework has to deal with the various legacy platforms used

This means compatibility with those languages, platforms is essential

In some cases it can be advantageous too, e.g. front ends built with some JQuery or custom components might make things easier

The backend might not be as ‘automation friendly’ since it was never designed for anyone other than the developer to understand

Processes, tools and the team, all would come into play

More insights in this article: