Identity management testing: Why test?

What comes to mind when you think of Identity Management Testing? Is this the first time you’ve thought about testing? Is it something you push to the back of your mind? 

This is the first entry in our seven-part blog series on Identity Management Testing. We will explore and explain why testing is important, and how you can manage and automate it to become more efficient.


The challenge with idm testing

Identity Management Testing has always been a particular challenge for IT departments. It can sometimes be perceived as an unnecessary overhead, getting pushed towards the end of projects and rushed through at the last minute.

In the past, when we used Microsoft Identity Integration Server (MIIS) and Information Lifecycle Management (ILM), the solutions were limited to the Sync Engine and several connected systems. This meant all of the solution logic was in Visual Studio, which made it easy to build and then test at the end.

Forefront Identity Manager (FIM) then came along and introduced the Portal. This meant we had parts of the solution in two different places, with the portal allowing a lot more functionality. We went from small to medium identity management solutions, then to medium to large identity management solutions, where the approach to delivering MIIS and ILM solutions no longer works.

We now have code to test in the Sync Engine, and Workflows and Permissions to test in the Portal. Testing all of this functionality can no longer be left until the end of a development as there is so much to do. Regression testing, when adding new functionality, is now more important than ever.

Testing can no longer be thought of as a nice-to-have, or something that you can just fit in at the end. If you try this approach, the project will overrun, go over budget and require remediation afterwards.

Approach to testing

When delivering a solution, testing needs to be the first thing on the agenda. It’s not just about making sure the solution is working correctly. When thinking about testing, ask yourself HOW exactly you are going to test and WHAT exactly you are going to test at the start of the project. This ensures both the consultant and the client have a comprehensive and cohesive understanding of the testing process.

Testing starts as soon as the design of the solution is complete, involving the key stakeholders agreeing on an Acceptance Criteria for the user stories. This enables the client and consultant to agree and be sure on what is going to be delivered, and what the outcome of that user story is going to be. Depending on how complex the user story is, that could mean four, five or even 25 Acceptance Criteria.

For example, an Acceptance Scenario for a new user could be “When a new account is created, a unique account name (five characters from surname + Initial + NN – must be unique – for example, SMITHJ01) is created.” When you write the tests, it is possible and normal for one test to cover more than one Acceptance Scenario.

Once the client has agreed the Acceptance Criteria, we are now in a position to get the Acceptance Scenarios written. These are tests written by the consultant, that go into detail about what attributes the initial object has and what attributes will be set at the outcome. The aim is to assess the life-cycle that object will go through in the production environment. The steps should include enough detail to enable anyone with a little identity management knowledge to follow. An Acceptance Scenario example might be a new employee in HR getting an account created in Azure Active Directory (Azure AD) with the correct attributes set, including account name, email address, manager, etc.

Test Scenarios are tests that confirm a small part of the functionality is working as expected. This test being passed does not mean an Acceptance Criteria is passed. An example of a Test Scenario could be that the Portal sets new account names correctly.

Once the Acceptance Scenarios and Test Scenarios are written, they are then reviewed by someone else on the team to confirm they have enough detail, ensuring that the Acceptance Criteria is covered.

We now have our tests up and ready to go. When we start developing the solution, as we complete each User Story, we can run these tests to confirm that what we have developed is going to meet the client’s expectations.

Ideally, the client will have someone who is creating their own Acceptance Scenarios to run once the development is finished, so that they can also test the solution and confirm it is working as anticipated. During this stage, the client can log any defects with the consultant, who can then resolve any issues that may have been identified.

Benefits of this approach

  • By writing the Acceptance Criteria before we start putting together any tests or code, we can ensure the consultant and client have a unified understanding of the project outcomes. Without this process, a recent change in policy could go unnoticed until the end of the development cycle resulting in costly remediation work. By capturing this information before development has started, the design document can be updated accordingly.
  • If tests are written at the start, they can be used during development to ensure that what you have developed works as expected and that other parts of the solution have not been broken as a result of the new changes. This should reduce the number of defects raised during User Acceptance Testing (UAT), therefore reducing any chance of the project being delayed. This can also help the project to deliver its crucial go-live dates – and on budget.
  • I used this approach with one of my recent customers. It ultimately led to 95% of the defects raised being resolved the same day and in the latest go-live, no defects in the production system being immediately raised.

Hopefully, this article has got you thinking about how you are going to best test your identity management solution and how you can reap the benefits of being pro-active.

There is, of course, an overhead to testing: capturing the Acceptance Criteria, creating the Acceptance Scenarios and Test Scenarios, running the tests and supporting User Acceptance Testing. However, that overhead could work out significantly less than a project that over runs and has defects in the production environment.

There are four key factors to an identity management project:

  • Time
  • Money
  • Quality
  • Functionality

In our next Identity Management Testing blog in this series, we will go through how we use Visual Studio Team Services to document and store Acceptance Criteria, Acceptance Scenarios and Test Scenarios, and the results of previously run tests.

Navigate the (new) world of identity and access management in our webinar

Watch our Privileged Access Management webinar and discover MIM’s new security feature