Service Orientated Architecture Conference -- June 2008
The importance of effective discovery and the issues that arise from inaccurate discovery including understanding the importance of communication in the first hour of engagement
EFFECTIVE DISCOVERY IS ESSENTIAL
As I work with clients seeking to achieve maximum value from their investment in business information systems I regularly encounter situations where considerable dedicated work has been done, yet is producing a sub-optimal or entirely inappropriate solution. I have come to realize that the essence of discovery -- the process of understanding the business and its requirement -- is a fundamental art that is little understood and frequently incorrectly conducted -- this article discusses this aspect of achieving successful project outcomes.
Introduction
As discussed in a previous newsletter Strategy is "the essence of why the organization exists and how it thrives".
The essence of why an organization exists and how it thrives (strategy) is fundamental in undertaking ANY project that is intended to add value to the organization.
If the essence of why an organization exists is NOT understood, any number of actions may be taken which will at best not add value to the organization and at worst will damage or even destroy the organization.
If a project is to create real, sustainable value the project team must understand how the organization thrives and how to support the organization to thrive. An organization that is thriving will be delivering exceptional value to its customers and will be delivering bottom line return.
In order for a project to contribute to an organization thriving, that project must be conceptualized with a clear understanding of what is required for the organization to thrive, that is, the essence of the strategy of the organization.
Strategy, the essence of how an organization thrives, is generally defined intuitively by the people who give birth to an organization and is frequently imbibed by like minded individuals who join the organization as it grows.
Many "strategic planning" processes fail to accurately identify and document the strategy of the organization in the manner defined above and many processes undertake an intellectual process rather than unlocking the essence of the strategy which is an intuitive, cognitive understanding.
Most I.T. projects are undertaken without the strategy being explicitly or even implicitly defined and therefore frequently end up cutting across the objectives of the organization. The worst case scenario is typified by the lead consultant on a project that was going nowhere and giving rise to great client frustration who, when asked for the essence of why the business existed and how it thrived replied "that is an unfair question".
While this may be an extreme response, it is my regular experience that consultants and implementers have NOT accurately determined the essence of the business and how the project they are engaged in is intended to help the business to thrive and as a consequence they undertake all sorts of activities which are at best NOT essential and at worst highly destructive.
When a consultant or implementer understands the essence of the business and how it thrives they are able to formulate solutions that support the business in its endeavours.
In order to do this they need to DISCOVER this information at the outset of their engagement in order to chart a course that will create real sustainable value.
In doing this they may well discover that the project in question is NOT really necessary. 70% of I.T. projects fail outright and one of the major reasons for this is that many should never have been started in the first place!
In the sections that follow I will outline some points that are vital to understanding the importance of discovery -- discovery of the business by the external service providers AND discovery of the technical / methodological approach of the service provider by the business -- both are critical.
I hope that you will find this information useful.
1. Client: "I told you! This is ridiculous! How can you tell me this is NOT in your quote?!"
I regularly encounter situations where the client is intensely frustrated because the consultant or implementer tables a "Variation Order" for something that the client considers fundamental.
The client holds that the item is an essential part of the business and that the implementer should have realized that the item concerned was a requirement.
The implementer, on the other hand, considers the item to be a change in scope.
Frequently the dispute relates to something that the client holds they expressly stated at the first meeting with the implementer or the principals of the implementer. Generally these people are no longer actively involved in the project.
In other cases the client did not think to mention the item because it was so obvious to them that it did not seem necessary to mention it, let alone to state it explicitly in writing.
The bottom line is that the service provider did NOT adequately DISCOVER the business and its requirement.
Frequently this means that they did not ask the right questions of the right person at the right time AND then communicate an accurate rendition of the answers to all members of their team.
Frequently external service providers allocate executives with excellent strategic insight to the marketing phase of the project. These people demonstrate intuitive insight into the client's business and requirement and it is on the strength of this insight that the contract is awarded.
However, in many cases, these same executives seem to miss the point that their insight is critical. Often they seem not to realize that their more junior staff do not have the knowledge, experience and intuition to gain the insight themselves and therefore they do NOT capture this insight in formal project documentation.
In a similar manner, frequently client executives participate in the initial discussions leading to the decision to appoint a particular service provider and they, in turn, fail to document what they understand as critical.
The result is an initial "meeting of minds" at an executive level that is NOT communicated to the operational staff on either side with the result that the project team heads off on the basis of a technical understanding that is disconnected from understanding why the client organization exists and how the project is intended to support it to thrive.
Sometimes the situation is compounded by hungry service providers in an intensely competitive environment where some would argue that somewhere there are competitors who are "less ethical than we are" and thus it is "not practical" to do the job properly. The reality is that it is ALWAYS cheaper to do the job properly first time round.
I have increasingly come to conclude that while this situation results to some degree from a lack of ethics it primarily results from a lack of appreciation for the criticality of effective discovery and the impact that such discovery has on project outcomes. This, in turn, is a reflection of a lack of maturity of the software industry.
By lack of maturity I am NOT suggesting lack of experience, I AM suggesting a need for an approach which results in higher levels of accountability.
My frame of reference for this is the engineering related industry as a whole and the construction of architect designed buildings in particular. In the business of architect designed buildings there are typically four distinct groups of players -- the client, the architects, the engineers and the contractors.
Generally the three external roles are played by at least three distinct organizations and, in practice, each of these three distinct functions may be performed by a number of specialist firms within each area.
This results in a large architect designed building, such as an office tower, being executed by a project team in which there are distinct roles AND distinct tensions.
a. The client concentrates on obtaining the best possible, most aesthetic and most functional building for the best price.
b. The architect concentrates on aesthetics and usability.
c. The engineer concentrates on structural safety and reliability.
d. The construction contractor focuses on getting the job done to the agreed standards at the agreed price in the agreed timeframe and relies on a very detailed estimating and costing approach, frequently arbitrated by independent third party "quantity surveyors" to ensure that they get paid for what they do in a manner that enables them to work profitably.
The business software industry at this stage lacks these distinct roles or anything approximating them and relies frequently on a single firm to perform all functions ranging from the discovery and documentation of the strategic requirement through to the execution of the project to successful business outcome.
This situation is compounded by the reality that the client is, in fact, the prime contractor on an I.T. or strategy project -- such projects are about the business changing itself -- accordingly the ownership of the change process rests with the client.
All of these factors have a significant bearing, however, in the remainder of this document I will concentrate on "discovery", the first step in the journey for each of the role players -- ensuring that all role players are on the same road headed in the same direction.
2. Service Provider: "The client keeps changing the requirement!" They 'signed off' on this!"
The flip side of the above situation is frustration on the part of the implementer / developer / consultant because there were documents tabled and agreed to that defined the scope of the project and now, as the project proceeds, they are told that there are requirements which they consider to be "out of scope" and which the client insists they should have known.
The reality in many of these situation is that BOTH parties are, to a point, correct. Key issues were omitted from documents because the client did not explicitly think about them, they "just know" that these things are important.
On the other hand the external service provider did NOT ask about the subject because they were NOT aware of it. The net result is that the two parties talk about the business with drastically different understanding.
This is often compounded by either or both parties allocating staff to the overall management of the project who are not senior enough to have the full picture of what is really important.
The CEO's on both sides hold that their people are competent and that the project can be delegated and do not recognize the importance of a meeting of minds at a strategic level.
In reality the executives on both sides need to invest time in ensuring that they see the same picture of the business AND the technology -- the service provider needs to devote time to discovering and documenting the essence of the business and how the business proposes to use the investment to thrive. In the process it needs to be sober about the capabilities of its product and services to deliver the expected "thrive" outcome.
30% of I.T. projects fail because of mythology, the tendency of human beings to ascribe superhuman or human characteristics to computer systems or to expect computer systems to do things that only human beings can do.
Part of discovery on the part of the service provider is to ensure that there REALLY is a business case for the proposed investment AND that they can clearly see how this will be achieved AND that the role of the client in achieving this outcome IS explicitly documented.
The reality is that this is seldom done and thus the changing requirement is generally a reflection of the service provider's staff better understanding the business OR the client's staff better understanding the technology and proposed solution OR both.
A vital part of this ongoing discovery process is to prototype the solution with the software as soon as possible so that the business can start to get an understanding of what the service provider THINKS they have understood.
At a more fundamental level, the absence of an understanding of the essence of the business AND what it is that the client expects the investment to do in order to assist the organization to thrive results in the service provider's personnel doing things which are entirely inappropriate and often counter productive.
This is compounded by lack of understanding on the part of the client which results in the client stipulating requirements that they think they need based on a generally incomplete understanding of the technology. The net effect is that both parties talk past each other.
3. The tension of talking past each other
In previous newsletters I have written about techniques of strategic analysis that translate unstructured thoughts into structured lists comprising a limited number (preferably about seven) of concise headline points which are then weighted in terms of relative importance.
I have also written about the extent to which a group of people who have been in the organization for years will see the problem and the solution differently.
I find the metaphor of climbing a mountain to be useful -- the same mountain looks completely different depending on which point of the compass one approaches it from and the experience of climbing the same mountain can range from a comfortable stroll to a vertical ascent that requires special equipment and special techniques by highly trained athletes or which may require a helicopter.
This diversity of view increases dramatically when one introduces two or more teams of people who have never previously worked together developing a solution for a business that one or more of the parties have never experienced before.
In considering this statement, one of the critical things that I have observed is the trap of "same industry" -- implementer and client alike hold that because the implementer has experience with the same software in the same industry that they "know" how to implement the software in the client business.
While at one level this IS correct, at another level it is entirely incorrect.
Two client organizations can deliver essentially the same product or service to apparently the same clients using apparently the same methods and yet be fundamentally different.
For example, I once did work for a bulk chemical manufacturer whose key differentiator was that they produced their product on a custom recipe basis to order and on credit. Their order takers were technical specialists who used proprietary technology and method to analyse the client requirement and determine the exact mixture of components to produce an optimal outcome. Their competitors produced standard recipes to stock.
Thus, while the mundane day to day content looked the same there were essential elements of content that were unique and the nature of the processes was different. It is a fundamentally different thing to produce a custom recipe (that no other customer will use) on credit compared with producing a limited number of standard recipes to stock. In the first case the credit granting process is a bottleneck process in the entire manufacturing process, in the second case credit granting is incidental to manufacture, there are a number of other critical differences.
The implementers of this solution failed to take account of the "manufacture to recipe on credit" essential driver of the business with the result that after three years the client found themselves loaded with a hugely inefficient and very expensive solution that they were seriously considering scrapping. Diagnosing the gap together with the actions necessary to close the gap averted a very expensive abandonment of a system that was basically sound.
I regularly encounter situations like this, fundamental strategic misfit between the solution that is being implemented and the business. In extreme cases this has resulted in projects being abandoned and in other cases diagnosing these factors has resulted in a rapid turnaround of the project to achieve the desired project outcome.
In all cases the situations have been characterized by significant to high levels of tension between the parties as they constantly talk past each other with regard to the critical issues.
4. The FIRST HOUR -- the things that seem obvious and are therefore NOT documented
Over the years I have come to conclude that in any engagement the first hour is the most important.
In the first hour of interaction the client will state things which seem so obvious to them that they may never mention them again unless expressly asked to comment.
The service provider may likewise make statements about their product or solution that seem so obvious that they never mention them again.
Many of the problems that I encounter could have been avoided if both parties had communicated more precisely during the first hour of their interaction AND documented the essence of what they said.
This requires particularly that the service provider listens attentively and asks questions directed at gaining concise insight into the essential issues AND takes notes which are converted to a reference document for use by members of the project team going forward.
Asking the question "what is the essential reason the organization exists and how does it thrive?" may well be the most important part of the whole project. Answering the question clearly and concisely follows a close second.
Having for many years sought to understand the essence of the strategy of client organizations I have used various methods to seek to understand the core strategic drivers of the business.
At one stage I directly asked the executives I interviewed what the strategy of their organization was and, to my surprise, repeatedly found that I could NOT obtain a concise and consistent answer and in some cases even got a considered "don't really know" type of response.
In time this lead me to the question "what is the essence of why the organization exists and how does it thrive" which I have found to be more useful.
Nevertheless it is STILL vital to probe AND to listen attentively.
5. Are You (Service Provider) listening?
I have found that asking a few key questions, such as the "essence and thrive" question, saying very little and taking copious handwritten notes is a fundamental prerequisite for gaining insight into the real reasons for a project and whether the project is potentially viable.
I have heard it said that verbal communication is 80% listening and I was recently told that verbal communication was about 5% talking and the rest listening and non-verbal communication (body language, tone of voice, etc).
Frequently the client has NOT specifically thought about these key issues and it takes time for them to unpack the intuitive reasoning behind the project. Attentive listening is vital.
In the process the listener should be analysing the information presented by the client and seeking to identify the things that the client is NOT saying because they seem so obvious as not to need saying.
As mentioned above, strategy in the early years of an organization is intuitive, as are the reasons for undertaking the project.
In the book "How to Win Friends and Influence People" Dale Carnegie cites an example of someone listening "intently" and being "genuinely interested" and states that this is one of the highest compliments it is possible to pay anyone.
Many times service providers need to move from sales mode to listening mode AND record what they hear. A sales person who listens and accurately reflects back what they have heard and then FAILS to record that information and pass it on to their operational colleagues is doing their organization a huge disservice and could potentially result in a situation that costs their organization or their client a huge amount.
The service provider should constantly test the assumptions they are making and ask themselves what they do NOT know that they do NOT know -- because, if you don't know what you don't know then you don't know what you don't know and when you don't know what you don't know then you don't know what you don't know and then you CANNOT provide a relevant and valuable solution.
Then again, is the client listening?
Clients are also guilty of NOT listening or not taking notes -- service providers also provide information which client representatives do NOT fully understand and do NOT seek clarification of, making the assumption that what they think they heard is in fact what the service provider intended to communicate.
In this context one of the roles that I find myself playing is as a translator between the two parties, listening carefully and feeding back where I think that they have missed one another thereby facilitating greater clarity.
There is a great need for this role as a standard component of any significant I.T. project.
6. Are You (Client) accurately stating the essentials of your business and your requirement?
For the client it is critical to accurately state the essentials of the business and the essentials of the requirement.
In order to do this the client must think carefully and critically about what these are AND document them in a concise and easy to understand manner that facilitates the service provider accurately assessing the requirement and specifying the solution.
In doing this it is important for the client to realize that the solution is a BUSINESS outcome and NOT a technology outcome. Clients who focus on the technology as being the solution are likely to miss the point and thereby contribute to a failed or sub-optimal business outcome.
The client should concentrate on the business outcome, what the business is going to do differently that will create new sustainable value and how the business will use the proposed new technology to assist the PEOPLE in the business to unlock this value.
In considering this question the business should ask what assumptions it is making and what they do not know and then seek to clarify these issues with the service provider.
As long as the business specification is technical, then it is NOT a business specification -- requirements like "easy to use" or "user friendly" are NOT business requirements, they are what Professor Malcolm McDonald refers to as "motherhood and apple pie". The most user friendly software is the software that operators have been using for the last few years and are thoroughly conversant with and have mastery of.
What IS relevant is the essence of why the business exists, how it thrives and how it expects to use the software investment to support it in thriving.
If this is concisely documented then the possibility of the service provider producing an outcome that adds real business value is significantly enhanced.
7. Only PEOPLE effectively using technology deliver value
In previous articles I have made the point that technology and methodology do NOT deliver value, it is PEOPLE effectively using technology and methodology that deliver value.
A pen is inert and without value, except as an ornament, until it is held by a person who knows how to write. If that person has wisdom and insight to share then the writing can be of great value and, by extension, the pen gains value. Yet, if the wisdom shared is truly valuable it matters not whether the pen used was a cheap pencil or the most expensive hand crafted gold pen available.
Computer systems are just like pens, a simple system effectively used by well trained craftsmen (operators) is far more valuable than the most expensive system used by poorly trained and poorly motivated users.
Accordingly, discovery must focus on what people do in the business and what they will do differently to create sustainable value in the organization.
Conclusion -- Effective Discovery is Essential
Discovery is the essence of a successful I.T. project.
Discover the essence of why the organization exists and how it thrives AND the essence of how it will PRACTICALLY apply the proposed investment to deliver sustainable value and that will enable you to start the journey down the right road and stay on the right road.
Discover the essence of what the technology can do and how to apply it effectively and this will support the business outcome.
However, the essence of the solution lies with strategic insight and interpretation of that insight and business actions by business people effectively applying technology with quality content also designed with strategic insight and interpretation.
If you would like to discuss your I.T. issues and obtain advisory input on how to manage them please do not hesitate to contact me.
I would welcome the opportunity to advise any organization that is in pain or dissatisfied with regard to information technology as to the steps necessary to overcome failure and achieve success.
I offer a concise I.T. diagnostic "pulse measurement" investigation to establish the causes of sub-optimal I.T. performance and recommend specific actions to achieve success and also offer advisory services with regard to the implementation of these recommendations.
The essence of all advisory and other services is on assisting clients to achieve high value sustainable outcomes that assist the organization to thrive.
068a Discovery -- An Essential Element of SOA Implementation -- by Dr James A Robertson PrEng from James Robertson
Download Discovery -- An Essential Element of SOA Implementation -- Slides in Adobe pdf format
Download Companion Article -- Effective Discovery is Essential in Adobe pdf format