| Project had been running for three years and cost millions with NO deliverable with seriously toxic relationships throughout the Project Team |
|
|
Clear evidence of implementer incompetence and possibly dishonesty, range of key measures prescribed to turn the situation around
Implementer was the IT consulting arm of one of the "Big 4" accounting firms
|
|
| Report |
|
A Pulse Measurement investigation into the health of the Microsoft AX Project and associated services from Implementer28 was undertaken for ClientCompany28 from 14 to 19 March 2012.
|
|
This report summarizes headline level findings and recommended actions.
|
|
All recommendations should be regarded as provisional and might be refined in the event of further information coming to light but are regarded as a reliable headline level result.
|
| Findings |
|
Following are the headline findings, percentages at the end of headings give an indication of the relative importance of each point in making my recommendations.
|
| 1. Toxic project psychology and low visibility of deliverables – 7% |
| The level of conflicting opinions encountered during this Pulse Measurement was in the upper 10% of what I have encountered in doing dozens of Pulse Measurements |
|
| There is a lot of finger pointing, Implementer28 are blaming ClientCompany28, ClientCompany28 are blaming Implementer28, Implementer28’s people are blaming their management, ClientCompany28’s people are blaming their management. There are people who were only prepared to express opinions subject to explicit assurance of confidentiality. |
|
| A significant number of reports of people in tears, people swearing and shouting, planning to resign or resigned, etc |
|
| The psychology of the project in many areas is very toxic although there were some suggestions that there were improvements as a result of recent developments |
|
| Very difficult to get real clarity with regard to what was going on, had to dig for sight of deliverables, reports of work trashed and redone repeatedly, etc |
|
| Ratings of likelihood of a successful outcome ranging from 0 (trash immediately) to 6 or 7, several people thought the project should be aborted |
|
| I have given this point a relatively low rating because it is my opinion that while these are real issues they are the symptoms of the issues addressed below |
|
| 2. Lack of a clear strategic project definition – 22% |
| The thing that stands out about this project is the complete lack of a clear definition of the project goals, divergent opinions as to the essence of the business and the project, etc |
|
| There is no visible document that clearly and concisely sets out the goals of the project, the Essence of ClientCompany28’s business, what the final deliverable will be and how to get there |
|
| The bottom line is that the project is rudderless and drifting with no coherent view of what is being achieved and how it is to be achieved |
|
| It appears that Implementer28 do not see such a definition as important and have never made any attempt to obtain clarity – Implementer28CEO thinks he knows what is required but his definition is drastically at odds with what I was consistently told by ClientCompany28 management – this seems to evidence a level of arrogance or disinterest which is cause for concern |
|
| My understanding of the Essence of ClientCompany28’s business as seen by ClientCompany28 Executives and Management is “financial services – we want to be the leading provider of leasing solutions through the use of highly automated and highly advanced financial service software systems” |
|
| The goal of the project from ClientCompany28’s side is “Internet banking on steroids – fully interactive computer applications on the customers desk that allows them to tailor a vehicle lease, get a quote and authorize the quote and then allow ClientCompany28 to process that order through to delivery with minimum human intervention, includes allowing suppliers to pay themselves” |
|
| On a scale of 0 to 10 where 0 = we do not need computers and 10 = we are completely dependent on computers as in financial services and banking, ClientCompany28 management see their goal for this project as being to move them to a 9 or 10 and see themselves as already being at a fairly advanced level. Implementer28CEO on the other hand sees ClientCompany28 as currently a 4 with the initial aim of the project being a 6 with possibly an 8 in due course but does NOT see this as a critical requirement |
|
| Implementer28CEO does not see 10 as necessary or achievable |
|
| Implementer28CEO’s description of the project and its goals is flowery sales speak and lacks precision and clarity |
|
| The biggest single reason this project is in difficulty is a lack of a clear, consistent and cogent statement of the goals of the project coupled with a substantial gap between Implementer28’s interpretation of the requirement and the actual requirement as stipulated consistently by ClientCompany28 Executives and Managers – it is my view that it is the responsibility of Implementer28 to make sure that they accurately understand the goals of their client |
|
| There are references to head-count reduction goals which, in my experience, are likely to be difficult to deliver on – head count reduction is the natural consequence of a well configured, well implemented ERP well operated and is a spin-off. Making head-count reduction a goal introduces negative psychology and results in people trying to figure out how to reduce head-count rather than now to implement the system optimally |
|
| 3. Development project NOT an ERP project, AX is an inefficient development environment – 15% |
| As a direct outworking of the previous point, ClientCompany28’s requirement is for a highly tailored solution geared to delivering the highly intelligent customer interface and all supporting services necessary to create the competitive position outlined above |
|
| It appears that Implementer28 only fully realized that this was primarily a development project about six months ago and have since been resourcing accordingly |
|
| As far as I can determine Implementer28 are primarily a ERP configuration service provider and NOT a development house. They have had to acquire development resources but still have to learn how to manage such resources (fundamentally different management to configuration) and the massive gap in strategic understanding referred to above will hinder this |
|
| It is important to understand that from a fundamental first principles consideration of ClientCompany28’s objective, as set out above, this was NEVER an ERP project, it has always been a development project |
|
| It appears that either Implementer28 totally failed to comprehend this in their initial presales negotiations or chose to ignore it and try to shoe-horn ClientCompany28 into AX no matter what the cost |
|
| Estimates of the level of custom development on this project run to as high as 75% but not lower than 30%. There is NO question that this is a major custom development project and that Implementer28 grossly failed to recognize this and run the project accordingly |
|
| Further indications are that AX is a complex, cumbersome and highly inefficient development environment, one ClientCompany28 technical staff member referred to it as “horrible” – I do not have independent verification of this but repeatedly observed that it was very difficult to get clarity on what is being developed and that there is considerable jargon on the part of Implementer28 personnel |
|
| It should be noted that the development of an “Internet banking on steroids” solution for ClientCompany28 requires a front end customer interface that Is extremely lean, highly efficient (fast), robust, error free, etc – inherently this implies the use of high performance technology which as far as I can determine is NOT what AX offers. There appear to be strong reasons to suggest that the customer interface should NOT be developed using the approach currently proposed |
|
| Coupled to this there is a fundamental issue in that the solution should be designed top down with the customer interface as the starting point – determine what is required in the customer interface and then build the rest of the customization to deliver this. In this case there is no attempt to design and build in this manner |
|
| There appears to be a basis to circumscribe the extent of development undertaken in AX versus what might be undertaken using other technology |
|
| It appears that AxAddOn28 may only provide a 15% fit according to some interpretations and is therefore an inappropriate technology choice – at that level of fit it would be better to redesign from scratch. Xxx appears to know of alternative technology which is allegedly a better fit |
|
| Changing from AX 2009 to 2012 mid-stream seems questionable given that there was so much custom development required – there is the paradox that many of the reasons why Informix has been rejected as a future platform are manifesting in the AX environment – e.g. massive upgrade effort – it is estimated that the move from 2009 to 2012 has delayed the project by 9 months, this is roughly the length of time it would have taken to port the current FourGen / Informix Xxx solution to the Xxx database platform which would allegedly have remediated the technical constraints on Xxx (note that Informix is owned by IBM) |
|
| 4. Implementer28 have failed to provide necessary professional guidance – 14% |
| From consideration of the previous points it will be apparent that I am of the opinion that Implementer28 have failed miserably to provide the required level of professional guidance and thought leadership with the result that a year to two years of time, hundreds of thousands of Pounds and tens of thousands of hours have been sub-optimally employed if not totally wasted |
|
| It seems that Implementer28 have spent three years without a robust clearly understood definition of the project goals – I would expect to see carefully designed posters setting out the project goals and main design elements, I asked close to ten people for the strategic concept document that allegedly exists but at the time of writing have not received a copy |
|
| Implementer28 have spent at least two years treating the project as an AX ERP configuration project when fundamentally it is a development project and AX is not necessarily then the most appropriate technology platform – by all means use AX as the ERP but that does NOT necessitate that high performance customer facing software has to be developed within the AX context and if AX is inhibiting development to the extent that some people seem to suggest then AX needs to be reconsidered |
|
| Implementer28 appear to have shifted considerable responsibility for the design of the solution to ClientCompany28 people with ClientCompany28 staff being taken out of production for very considerable periods of time to undertake documentation and workshops that appear to be outside their scope of knowledge and experience and producing mountains of theoretical documentation. At this stage I have not been convincingly shown that these documents are resulting in a concrete and robust deliverable |
|
| It is my opinion that Implementer28 should have facilitated workshops with groups of ClientCompany28 people and that Implementer28 personnel should have produced the documentation in close cooperation with the development personnel in order to ensure robust specifications. The basic process should entail: |
|
| - Walkthroughs of current way of doing things |
| - Outline by ClientCompany28 management of the future desired state |
| - Workshops with user populations, with Implementer28 analysts AND developers present |
| - Implementer28 analyst personnel produce documentation in consultation with business and developers |
| - Review draft specification in workshop setting with business users and developers |
| - Refine |
| - Review and refine again |
| - Adopt |
|
| It appears that the process that has ACTUALLY been followed is: |
| - Pull ClientCompany28 personnel out of the business |
| - Leave them to draft all sorts of documents they really do not understand |
| - Workshop in some way |
| - Present to management and get an explosion because management were not consulted at the start |
|
| As far as I can see Implementer28’s way of working is fundamentally flawed from a systems engineering perspective, it appears to be defective even from the perspective of limited tailoring of standard AX components in that it seems to be fundamentally a bottom-up approach |
|
| The ClientCompany28 team for the most part appear to be dedicated, loyal, passionate and trying really hard but from what I can see the defective Implementer28 approach has set them up for failure |
|
| It does not appear to occur to the Implementer28 consultants that if they are repeatedly doing weeks of work and being told by ClientCompany28 management that the results they are producing fail to meet business requirements that there is something wrong with the way Implementer28 are doing things rather than blaming ClientCompany28 – UNLESS ClientCompany28 have been consistently interfering with the work of the Implementer28 consultants and giving them direction and then not take responsibility for the outcome |
|
| It is firmly my opinion that a client engages the services of an external professional services firm like Implementer28 on the basis that they expect that firm to provide comprehensive guidance, thought leadership, strategic architectural guidance, etc – this appears to be consistently lacking in the services that Implementer28 have supplied |
|
| There are some VERY GOOD PEOPLE on the Implementer28 team but the high level leadership is lacking and there are rumbles that the top management of Implementer28 have refused to listen to the consultants on many occasions and made sales promises to ClientCompany28 that the consultants knew were false and that they could not deliver on |
|
| It does appear that Implementer28 are course correcting, Xxx and a number of his people appear to be highly competent and Xxx appears to be competent but they are operating within a defective governance framework from Implementer28 management and at this stage seem likely to produce more of the same unless the fundamental strategic definition issues outlined above are fully addressed |
|
| I have not been able to understand why moving to an “Agile” methodology is going to fix the problem when huge amounts of work have been done and it appears that the basic content of the work is defective |
|
| Certain Implementer28 consultants come across as very cocky and confident to the point of being arrogant without necessarily being able to demonstrate delivery |
|
| There is a massive deficit in terms of high level strategic guidance with regard to the design of the overall solution and, at this stage, while I think that the development team MAY be capable of delivering technically they need much clearer strategic direction and I am not sure that Implementer28 management have the knowledge and experience to run a project of this nature or else they are stubbornly refusing to acknowledge that there is something that they are doing which is fundamentally getting in the way of the core objectives of the project |
|
| There also appears to be a lack of Governance clarity on the Implementer28 team in terms of who is guiding the project overall |
|
| Fundamentally ClientCompany28 have very substantial and very valid reasons to deal very severely with Implementer28 – this having been said, the fact is that ClientCompany28 have made a huge investment in getting to know the AX environment and development platform and build relationships with the Implementer28 team and Implementer28 have made a commensurately huge investment in getting to understand ClientCompany28’s business and build relationships with ClientCompany28’s people – it is NOT in the interests of ClientCompany28 to terminate Implementer28 at this point BUT a major change in the way the relationship is managed is going to be vital |
|
| 5. ITManager28 is stretched excessively – 10% |
| It seems that ITManager28 is excessively stretched and that she has stepped up to the plate in various areas which have resulted in this situation |
|
| It is recommended that actions are taken to assist ITManager28 to focus on her core areas of real high value knowledge and experience relative to achieving the project outcome and provide resources to ensure that other activities are delegated to others – I understand that some recruitment measures are in place but would recommend that the exact specification of the human resources required be carefully evaluated in the light of this report before personnel are appointed or contracted |
|
| 6. ClientCompany28 need a very experienced and very senior project leader / director – 12% |
| From ClientCompany28’s side it seems to me that the biggest gap if for a very senior, very experienced project leader / project director, not necessarily full time, to come in and take over the running of the project from the client perspective, someone who can engage in tough negotiations with the service provider, negotiate in terms of project scope on a peer level with executives, give strategic solution guidance, etc |
|
| This is a role that I sometimes play, ClientCompany28CEO has asked me for a separate proposal in this context |
|
| 7. Outsourcing of the development of core strategic capability is not advisable – 2% |
| Given the very ambitious vision of ClientCompany28 and the core strategic capability that it is desired to develop I am not entirely comfortable with an external third party developing the elements of the software that are critically strategic, that is the software to sit on customers desks |
|
| I realize that it is late in the day to change the current contractual arrangements but I would suggest that the current arrangements are reviewed – it is vital that the core software development capability for core strategic value should be tightly held by ClientCompany28 and NOT be out in the market to be sold off to the highest bidder |
|
| Undertaking key elements of this development in-house should be carefully considered |
|
| 8. Fundamental database design and business intelligence design can start immediately – 8% |
| I strongly recommend a drastic rethink of the way the solution is being designed |
|
| The fundamental logical entities that make up the customer user interface and the major building blocks of the solution, that is vehicles, accessories, customers, lease terms, etc, etc are well known and will not change – there is no reason a robust entity relationship map of fundamental data elements cannot be developed immediately |
|
| This document will then form the basis of: |
|
| - Database design for the customer user interface |
| - Design for the data warehouse and Business Intelligence solution loading out of Xxx as an interim solution if required |
|
| The fundamental database design is a function of the business NOT AX, there needs to be linkage back into AX in some cases but if the view that up to 60% or more of the total solution will be custom developed then constraining this development because of the existence of AX must be questioned. |
|
| 9. Quality of validation data is average to poor requires critical attention – 6% |
| Quality of validation data is average to poor, the new Chart of Accounts seems fair but could be improved but other lists like the Accessory code require significant investment: |
|
|
|
| There is no logic in the above list on which software intelligence can be built |
|
| A well designed hierarchical list should replace this |
|
| Similar comments apply elsewhere |
|
| Note that a poorly configured ERP leads to all sorts of operational inefficiencies and increased operating costs – precision configuration is a critical requirement to achieving the goals set for this project and I have seen very little evidence of precision configuration – thus, while this point has a relatively low weight it is nevertheless critical once the other issues have been addressed |
|
| 10. Remediating and strengthening Xxx should be investigated further – 4% |
| The possibility of falling back to Xxx was explored |
|
| There ARE apparently interface issues with the Informix Database that are inhibitors to change but further research in this field may be warranted |
|
| Conversion to the Xxx database would open up the database but is estimated to take two senior developers twelve months and does carry risks. Once this was done ClientCompany28 would be able to use a diversity of development technology to enhance the Xxx solution long term |
|
| Note that it would be necessary to pay premium salaries for key developers but it is likely that these salaries will NOT be any more than are required for AX developers |
|
| There does appear to be a basis to conclude that the decision to trash Xxx was not necessarily the best decision given the real strategic priorities, it is almost certain that if ClientCompany28 had remained with Xxx, converted the database and run a tight project in lines with past experience that a new customer interface would have been deployed and fully operational at least a year ago |
|
| The above findings point to a number of areas of uncertainty that require further investigation in order to arrive at a robust way forward. Proposed actions are discussed in the next section of this report |
|
| Overall it appears that the project can be stabilized and managed to achieve a successful outcome but some significant changes in approach are called for |
|
| Recommended Actions |
| Following are the recommended actions following on tabling of this report: |
| 1. Appoint executive level project leader / director – 28% |
| ClientCompany28 need an executive level individual, probably part time, to drive this project for them – need considerable systems, project management / leadership experience, architectural ability, etc – this person should lead and guide the execution of all the points below |
|
| 2. High level solution design and documentation – 23% |
| Run a series of workshops at the Executive level to arrive at a robust and comprehensive high level specification of the required business outcome (Internet on steroids) and produce comprehensive documentation down to the level of the individual user interface screens that the customer will interact with |
|
| 3. Evaluate sustainability of Xxx – 14% |
| Undertake further research to establish whether there are alternative solutions to the problems with Informix interfacing in order to make Xxx more viable as a fall-back position |
|
| Note that Informix is now owned by IBM so I would expect there to be fairly robust alternative solutions to the problems that were described to me in terms of interfacing |
|
| 4. High level walkthrough and more in-depth evaluation of state of AX project – 11% |
| I had great difficulty getting a real hands-on feel for what is there, some people said almost nothing and others say a considerable amount |
|
| The only way to answer this is to get all the key people in a room for a day or two days and have the Implementer28 and ClientCompany28 teams walk us systematically through what has been done in terms of practical demonstration of documents, screens, etc |
|
| At the end of this process take a final decision as to whether there really is a case to continue with AX or fall back to Xxx and alternative technology is warranted |
|
| 5. More detailed evaluation of Implementer28’s performance – 9% |
| For the reasons indicated above it is difficult to gauge Implementer28’s performance – as far as I can see I would give them about 2/10 but a few people seemed to point to better performance |
|
| The proposed walkthrough above will give a better indication of whether Implementer28 is just offering good sales talk and technical sub-standard work or is able to deliver – note that I think that provided the way in which they are managed is significantly changed there is a reasonable basis to expect that they can deliver provided we prevent loss of further key personnel |
|
| I do NOT think that Implementer28 should be terminated unless it is found that there really is nothing of substance to show for the work done so far but I DO think they need to be managed very differently |
|
| 6. Project to deliver customer solution as rapidly as possible – 7% |
| I am reasonably certain that with limited technical adjustments and compromises it should be possible to start designing and developing the software to sit on client desk-tops in the near future, to run against Xxx if necessary initially |
|
| 7. Precision configuration (taxonomy) activities – 5% |
| Precision code tables should be developed – refer my Taxonomy Manual for reference |
|
| 8. First principles data warehouse and business intelligence design – 3% |
| As with the customer interface, the design of the master tables for the Data warehouse should commence soon and not be further delayed and the same with the Business Intelligence design |
|
| Conclusion |
|
There appear to be major reasons for dissatisfaction with Implementer28 and reasons for concern with the current project status
|
| Measures should be taken that improve team morale, improve focus and build on what it good in the project environment which appears to be significant and improving – we should NOT break what is in the process of getting fixed |
|
|
Having said this there is a need for appointment of a senior advisory / leader to ClientCompany28 to direct the project, proper definition of high level project goals, more in-depth assessment of the value of the work done to-date and other actions directed at ensuring a high value outcome
|
| I am available to assist in whatever way ClientCompany28 require to give effect to these recommendations. |
|
|
|
| Dr James A Robertson PrEng |