Summary
There are many Business Intelligence or Data Warehouse projects that end with failure. This is understood as a 'non-fulfillment of expectations': cost, delivery times, utility, data quality, usability by users, company involvement, contribution to results, etc…
Here we try to explain some keys, based on in real experiences that allow us to successfully tackle such an implantation.
1. What is Business Intelligence?
In order to achieve this objective, it is first necessary to know what Business Intelligence is:
Unfortunately, this term has nothing to do with the average IQ of people who work in a certain business. In fact, business intelligence (BI) has to do with the data and applications of a business to understand it better. Similar to military intelligence, which seeks to understand the enemy, business intelligence is primarily about itself. Specifically, business intelligence systems are based on creating business computing models so that it can function more efficiently.
Data storage is at the core of business intelligence processes. In the ETL world, business intelligence generally refers to the entire space of database systems, software, analysis, and user evaluation that is intended to understand and evaluate a business.
There are generally one or more analytical uses of the software (for example, Business Objects, Cognos, or Microstrategy).
BI systems differ from operational systems in that they are optimized for querying and reporting on data. This typically means that, in a Datawarehouse, data is denormalized to support high-performance questions, while operational systems are generally fully normalized to support reference integrity and to continuously insert data. ETL processes loading BI systems have to translate from normalized to denormalized operational system. And, typically, they have severe malfunctions because they should not degrade the performance of operational systems, and they should not prohibit access to the warehouse.
That is why Business Intelligence arises, based on new analysis structures, basically multidimensional, in contrast to the relational one.
2. How to choose a Business Intelligence application?
The first thing I can say is that we have to identify what the needs are and the type of tool we are looking for: analysis, reporting, database, OLAP, etc…
At the moment, I am going to leave you with a few brushstrokes, according to my criteria, of the main factors (in no order of importance) to take into account when we choose a Business Intelligence tool:
1) The Platform: It is not the same to be tied to Microsoft, or to be able to work on Unix, or to have an Open Source Linux strategy. The same applies to hardware. Some manufacturers are restrictive.
2) The seller's CV: It is very useful to know the type of implementations that have been made, if they have been carried out in time, if they are used, user satisfaction, etc…
3) The size of the cube: It is essential to do a prior analysis of the breadth of the information to be stored. Some applications may 'explode' at a certain level.
4) Query speed: Users always want speed in their queries. And if 20 seconds of waiting is too long, you may have to find another tool.
5) Support and help services worldwide: We have to be sure that if something fails in the application (and it will fail, this is for sure) we can solve it in the shortest possible time.
6) Analyst evaluations: Gartner, IDC know what they are talking about… and they are usually objective. It is worth looking at their 'quadrants'.
7) The seller's ecosystem (consultants, partners, agreements, developer community…).
8) Installed user base. If there is much better in my sector. If I can talk to them and see the tool live, even better.
9) Graphical User Interface (GUI). Remember that we are talking about a tool for end users and if they do not like it, they will not use it and it will be wasted money.
10) The price: It does not have to be the most important… but… it is important !!!
11) Integration with other tools: No tool works as an island isolated from the rest. The same as a company, if you create islands, you will create isolation.
3. Why do many Business Intelligence projects fail?
Sometimes we are surprised that with the development to which many tools have arrived, the use of contrasted methodologies and the higher level of knowledge of technicians and users, so many disasters occur in the implementation of Business Intelligence solutions, in terms of excess cost over the planned, non-use by users, not meeting expectations, erroneous information, etc…
Based on my experience, I am going to tell you about some of the main flaws:
1) Many Data Warehouses grow in size disproportionately because technicians fail to say 'no' to 'excessive' user demands.
2) It is preferred to carry out the project with people from the company itself, when they have neither the time nor the knowledge to be able to cover it.
3) Unrealistic dates of entry into production are set, which
causes new dates and more delays.
4) The budget allocated for the project is low compared to the degree of complexity that is to be developed.
5) The selection of software and hardware is sometimes made following criteria of general agreements or commitments, rather than purely technical.
6) Before the project, no benchmarks or 'proofs of concept' are performed to determine feasibility.
7) The source data is not clean. Duplications, errors, wrong characters… imply a more expensive ETL process, greater Database size and worse performance.
8) The project sponsor does not act as such during the project. It does not 'go down to earth'.
9) Bad choice of consultants and excessive rotation among them.
10) Little involvement of end users that leads them to feel some frustration with the results obtained.
11) Fall into the error of 'everything can be done in computing' and start with customizations, write code outside the standard functionalities.
12) Not aligning the project within a business strategy.
There are many more factors that can make a Business Intelligence project fail, but these can literally "knock it down", not getting more projects for the consultants, bad image of the product and internal risks for the CIO and other sponsors.
4. OLAP systems. Tips for its correct use.
We are going to suppose that we have carried out a detailed analysis of the needs of the company, we have spoken to all the interlocutors and users, we have identified the needs of reporing and access, and finally, we are clear about the model (what variables, formulas, dimensions…) we will include.
It is at this point that we ask ourselves the key question: What storage method are we going to use? We can have all the data in our transactional system, which allows us to mount it faster, but it can be less efficient. Or we can pre-calculate the information so that it is obtained quickly and accurately. It is a very important decision, because it can imply a higher cost of maintenance and licenses.
It is here where it is convenient to clarify these acronyms:
OLAP is online analytical processing. It is a way of storing the information in a Database that allows queries to be carried out more effectively. It is an abbreviated definition, of course, reality is more complex.
MOLAP: Multidimensional OLAP. Both the source data and the aggregated or precalculated data reside in the same multidimensional format. Optimize queries, but requires more disk space and different software. The first point is leaving a problem: disk space is getting cheaper.
ROLAP: Relational OLAP. Both the pre-calculated and aggregated data and the source data reside in the same relational database. If the DataWarehouse is very large or speed is needed from users it can be a problem.
HOLAP: Hybrid OLAP: It is a combination of the previous two. The aggregated and pre-calculated data are stored in multidimensional structures and those with a lower level of detail in the relational one. It requires a good job of analysis to identify each type of data.
From a practical point of view I would like to add some other characteristics of an OLAP system:
- It must be fast. It should not be long between the need for information and the result. It must have a functional and business language. It must be easy to use, with wizards and templates. It must be able to integrate APIs. It must have powerful graphical possibilities. Usual. Possibility to store and share reports and calculations created by users. Administration should be carried by users, not IT. Implementation time (project) should be very short. Must generate measurable responses for decision-making. than being able to get ROI with OLAP applications.
As a final summary we can say the three main aspects to take care of are the choice of the people who will use the tools, of those who are in charge of the project and of the external consultants. In addition to all this, the system must be within a clear medium and long-term business strategy, to avoid patch solutions and unnecessary expenses.
