Caremark Services Microsoft Technologies Consulting

Pick and Choose

Lots of "Best Practices", Methodologies and Experts out there. None of them are 100% correct so Pick and Choose from all of them, the best parts, the parts that FIT well in your specific situation.
This does mean you need to constantly do your homework. Study and read everything you can about the subject.

Ensure There's a BIG GUN

Most methodologies call for a top level manager "sponsor" of the BI Project.
This doesn't go far enough! ENSURE there is one and speak with him/her, talk to him/her and make SURE he/she understands you NEED him/her to be there, to be involved and to pull rank when and as needed. BI Projects inherently NEED data from many areas of the business. Your sponsor will definitely be needed when you inevitably meet with obstructions (which are guaranteed)!

Plan Your SUCCESS

Do NOT set your sights on "Profitability" as your first BI Project! If you aren't sure why, please talk to us asap.
Choose targets that can be achieved in relatively short time periods. Show incremental and positive progress all along the way. Borrow this from Agile, but don't get religious about Agile either!

Do NOT Start an "IT" Project!

If you start by interviewing everyone possible and "requirements gathering" and blah, blah, blah, you are guaranteed this project will fail.
The project will quickly be viewed as yet another never ending effort, wasting time and money and never returning a thing of value.

Teams

We use the term "Teams" quite a bit so let's talk about that.
Some projects start as "on demand" reports, adhoc in nature and no formality involved. Perhaps not even recognized as a "project".
- In this case the "Team" is probably one or two Excel Pros. Their best bet is to grab source data as they can, Excel spreadsheets and text files are usually the MO here.
- This is certainly a "Self-Service BI" situation. There is no formal support from IT and database experts.
- Typically, these Excel Pros are financial analysts with great Excel skills.
- Using tools like PowerPivot and PowerBI, they can create some great reports for analysis but, as demand for their greatness increases, they get mired in sourcing data and "fixing" data problems. Analysis, on their part, goes out the window. As they get swamped with reports to create, the chance of missing egregious errors dramatically increases.

The Caremark BI Method

Be Clear and Be Brave - Create Reports!

Be clear to EVERYONE, VERY CLEAR, to set the correct expectations, that REPORTS will be delivered QUICKLY, targeted sets of REPORTS in each iteration that will be our proof of concept and proof that we can succeed!
It is critical to start delivering some reports. If we spend all that initial time interviewing and dragging through the mud, we lose attention, energy, focus and we ultimately go down the tubes!
This may go against the grain, according to the "experts", but remember, many more BI Projects fail than succeed. Following SDLC waterfall development ideologies (etc) will lead to unemployment!
Be CLEAR that we are focusing on incrementally producing reports. These reports are NOT YET PRODUCTION READY! We are NOT spending the time, at this juncture, to put the ETL plumbing in place that we know will be needed to make our reporting/analysis system timely, accurate and reliable. Again, if we DO spend all our time on these components of our system, we will not be able to SHOW what we can do! No one cares about this plumbing, although it is so necessary! Be BRAVE and make this pronouncement LOUD and CLEAR! Repeat it as often as needed. We are here to SUCCEED!

Never, Ever, Ever, Deliver Reports With Bad Data! (aka Source Data Is UGLY!)

This is a tough one BUT the prototype reports, even when we're being extremely loud, clear and brave, will make it into the hands of a number of managers and executives. There's an EXCITEMENT!! They're SEEING STUFF they have wanted to see for a long time! This is good, real good! They will start asking for more...more...more. This is a far better problem than them pulling the plug and telling us to hit the streets, BUT we have to manage the situation, to constantly remind everyone that we also need to ensure this will grow into a robust, timely and accurate system.
While creating the initial report revisions, you will uncover all kinds of issues with the raw data. It is very difficult to keep "bad" data from creeping in but it is essential to make every effort to do so.
No matter what we say, when executives (who are incredibly adept at immediately identifying wrong numbers in reports) do spot egregious errors, you are likely to lose their trust for good.
Team members who are skilled at QA'ing reports and data are essential. The more skilled eyes on reports, the better.
Be SMART and create repeatable regression tests and run them ALL after every report revision is created. Work HARD to catch and exterminate bugs!
In our experience, detection of and resolution of source data issues and anomolies will be one of the most time consuming segments of the project. The concept of "Self Service BI" for anything other than one-off type reports or perhaps for managing your candy store, belies the fact that source data is almost always "ugly" and requires skilled evaluation and processing.

More Teams Talk

For any effort beyond ad hoc report generation, you do really need a team.
Team size should be JUST the number and the talents you need! No more! Smaller teams are much more effective than just throwing bodies at the effort. Again, time to be brave here and press for what is needed. You've heard the saying "if 1 developer can finish in 30 days, 30 developers can finish in 1 day". BS! Be brave...Caremark can help you, let us be brave WITH YOU!
You WILL need database expertise so don't fool yourself that you don't. Caremark can help here for sure!
You WILL need ETL expertise. Again, Caremark can help here as well. It's quite possible your database expertise may also serve as your ETL expertise.
You WILL need those Excel Pro Analysts full time and they are crucial team members! He/she/they should be able to drive quite a bit of the project. Picking the most meaningful and achievable reports as the project moves along, providing important business context to the rest of the team, participating in the CRUCIAL QA of reports and finally, getting more time to actually anaylze the data!
Team members are needed FULL TIME on the effort. Grabbing bits and pieces of people, as they do their "real" work, is a common mistake and a big contributor to BI Project failures.
So, we're saying that the "Team" can be small, but needs the proper skilled and experienced members. Caremark can help you build your team AND can provide talent for any missing expertise and experience.

Self-Service BI?

Some state you can "stop" after analysts' sourcing of data and creating reports. This is "Self-Service BI". Allow your power data users access to data and advanced reporting tools like PowerPivot and Power BI and that's that!
We disagree whole-heartedly.
We strongly believe that this "Self-Service BI" approach is great at start-up. It's a great method of getting to reports quickly and by all means this should be done initially!
It is also GUARANTEED that, during this phase, troubles, complications and issues with the various source data sets will begin to emerge. We have NEVER seen completely smooth sailing with source data in ANY project, hence our GUARANTEE!
This is where we strongly believe "Self-Service BI" falls down and is not the dependable, repeatable method that we want and need.
Our method calls for use of "Self-Service BI" techniques to craft all the initial reports. During this phase, we also have database expertise reviewing and learning about the source data sets, their troubles, complications and issues. It's a tremendous "jump ahead" compared to other top-heavy approaches to BI Projects.
This does NOT mean we remove PowerPivot from the equation. PowerPivot REMAINS as an important component of our report development, delivery and analysis.
As we prototype reports, we learn the nuances of the data. This learning process may require a significant amount of time. There are situations within data (eg; Financial Period Close and Year Close events, Sales situations that are rare but do occur periodically, Third-Party data changes over which there is little control) that, in many cases, are impossible to anticipate what the data "looks like". These cases can only be dealt with as and when they actually occur. It is not always possible to construct test data for these scenarios. Therefore, it is critical the team has sufficient time, while report prototyping is going on, to encounter and experience as many "special" data situations as possible.
During this period of the project, the Data Team, working with the Reporting experts are:

Constructing a star-schema staging database that will feed the reporting front-end tools.
Beginning to contruct the repeatable, production-worthy ETL process that will load the report staging database.
These Data Team processes will be necessarily iterative as new realizations of data issues arise with each load performed.
During this phase we begin shifting the Reporting Team's data connections to the report staging database.
The goal is a reliable ETL process that runs on the proper "regular" basis as appropriate for the project goals. This ETL process feeds a star-schema database which feeds standard reports, ad-hoc reports and which can be built upon as business needs dictate.
We feel STRONGLY that following our method puts our clients in the best possible position to support current and future needs. It's relatively easy to "grow" such a robust, accurate, maintainable, extendible Reporting Environment by adding Server versions of Tabular Analysis Server and/or OLAP Analysis Server since the base is a solid star-schema database with shared bus-matrix dimensions (which we happily take from the venerable Ralph Kimble).