Why Every BI Solution Needs a Data Exploration Tool like Endeca

There are dozens of fantastic BI tools for reporting and analytics available today.  Market demand keeps growing and the industry responds with an ever evolving toolset and the knowledge capital to deliver.  They excel at offering rich visualizations, intuitive interactivity and powerful data analysis.  Predictive analytics and advanced data mining reveal patterns and relationships can be dramatic game changes for our business clients.  This is an exciting space to work in.

Ultimately however BI solutions remain largely in a reactive mode.  Not only the delay translating more current data into more current business decisions, but the time to adapt to changes in business rules or model new algorithms for analysis.  Even forecasting and predictive analytics need to be modeled and routinely tuned using available data.  Our challenge in delivering effective BI is how to reduce that response gap so users can recognize and react as quickly as possible to changes in their business.  Existing BI tools offer tremendous features, but how can a data exploration tool like Oracle Endeca Information Discovery (OEID) complete your BI stack?

First and foremost the time and effort to supporting new or changed data of interest must be reduced to its absolute minimum.  With most BI solutions the turnaround effort necessary can be unavoidably expensive.  The complete data flow from source to user interface can have many dependencies and steps that need to be touched.  Building comprehensive warehouses and data marts are significant ventures that offer huge value, but if any data of interest was missed or not available you’ll often find those warehouses become very inflexible beasts.

For OEID the time and effort to add those bits of data are significantly less.  I’ve seen this first hand at one client after the other where expanding or altering their data required minimal effort.  The ETL tool Integrator is the only place where you have to make a change so the data is added to the ingestion.  There can be some rare scenarios where complex transformations may become necessary, but typically I find a single reader component to update and a few metadata links in the graph to pass the field along.  There are no further changes required to make that data accessible to end users.  This velocity is a fantastic win for IT and for business users able to see and work with more current data as quickly as possible.

The second point is to reduce the gap between business users and the breadth of enterprise data which most find a complex mysterious resource that is difficult if not impossible to tap.  Coming from a traditional BI background the tactic of including all available attributes on a record was and I’ll confess still is a little disconcerting.  I was convinced less was safer, faster and for some tools absolutely necessary.  Most business users however don’t want IT placing limits to their data.  They like spreadsheets not only because they support custom calculations but because they flexibly scale out in both rows and columns.  They don’t want to look at a predefined selection of their data, and they don’t want to be constrained down a path where they can’t be creative in their business questions.

OEID applies the concept of Unstructured Data beyond simply the acquisition of data from unstructured sources.  Unlike traditional data marts, cubes, and reporting systems there are very few schema requirements or rules around formalized data structures.  The “record” stored in Oracle Endeca Server goes beyond the definition of a denormalized record and seems to thrive on fat ragged width records.  The user interface is like a text search engine that will consider whatever columns happen to exist on each record and the performance of returning results is fantastic.  They can enjoy complete access to all the details on a record.  The power this offers is a freedom to explore data in any direction desired.  The unanticipated questions can be answered; new and novel patterns can be explored.  OEID helps ensure your business users are not limited in the data they can work with.

The third and last reason for OEID to be part of your BI stack is how it seamlessly brings together data from disparate systems into a single interface.  This is not only a business challenge but a complicated technical one.  Portals and dashboards can certainly cross boundaries from where they source data, but that doesn’t mean individual portlets will be related to each other.  Conformed dimensions are common enough, but reporting data sets sometimes need considerable ingenuity to bring transaction facts from multiple systems together meaningfully.

OEID addresses disparate records first through an accommodating methodology called Faceted Modeling.  There are lots of great explanations for how this works, but it boils down to records that may have virtually nothing in common being automatically related by whatever attributes they do happen to have in common.  The Guided Navigation component can display as many attributes as desired from across all the records in the data store and dynamically refreshes itself as you select from that collection.  In parallel the Navigation State responds to those same choices and applies them as filters to all the visible portlets.  The work required to align these common attributes is almost nothing.  Combined these things deliver a capacity for users to see records from different systems being automatically related together.  This can provide novel ways to explore enterprise data and reveal unexpected patterns.

OEID is not going to be your one stop BI solution, there are other tools available for monitoring performance or performing complex analytics and forecasting.  Though it includes familiar reporting components I believe its true value add is as an intuitive tool for business users to freely explore their data, while still being easy for IT to support.  OEID can make your complete BI stack more effective by giving your business users a robust tool for early access to whatever data they want.  OEID helps close the gap separating business users from their data of interest.  The faster your BI solution can support change, the faster your users can react.  

This entry was posted in Endeca and tagged , , , , , , , , , , , . Bookmark the permalink.