Oracle Endeca Week : So Just What Is Endeca?

Visualizing complex relationships shouldn't require a Ph.D. in data science or hours of manual labor. With Python's NetworkX library, you can map intricate connections—like the web between authors, books, and readers—in minutes, turning raw data into actionable network graphs instantly.

Mapping Complex Data Relationships with Python

So whilst we're waiting for Oracle to release the 11.1.1.6 version of OBIEE, I thought it might be interesting to take some time to look at Oracle's newest BI tool : Oracle Endeca.

You may have noticed in the press and on the web that Oracle agreed to buy Endeca back in October last year, and it's just recently that all the paperwork has been settled and Endeca brought into the Oracle fold. According to the press, Oracle paid just over $1.6bn for Endeca, and most commentators thought the acquisition was a smart move by Oracle, especially as HP recently bought Autonomy, one of Endeca's competitors, for around ten times that amount. So what is Endeca, how does it relate to Oracle Business Intelligence, Essbase and the Oracle Database, and how do you get started creating an Endeca project?

Endeca the company, has been around since 1999 and was based in Cambridge, MA. For most of it's time, Endeca has focused on the e-Commerce market, providing the technology behind websites such as HMV.com, Sony's online store and Autotrader.com; what all these sites have in common is a feature called "faceted search", where users are able to query and drill-into the retailer's dataset, using any combination of attributes.

In traditional e-commerce inventory query systems, you had to start at the top by selecting, say, mens clothes or women's clothes, then select from mens trousers, mens shirts, mens coats and so on. Eventually you'd get to the 36" mens trousers in black, but it was a very linear, and "guided" route through the data. Websites that used Endeca's technology, by contrast presented a list of dimensions and attributes down one side, and the user could make any selection from them to narrow down their search. All of this happened lightening-fast, and with a back-end that was very easy for the customer to maintain.

So Endeca focused on this e-commerce market first, and developed the MDEX engine to support this, marketing it as a column-store, rapid-development query engine that allows "faceted searches" across lots of different, "jagged" data sets (i.e. data sets that don't have the same data model, but with some commonality between them). About a couple of years ago, Endeca took the core technology from this product and created a standalone BI tool called Endeca Latitude, complete with dashboard components, an ETL tool, dashboard and report designers, and a story that revolved around "agile BI", based on the fact that the MDEX engine doesn't require a strictly-defined data model. By the time of the Oracle acquisiton, Endeca's product line looked like this, with Endeca Infront, the product behind these websites, accounting for the majority of revenue.

Endeca Infront is aimed at retail, manufacturing and other industries with large inventories that need to be searched through and analyzed. Endeca, like nQuire funnily enough, positioned Infront and Endeca Latitude as tools to search and explore large data sets, and Infront was squarely positioned as back-end data server that could then be integrated in with a company's e-commerce platform. Endeca had a lot of success with Infront, and then went on to create Endeca Latitude, a more general-purpose BI tool that shared much of its underlying technology with Infront.

Endeca Latitude has two main components (leaving the MDEX engine aside for a moment):

• Endeca Studio, used for creating applications and dashboards, based on the open-source Liferay Portal Enterprise Edition, and
• Endeca Data Integrator, based on an open-source ETL tool called CloverETL

Using Latitude Studio is a similar process to using tools such as Oracle Business Intelligence's analysis and dashboard editors, with components that you can add to the canvas, data sources that are connected to them, and interactivity through guided navigation areas, clickable graphs and so on. All of this is delivered through the Tomcat J2EE application server with the tooling, and the end-user experience, all web-based. The screenshot below is of the sample application that comes with Endeca Latitude, and we'll see more on how the development process works later in the week.

So with Endeca Infront, Endeca's main market proposition was to add faceted search behind websites, using tools such as their Page Builder application that made delivering this type of content fairly straightforward. The angle that Endeca took with Endeca Latitude was around agile business intelligence, based on the fact that the MDEX engine didn't require you to spend a long time defining a strict dimensional data model, and you could add in additional data sources that had elements of commonality but didn't necessarily have all the same sets of columns or tables (something Endeca referred to as "jagged" data sets). We'll look tomorrow at how Endeca's products complement Oracle's existing product set, and how tools like Endeca Latitude will fit in with tools such as Oracle Business Intelligence, and the Hyperion/Essbase product line.