By Charles King, Pund-IT, Inc. March 16, 2016
Analytics has been such an elemental part of the “big data” news cycle over the past couple of years that it’s all too easy to forget that success is anything but guaranteed. That’s partly due to the complexity of the data preparation processes that provide the foundation for successful analytics, a subject that doesn’t get near as much attention as it deserves. Fortunately, IBM’s new partnership with Datawatch, a purveyor of innovative self-service data preparation solutions, offers a good excuse to consider the issue more closely.
Step by step data prep
To set the stage, remember that a central tenet of big data’s value for business users is the ability to utilize and even blend data from multiple sources and formats, including JSON, machine log files, PDFs, text reports and web pages.
The challenge is that few, if any, of these fit handily into structured formats for analysis. Sure, the process can be performed manually, but doing so can place a considerable strain on an organization’s staff of data professionals. In fact, according to a report by Forrester, business analysts and data scientists spend up to 80% of their time manually preparing data. That is likely also a factor in Forrester’s estimate that just 12% of corporate data is typically used in corporate decision making.
While it’s logical to focus high-paid staff on high-value information, that also undercuts a central big data value proposition – exploring little used information to find out that what you don’t know you don’t know. But those same complexity and time issues also prohibit most line-of-business workers from using and benefitting from analytics solutions.
That’s really where Datawatch comes in, mainly via its Monarch self-service data preparation solution that is used to manipulate, filter, enrich, blend and combine disparate data sets, and visual analytics solution which includes built-in connectors to all major big data, ECM and relational sources. As a result, users can take information from virtually any source, from traditional data bases to multi-structured documents and convert it into easily analyzed formats.
Sounds good, but what is the company doing with IBM that’s new and different? First and foremost, Datawatch’s self-service data preparation solutions can be used by IBM Watson Analytics and IBM Cognos Analytics customers. As part of the agreement, IBM will also resell Datawatch Monarch. The Datawatch/IBM solution also offers additional features, including,
- Automation and sharing features, including auto-save of data preparation steps in human-readable formats
- Easy data preparation, including a simple UI and 80 pre-built functions
- Integration of disparate data using a join analysis recommendation engine
- Security/governance support, including masking to protect private user data and other sensitive, compliance-controlled information
Analytics and big data have developed a cachet in the IT industry, based on the often remarkable insights and business benefits they are capable of delivering. But like any elegant structure, they require a strong foundation, careful processes and attention to detail in order to stand the test of time. Data preparation is just one of those processes, but solutions like Datawatch Monarch can make it one of the most effective.
In essence, the company has turned what is often a complex, time-consuming slog into a self-guided walk in the park. Numerous other customers have gained significant advantages from Datawatch’s innovations so it seems entirely sensible that IBM’s big data clients will have the chance to simply send data accessed and prepared in Monarch directly to Watson Analytics and Cognos Analytics.
In essence, this is a deal that should benefit everyone involved. Plus, if IBM and Datawatch fully deliver on their self-service promises, the result will make what is already one of the industry’s strongest analytics portfolios even more compelling.
© 2016 Pund-IT, Inc. All rights reserved.