Analytics catch-up this week...

Plenty of interesting developments over the last couple of weeks that I can see impacting Industrial IoT analytics over the next couple of years:

• I left Oracle about 15 years ago.  At that time, the database world was relatively mature, with really only incremental improvements being made.  The internet changed all that, with massive data volumes bringing an increased demand for performance and scalability.  That in turn gave rise to Hadoop (my thoughts on that here) and NoSQL databases.  NoSQL is applicable to some applications, but not to all by a long stretch because of the integrity/consistency models typically used (my thoughts on that).  Anyway, to cut to the chase, there is now a new wave of relational databases that aim to combine the best of both worlds - the rigor of the relational database model with the scalability of NoSQL.  You can find a good summary of those vendors here.


• A nice case study on self-service BI in pharma (Merck Animal Health), featuring FusionOps.  Self-service BI and analytics is a key transition that  many companies still need to fully embrace.  My research in a previous life showed that self-service can greater improve the chances of finding timely information, while simultaneously cutting the cost of support.  You can find that prior research here, courtesy of Qlik.


• Apache Spark is the latest big data analytics solution to be hyped, taking over that mantle from Hadoop.  Here's a pretty balanced piece on how those two technologies can be used productively together.


• To build on that point, there is a new breed of analytics vendors that are leveraging both Hadoop and Spark.  One of the challenges with both of those technologies is that they are open source.  So, if you use the "free" Apache distribution it's a bit like a major Ikea project - lots of assembly required.  And probably a few piece left over...Many emerging vendors take those rough Apache diamonds, deal with the integration issues, and provide other value added products and services on top.  Platfora is one of those vendors, and it just closed a new round of funding to fuel growth.


• In other Hadoop news, Hadoop basically started life as a massively scalable file system and batch processing engine.  Since then, many improvements have expanded its scope and applicability.  MapR just announced one of the latest, improving the ability to integrate real-time data into the mix.  The ability to handle real-time data effectively is going to be key to the Industrial Internet of Things.  Adding this capability into Hadoop is going to help make that platform much more attractive for industrial companies.  Storing and processing IIoT data is going to require massive storage and processing capabilities.  Hadoop can provide that as it matures and becomes easier to use.


• And finally this week, you may have noticed I've been pretty bullish on the cloud from time to time.  But, among the clients I work with, security is usually the biggest concern holding people back from cloud adoption.  My response to that is typically along the lines of "Do you really think you can afford to employ the same class of data security talent as Amazon, Google, Microsoft, or IBM?"  In most cases, like it or not, the answer to that is going to be no.  And here's a potential consequence of that:  Target’s legal bill from the 2013 holiday season mega-breach tops $300 million.  If you want a more detailed technical account of how it went down, you can find that here.