Representative End User Clients
Representative Automation Clients
Representative Software Clients
Yeah, sorry about that, never could resist a cheap pun....
So, did the world of Hadoop just become more complex, or did it take a step on the road to simplicity...? IBM recently announced that it is putting significant muscle to work on Apache Spark. Although independent of Hadoop, Spark can leverage the various Hadoop components, and is often included in that unwieldy ecosystem. Essentially, Spark is the real-time foil to the many batch-oriented components of Hadoop. Whereas at its core, Hadoop provides a distributed file system (HDFS), Spark provides a distributed in-memory processing framework. And, since we increasingly live in a (near) real-time world, the potential of Spark has shifted it from relative obscurity to center stage in the last few months, if the job market is any guide.
And now, IBM has committed 3,500 researchers to Spark-related projects, will embed Spark into many IBM platforms, like Watson and Bluemix, and donate its machine learning library to the open-source ecosystem.
So, is this a pivotal moment for Spark? IBM clearly thinks so, comparing its commitment to Spark to its earlier commitment to Linux that, in Bob Picciano's view at least, helped to make Linux a dominant technology. (For the record, IBM made a strategic commitment to Red Hat Linux in February 1999. Red Hat then IPO'd on August 11, 1999 and achieved the 8th largest first day gain ever on Wall Street. Would that've happened without IBM's support? We'll never know for sure...).
Me, I think it might be pivotal both for Spark and Hadoop. In my view, the Hadoop ecosystem needs to coalesce into some coherent, well-defined solutions before it can cross the chasm. IBM's commitment - particularly with the donation of a machine learning library - can help accelerate that. Hadoop has the potential to support Industrial IoT projects, with its ability to manage vast volumes of both structured and unstructured data. But, that will happen much more rapidly when we see solutions such as "Hadoop: Machine Learning edition", or even "<Insert name of favorite Hadoop vendor>: Predictive Maintenance edition". That shift in positioning would have much more appeal to both industrial companies and the application vendors that serve them.
So, did the world of Hadoop just become more complex, or did it take a step on the road to simplicity...? IBM recently announced that it is putting significant muscle to work on Apache Spark. Although independent of Hadoop, Spark can leverage the various Hadoop components, and is often included in that unwieldy ecosystem. Essentially, Spark is the real-time foil to the many batch-oriented components of Hadoop. Whereas at its core, Hadoop provides a distributed file system (HDFS), Spark provides a distributed in-memory processing framework. And, since we increasingly live in a (near) real-time world, the potential of Spark has shifted it from relative obscurity to center stage in the last few months, if the job market is any guide.
And now, IBM has committed 3,500 researchers to Spark-related projects, will embed Spark into many IBM platforms, like Watson and Bluemix, and donate its machine learning library to the open-source ecosystem.
So, is this a pivotal moment for Spark? IBM clearly thinks so, comparing its commitment to Spark to its earlier commitment to Linux that, in Bob Picciano's view at least, helped to make Linux a dominant technology. (For the record, IBM made a strategic commitment to Red Hat Linux in February 1999. Red Hat then IPO'd on August 11, 1999 and achieved the 8th largest first day gain ever on Wall Street. Would that've happened without IBM's support? We'll never know for sure...).
Me, I think it might be pivotal both for Spark and Hadoop. In my view, the Hadoop ecosystem needs to coalesce into some coherent, well-defined solutions before it can cross the chasm. IBM's commitment - particularly with the donation of a machine learning library - can help accelerate that. Hadoop has the potential to support Industrial IoT projects, with its ability to manage vast volumes of both structured and unstructured data. But, that will happen much more rapidly when we see solutions such as "Hadoop: Machine Learning edition", or even "<Insert name of favorite Hadoop vendor>: Predictive Maintenance edition". That shift in positioning would have much more appeal to both industrial companies and the application vendors that serve them.