Episode 4 – Hadoop: Year in review

A bit of Hadoop history of what we have seen happening over the last 12 months, some trends and interesting technologies. Some ups, some downs and possibly even some round and rounds, capped off with some Bold Predictions for 2016.

• A number of engagements
• Apache Nifi
• Why some Hadoop users decide to go for separate clusters per use case or (internal) client

• A broad acceptance of Hadoop in Europe
• A shift from batch workload to multi-tenant, secure platform including IoT and Real time, in memory analytic.
• Apache Ambari making our life easier all the time
• Data Governance Initiative
• Open Data Initiative (http://odpi.org)
• Public clouds offer Big Data specific environment
• Tech advances in Hive (CBO/ORC/Zlib) and Transparent Encryption in HDFS
• Apache NiFi
• The year of Apache “open community” open source
• Bold Predictions!

• What new (incubating) projects should I invest time in today, knowing that they may never be included in any distribution?
• I’ve been looking into Apache NiFi and am curious whether or not I can use it to replace Apache Flume?
• Should I go for a Hadoop appliance or not?

38:35 End