Category Archives: ZooKeeper

How-to: Use Apache ZooKeeper to Build Distributed Apps (and Why)

Categories: How-to ZooKeeper

It’s widely accepted that you should never design or implement your own cryptographic algorithms but rather use well-tested, peer-reviewed libraries instead. The same can be said of distributed systems: Making up your own protocols for coordinating a cluster will almost certainly result in frustration and failure.

Architecting a distributed system is not a trivial problem; it is very prone to race conditions, deadlocks, and inconsistency. Making cluster coordination fast and scalable is just as hard as making it reliable.

Read More

Meet the Engineer: Kathleen Ting

Categories: Flume Hadoop Meet the Engineer Sqoop Support ZooKeeper

In this installment of “Meet the Engineer”, get to know Customer Operations Engineering Manager/Apache Sqoop committer Kathleen Ting (@kate_ting).

What do you do at Cloudera, and in what open-source projects are you involved?
I’m a support manager at Cloudera, and an Apache Sqoop committer and PMC member. I also contribute to the Apache Flume and Apache ZooKeeper mailing lists and organize and present at meetups, as well as speak at conferences,

Read More

Apache Hadoop in 2013: The State of the Platform

Categories: Avro CDH Flume Hadoop HBase HDFS Hive Hue Impala Mahout MapReduce Oozie Pig Sqoop YARN ZooKeeper

For several good reasons, 2013 is a Happy New Year for Apache Hadoop enthusiasts.

In 2012, we saw continued progress on developing the next generation of the MapReduce processing framework (MRv2), work that will bear fruit this year. HDFS experienced major progress toward becoming a lights-out, fully enterprise-ready distributed filesystem with the addition of high availability features and increased performance. And a hint of the future of the Hadoop platform was provided with the Beta release of Cloudera Impala,

Read More

Apache HBase AssignmentManager Improvements

Categories: Community HBase ZooKeeper

AssignmentManager is a module in the Apache HBase Master that manages regions to RegionServers assignment. (See HBase architecture for more information.) It ensures that all regions are assigned and each region is assigned to just one RegionServer.

Although the AssignmentManager generally does a good job, the existing implementation does not handle assignments as well as it could. For example, if a region was assigned to two or more RegionServers,

Read More

Apache ZooKeeper 3.4.5 Has Been Released

Categories: Community ZooKeeper

Apache ZooKeeper release 3.4.5 is now available. This is a bug fix release covering 3 issues, one of which was considered critical. These issues were:

  • ZOOKEEPER-1550 ZooKeeperSaslClient does not finish anonymous login on OpenJDK
  • ZOOKEEPER-1560 Zookeeper client hangs on creation of large nodes
  • ZOOKEEPER-1376 zkServer.sh does not correctly check for $SERVER_JVMFLAGS

Stability, Compatibility and Testing
3.4.5 is a stable release that’s fully backward compatible with 3.4.4.

Read More