This blog post was published on Hortonworks.com before the merger with Cloudera. Some links, resources, or references may no longer be accurate.
This article is second in the series about Ozone – a distributed key-value store that can efficiently manage small and large files alike.
An earlier article introduced the Ozone design philosophy. The current article is for users who want to try Ozone. We discuss an overview of Ozone concepts, describe how to bring up an Ozone cluster and use the Ozone command shell.
2. Ozone Concepts
Ozone is made up of volumes, buckets and Keys.
- Volumes – Volumes are similar to accounts. Volumes can be created or deleted only by administrators. An administrator will typically create a volume for an organization or team.
- Buckets – A volume can contain zero or more buckets. Ozone buckets are similar to Amazon S3 buckets.
- Keys – Keys are unique within a given bucket and are similar to S3 Objects. Key names can be any string. Values represent the data that you store inside these keys, currently Ozone enforces no upper limit on the key size.
3. Installing Ozone
3.1. Get the Binaries
- Download Binary release – get the latest Ozone release from the Ozone website at https://hadoop.apache.org/ozone/.
- Build from source code – The instructions are at https://cwiki.apache.org/confluence/display/HADOOP/Building+Ozone
3.2. Deploy a cluster
There are multiple ways to setup an Ozone cluster.
- Development cluster with Docker – Instructions at https://cwiki.apache.org/confluence/display/HADOOP/Development+cluster+with+docker
- Single-node development cluster – https://cwiki.apache.org/confluence/display/HADOOP/Single-node+dev+cluster
- Multi-node Ozone cluster – https://cwiki.apache.org/confluence/display/HADOOP/Multi-node+Ozone+Cluster
4. Ozone command-line shell
4.1. Working with Volumes, Buckets, Keys
The Alpha release includes the following interfaces:
- REST & RPC API
- Command-line shell to perform object store operations.
- OzoneFS – A Hadoop Compatible Filesystem interface.
Let’s look at a few examples of using the command-line shell to perform object store operations.
4.1.1. Create a Volume
The following command creates a new volume named videos, owned by the user alice. Creating new volumes is an administrative operation.
4.1.2. Create a Bucket
The following command creates a new bucket named vacation in videos.
4.1.3. Upload a Key
This is the equivalent of ‘putting’ a file into the object store. The following command will upload the local file video-001.mp4 to the ozone bucket with the name machu-picchu.mp4.
4.1.4. List Keys in a Bucket
4.1.5. List Buckets in a Volume
4.1.6. List Volumes
5. Further Reading
- Ozone web site – https://hadoop.apache.org/ozone/
- Ozone Command Shell Reference – https://github.com/apache/hadoop/blob/trunk/hadoop-ozone/docs/content/CommandShell.md
- HDDS wiki page – https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Distributed+Data+Store+and+Applications