Using Cloudera Deploy to install Cloudera Data Platform (CDP) Private Cloud

Using Cloudera Deploy to install Cloudera Data Platform (CDP) Private Cloud

Following our recent Cloudera Data Platform (CDP) overview, we cover how to deploy CDP private Cloud on you local infrastructure. It is entirely automated with the Ansible cookbooks published by Cloudera and it is reproducible on your local host with Vagrant.

CDP is an enterprise data cloud. It provides a powerful Big Data platform, built-in security with automatic compliance and governance of data protection, as well as policy-based, metadata-driven analytics for end users.

Deploying a CDP Private Cloud cluster is not a straightforward task. Therefore, we present a way of getting a local cluster up and running in a few simple steps. We will deploy a basic cluster composed of two nodes, one master and one worker. In our cluster, we will be running the following services: HDFS, YARN and Zookeeper.


You can use the local infrastructure of your choice to deploy CDP Private Cloud. In this tutorial, we will be using Vagrant and VirtualBox to quickly bootstrap two virtual machines that will serve as the cluster’s nodes.


VirtualBox is a cross-platform virtualization application. Download the latest version of VirtualBox.


Vagrant is a tool for building and managing virtual machine environments. Download the latest version of Vagrant.

Once Vagrant is installed, you need to install a plugin which automatically installs the host’s VirtualBox Guest Additions on the guest system. Open a terminal and type in the following command:

vagrant plugin install vagrant-vbguest


Cloudera Deploy is run from inside a Docker container. When executed, it bootstraps the cluster. Follow the official Docker instructions to install Docker on your machine:

Getting started

Bootstrap your nodes

A Vagrantfile is used to configure and provision virtual machines on a per-project basis. Make sure you have an ssh key on your host machine before going forward. If none is provided, the Quickstart (next section) will generate a SSH keypair. Create a new file called Vagrantfile in your working directory and paste the following code:

box = "centos/7"

Vagrant.configure("2") do |config|
  config.vm.synced_folder ".", "/vagrant", disabled: true
  config.ssh.insert_key = false
  config.vm.box_check_update = false
  ssh_pub_key = File.readlines("#Dir.home/.ssh/").first.strip
  config.vm.provision "Add ssh_pub_key", type: "shell" do |s|
    s.inline = <<-SHELL
      echo #ssh_pub_key >> /home/vagrant/.ssh/authorized_keys
      sudo mkdir -p /root/.ssh/
      sudo echo #ssh_pub_key >> /root/.ssh/authorized_keys
      sudo touch /home/vagrant/.ssh/config
      sudo chmod 600 /home/vagrant/.ssh/config
      sudo chown vagrant /home/vagrant/.ssh/config
  config.vm.define :master01 do |node| = box :private_network, ip: "" :forwarded_port, guest: 22, host: 24011, auto_correct: true :forwarded_port, guest: 8080, host: 8080, auto_correct: true
    node.vm.provider "virtualbox" do |d|
      d.memory = 8192
    node.vm.hostname = "master01.nikita.local"
  config.vm.define :worker01 do |node| = box :private_network, ip: "" :forwarded_port, guest: 22, host: 24015, auto_correct: true
    node.vm.provider "virtualbox" do |d|
      d.customize ["modifyvm", :id, "--memory", 2048]
      d.customize ["modifyvm", :id, "--cpus", 2]
      d.customize ["modifyvm", :id, "--ioapic", "on"]
    node.vm.hostname = "worker01.nikita.local"

The master01 node has the master01.nikita.local FQDN and the IP. The worker01 node has the master01.nikita.local FQDN and the IP.

Now run the following command:

It creates two connected virtual machines which constitutes a small cluster.

Edit your local /etc/hosts file by adding the following lines: master01.nikita.local worker01.nikita.local

Now connect to master01 using ssh:

Add or edit the following lines to the /etc/hosts file: master01.nikita.local worker01.nikita.local

Repeat the operation by connecting to worker01.

Download the quickstart script

The script will setup the Docker container with the software dependencies you need for deployment. Download it to your host machine using the following command:

curl -o

Run the quickstart script

The script will prepare and execute the Ansible Runner inside a Docker container.

chmod +x

You should see the cldr build-version #> orange prompt. You are now inside the container.

Create an inventory file

Navigate to the cloudera-deploy folder:

Create a new file called inventory_static.ini which contains your hosts:


master01.nikita.local host_template=Master1








Configure the cluster

Set use_download_mirror to no in the definition file located at examples/sandbox/definition.yml to avoid triggering behavior that relies on public cloud services.

Run the main playbook

ansible-playbook /opt/cloudera-deploy/main.yml -e "definition_path=examples/sandbox" -e "profile=/opt/cloudera-deploy/profile.yml" -i /opt/cloudera-deploy/inventory_static.ini -t default_cluster

The command creates a CDP Private Base cluster using your local infrastructure. More specifically, it deploys a cluster with HDFS, YARN and Zookeeper.


Cloudera Data Platform can be deployed in various ways which makes it a versatile option when considering a data platform. In this article, we described how to deploy a CDP Private Cloud cluster with Cloudera’s official deployment scripts. This allows the user to test the platform locally and make relevant business decisions. From there, you can add services to your cluster as well as configure CDP Private Cloud’s built in components.


Should you encounter any issues with SSH between the host and the two virtual machines, you can force the installation of Virtualbox Guest Additions for master01 and worker01 by adding the following line to their individual configurations in Vagrantfile:

node.vbguest.installer_options =  allow_kernel_upgrade: true 


The script can abruptly exit if it detects that the SSH_AUTH_SOCK path is not properly defined or empty. If you encounter this error, first run the following command:

This returns the path to the unix socket used by ssh-agent, which needs to be added as the variable SSH_AUTH_SOCK to the quickstart script for ssh to work properly; your quickstart script should now look like this:

SSH_AUTH_SOCK variable in

In this example case, the socket’s path is “/run/user/1000/keyring/ssh”.

Leave a Reply