PowerFlex Cloud Storage in AWS – a slice of Dell's Project Alpine story

Introduction

With the announcement of Dell’s Project Alpine at DT World in May 2022 (DT World Project Alpine blog), there has been lots of chatter and focus on Dell Technologies and their ‘all-in’ multi-cloud journey.

 

As part of Dell's Multi-cloud by Design approach, Project Alpine will see us bringing our Dell ISG storage software (specifically PowerFlex, PowerStore, PowerScale and ObjectScale) to the three major public clouds - AWS, Azure and GCP. This storage software, traditionally tethered to Dell hardware, will be decoupled from the hardware and moved into the public cloud marketplaces ready for purchase.

Ultimately, customers will be able to consume Dell storage by having these virtual storage solutions running on cloud-based infrastructure, think EC2 instances in AWS – with the same look and feel of the on-prem version of your Dell storage solution.

 

From an operations perspective, Dell’s engineering teams will have done all the grafting behind the scenes porting the APIs, command line and GUI to the public cloud’s infrastructure – which in turn will provide a consistent operational experience whichever of the three hyperscalers you have the software running on.

 

The cloud and on-premises OSes will be interoperable, allowing customers to move data back and forth using replication technologies and being able to purchase additional storage from their hyperscaler of choice. An ideal solution for customers with existing pre-commits with hyperscalers that want to consume Dell storage.

 

Focusing on just the PowerFlex element of Project Alpine for this post, customers can purchase PowerFlex storage software right now in the AWS marketplace with PowerFlex Cloud Storage, with plans for same to be available in GCP and Azure in the future. Customers will have their own AWS account to manage the service fees and Dell will supply the code and tools to deploy and manage it.

PowerFlex Cloud Storage is currently available in single-Availability Zone (AZ) or multi-AZ configurations in the following four regions in the United States: 

• Oregon (us-west-2)

• California (us-west-1)    

• Ohio (us-east-2)

• Virginia (us-east-1)

What is PowerFlex Cloud Storage?

 

Previously branded as Dell’s ScaleIO, PowerFlex software-defined storage (SDS) is a software-only solution that uses existing servers' local drives and LAN to create a virtual SAN that has all the benefits of external storage. PowerFlex utilizes the existing local storage devices and turns them into shared block storage.

The primary PowerFlex software components are:

• Meta Data Manager (MDM) - Configures and monitors PowerFlex
• Storage Data Server (SDS) - Manages the capacity of a single instance (or node for on-prem) and acts as a backend for data access. The SDS is installed on all instances/nodes contributing storage to PowerFlex
• Storage Data Client (SDC) - lightweight device driver that exposes PowerFlex volumes as block drives to the application that resides on the same instance/node on which the SDC is installed

For on-prem PowerFlex deployments, PowerFlex is available in appliance, custom node and rack consumption options with PowerFlex Manager providing the management and orchestration functionality.

With PowerFlex Cloud Storage in AWS, the PowerFlex software is deployed to AWS’ public cloud infrastructure either manually or by leveraging the PowerFlex deployment wizard via an AMI (Amazon Machine Image), which is much, much faster (30 minutes!). The AMI contains all the required PowerFlex software and a web-based wizard to install and manage your PowerFlex clusters.

 

                                                                                Figure 1. PowerFlex AMI

PowerFlex Cloud Storage currently only supports 2-layer configurations with compute (SDC) and storage (SDS) residing on separate instances. SDC must be deployed by customer post-install of PowerFlex.

PowerFlex Cloud Storage in AWS - Highlights and Advantages

• Enables massive scalability: Start small and grow to petabytes with extremely high throughput and low latency for mission critical applications.
• Extreme performance: Provides extreme performance with sub-millisecond latency
• Durability: Unique multi-AZ durability by using PowerFlex fault set architecture allowing for rapid data protection in the event of instance or AZ failure
• Data mobility: Data can be moved back and forth between on-prem and cloud PowerFlex deployments easily using replication technologies, along with the ability to move data between various PowerFlex cloud deployments
• Fast and easy deployment:
• Complete automated deployment in less than 30 minutes
• Scaling without physical hardware acquisition channels
• Network automation
• Code upgrades

Now, let's run through some of the steps you'll see when deploying your PowerFlex solution in AWS using the deployment wizard.

Deployment steps using PowerFlex Deployment Wizard

Now, let's run through some of the steps you'll see when deploying your PowerFlex solution in AWS using the deployment wizard.

1.    Input parameters for your PowerFlex cluster

 

The deployment wizard interacts with AWS APIs to configure and deploy the cluster. After launching the PowerFlex deployment wizard with the AMI, you can begin to deploy PowerFlex itself by selecting the Deploy PowerFlex button and specifying the region, instance type, availability and usable storage required for your PowerFlex solution.

Figure 2. Deployment wizard

Choosing the deployment type:

When choosing the deployment type for your PowerFlex Cloud Storage solution, customers will have the choice of AWS' Elastic Block Storage (EBS GP3) or AWS' EC2 instances which will leverage NVMe SSD drives.

You can deploy both virtualised and bare metal instances to meet the required computing infrastructure with the following three instance types available for PowerFlex deployments, with single AZ and multi-AZ options.

• i3en.metal (NVMe)
• i3en.12xlarge (EBS)
• c5n.9xlarge (EBS)

The use of dedicated EC2 instances is recommended for performance, scalability, availability and cost.

EBS volumes:

• Block-level storage, network attached at instance level
• Reserved I/O throughput
• Persistent storage - the data will persist if a PowerFlex cluster, or the associated EBS volumes, are shut down
• Performance IOPS: up to 360,000 IOPS/volume

EC2 instances (NVMe SSD drives):

• Locally attached at instance level
• No reserved throughput I/O
• Storage is ephemeral - upon shut down of the cluster, date will be lost
• Performance IOPS: up to 2 million IOPS

                                               Figure 3. Supported EBS and EC2 instance options

        Single AZ or Multiple AZ:

With PowerFlex Cloud Storage in AWS, Regions and AZs define the fault domain for the network infrastructure.

When inputting the required parameters for your PowerFlex deployment,

see figure.3 above, you will be able to select the availability required for your PowerFlex solution, Single-AZ or Multi-AZ.

Selecting a Multiple AZ deployment allows you to place different SDSs in different AZs thus creating multiple fault sets. If one zone is down, the PowerFlex cluster is still available. So you could have three fault sets, one per AZ, if multiple AZ is selected.

Selecting a Single AZ deployment will mean one fault set and will reduce the resiliency of your cluster.

    Figure 4. Fault set view in PowerFlex GUI

The tool will automatically calculate the number of instances required and provides estimates performance figures that will be achieved for your configuration.

    Figure 5. Instance number and Performance estimate (prior to actual deployment)

2.    Deploy PowerFlex

Select the ‘Deploy Now’ button in the PowerFlex wizard to begin the deployment of the cluster.

Once the deployment is kicked off, the Virtual Private Cloud (VPC), Internet Gateways (IGW), and required infrastructure (e.g. Presentation Server, MDMs, SDS instances, Protection Domain, storage pool etc) will be created and configured.

Figure 6. PowerFlex deployment – in progress

 

                Figure 7. Successful PowerFlex deployment 

3.    Provide Proxy instance access

To access the PowerFlex GUI from the current network location, a new rule must be added to the Security Group of the Proxy instance from the AWS console.

                                        Figure 8. Security Group rule addition for Proxy Instance

Once complete, we can navigate to the proxy host IP address via RDP and log in to the PowerFlex GUI as the admin user.

Figure 9. PowerFlex GUI

2.    Create and configure SDC instances

a. Create dedicated AWS instances for the SDCs

For each SDC you need for your cluster, you must create a dedicated AWS instance. This can be launched from the AWS console leveraging one of the supported AMIs (SUSE linux, RHEL, Ubuntu or Windows server).

Figure 10. Sample RHEL AMI for SDC instances

b. Retrieve SDC package and PEM key

Before we SSH to the instance IP, we'll retrieve the SDC package and PEM key. These will be available for download from the deployment wizard UI, as shown in the highlighted buttons in Figure.11 below.

Figure 11. PEM Key and SDC package

  

       c. Install and configure SDC instance(s)

• Open up SSH client, log in and download the RPM for the SDC - and then point the package to the MDM cluster IPs

Figure 12. SSH view

• The SDC will be installed, configured and the systemd service started - and once complete the SDC will be available for adding and mapping volumes

Figure 13. SDC Instances view

Figure 14. Volumes view

Conclusion

So that's all I have for you, for now, with Project Alpine and our PowerFlex in AWS cloud solution - stay tuned here for more of Dell's Project Alpine story as and when we see PowerScale, Objectscale and Powerstore storage software solutions being deployed to the AWS, GCP and Azure public clouds. 

#IWORK4DELL

Opinions expressed in this article are entirely my own and may not be representative of the views of Dell Technologies.