Learn firsthand about OpenStack, its challenges and opportunities, market adoption and Dell’s engagement in the community. My goal is to interview all 24 members of the OpenStack board, and I will post these talks sequentially at Dell TechCenter. In order to make these interviews easier to read, I structured them into my (subjectively) most important takeaways. Enjoy reading!
#1 Takeaway: Dell’s interest in OpenStack derives from market pressure on finding open source alternatives to Amazon and VMware
Rafael: Dell is one of the very early contributors to OpenStack. Why is Dell engaging in this project?
Rob: We were involved with Rackspace and NASA before there was public interest in OpenStack. Our involvement in open source cloud goes back even much further than that, because we were part of market dynamics pre-OpenStack. We were struggling to find open source cloud alternatives for our customers with a common API, a community, support and with momentum. There was a lot of pressure in the market to have an alternative ecosystem to Amazon as a public cloud and to VMware as a licensed internal cloud. So our team was looking for something in that space, and OpenStack appealed to us because it was open source, and it was a community driven effort with a group of people supporting it. So far the consortium of people involved in OpenStack has by far exceeded what I was expecting to be able to accomplish.
The open source clouds that were on the market at that time, Eucalyptus and Cloud.com, were both really interesting alternatives but neither of them had the type of collaborative community that was the goal of OpenStack. That was REALLY the difference. OpenStack started on day zero as a community of collaboration, and there wasn’t one company that was maintaining the code base. That really differentiates the project.
#2 Takeaway: Dell is focusing on operationability of OpenStack for customers rather than on code contributions to the community
Rafael: How does Dell contribute to OpenStack?
Rob: At Dell we knew from hard experience in field that the number one challenge for customers is going to be using OpenStack. Whether it worked or not, installing it, getting a repeatable build, testing and using was going to be a significant challenge that our customers would face.
We made a very deliberate choice, that rather than trying to contribute code to OpenStack, we wanted to tackle reference deployments. Our goal was to make OpenStack as deployable as possible. So, we have been mainly operationally focused on OpenStack in helping our customers and the community to get OpenStack installed.
Moving forward, we will foster OpenStack integration into the Dell portfolio and we will bring in more partners from the open source community. We are going to expand our footprint with Dell Crowbar around making OpenStack more deployable, upgradable and available.
In brief: Dell’s interest in OpenStack has been very pragmatic. OpenStack is something we really see a market need for.
#3 Takeaway: Dell Crowbar is an OpenStack deployment mechanism targeted at experienced high scale customers
Rafael: Let’s talk a bit about Dell Crowbar, your team’s deployment mechanism for OpenStack. I often times hear from the OpenStack community that it’s considered a solution targeted at the experienced admin, whereas most companies participating in the OpenStack game are building solutions that are supposed to replace that person. Why did Dell pick that route?
Rob: That’s an excellent question. Dell Crowbar is definitely a DevOps tool for customers who are planning to build the experienced run with OpenStack. The goal for Dell Crowbar is to get OpenStack running without having to read the manuals though, but it’s not a managed appliance. Dell has partners like Morphlabs who are selling a managed appliance. There are great companies out there like Piston Cloud and Nebula who are very focused on the managed appliance aspect. I believe there is plenty of room in the market for that type of solutions.
My team is focused on high scale customers. They don’t want somebody to take over their operations. They want somebody to accelerate their operations. That’s why we’re working with Chef, that’s why we are adding Puppet capabilities into Dell Crowbar. Over time, Dell Crowbar is going to be appliance-like, more and more easy to use, but our strategy is to work with a very targeted market of early adopters, open source experienced customers and to learn jointly about OpenStack best practices.
This is what I truly enjoy about open source … those communities know SO MUCH and it’s fun to have a pure relationship with them, to trade information among equals. Some of those community members are even more in field than I am, and if I switch into the “I am doing it for you!” mode, it changes the relationship.
#4 Takeaway: OpenStack raw is used by sophisticated community members, whereas OpenStack distributions target less experienced customers seeking for a managed appliance
Rafael: Let’s talk a bit about OpenStack raw vs. OpenStack distributions. What is Dell’s offering around both type of solutions?
Rob: OpenStack distributions are a bit tricky because of the way customers interpret this subject.
Dell actually supports a number of distributions and we will expand our offering in that area. For example, SUSE is using Dell Crowbar as the deployment mechanism for their OpenStack distribution. Currently our shipping products use Canonical’s distribution, our Hadoop solution uses Red Hat and CentOS.
To me, OpenStack distributions are about support. Who the customer can call to get a bug fixed?
As for the OpenStack raw part: We have a new feature in the open source release of Dell Crowbar that we call Pull From Source (PFS). I am really excited about that feature because it addresses a new market: That feature lets you pull OpenStack raw … not necessarily off trunk … we expect customers will set up their own clone off trunk and then pull from that. It allows a customer who is in dev mode and working on a new feature to test it against their own deployment, to create a real test infrastructure with multi-nodes and real workloads.
We consider this a very important use case for addressing deployability in pre-release. We expect our high scale customers who are a little bit ahead of trunk … they fix bugs, tweak and tune things … to use techniques like PFS.
During the recent OpenStack Summit Rackspace made a big point that their cloud runs on OpenStack pretty much off trunk, not on a distribution. They are pulling source code in and people are testing it, using it, fixing bugs and pushing code back. That’s exactly the type of vibrant community we want to see.
At the same time, there is a growing community that wants to use OpenStack distributions with support, certifications and they are fine with being 6 months behind OpenStack off trunk. That’s good, and we want that shadow, we want that combination of pure minded early adopters and less sophisticated OpenStack users both working together.
#5 Takeaway: OpenStack upgradability is currently the biggest barrier to adoption
Rafael: What are the biggest barriers to OpenStack adoption as of now?
Rob: That’s an excellent question. The most significant barrier to adoption is the release cycle. It delays customers starting with OpenStack because the next release is always just round the corner. OpenStack has a very fast release train.
As a community, we haven’t invested enough yet in upgrade strategies. My team’s top priority is addressing this issue. That’s what we hear in every customer meeting: “You must give me a way … if I take OpenStack Essex today, to get to OpenStack Folsom and from there to OpenStack Grizzly.” Customers want the innovation, but they want to be able to transit easily from one release to another.
On the other hand, things that I thought might be difficult turned out to be easy. For example, Linux and KVM have not been a barrier for customers, just as switching to a new API. At Dell we hit a barrier for how fast we can support new hardware versions. A lot of our early adopters like to get new hardware as fast as they possible can, and sometimes hardware shows up in a customer site before it does in our lab for testing. (laughs)
Rafael: Let me dwell a bit more on upgrading OpenStack, Rob. What does a customer specifically need to do when moving from OpenStack Essex to Folsom for example? Does he need to deploy from scratch or is there an easy upgrade path? What are the issues that a customer might face in that process?
Rob: There is now guidance in the community around that process … but it is a manual process. Our advice so far has been to do a new deployment. We’ve worked hard to make OpenStack deployment with Dell Crowbar as fast as possible, but at the same time it’s not a very good answer. We’re working on it. What we found is that most of our customers who are deploying OpenStack Essex are doing it as pilots and proofs of concept and upgrading OpenStack hasn’t been that much of a burden.
We do have customers who did much more significant investments in production deployment around OpenStack Essex and even in the previous release Diablo. They are trying to figure out the best way to do migrations. My advice to customers is always to use DevOps tools for cloud deployments, to invest time into automation to migrate your applications more easily.
#6 Takeaway: Most customers do OpenStack proof of concepts. Only very few use OpenStack in production
Rafael: Let’s talk about proof of concept versus production, Rob. How are customers using OpenStack and can you give examples for both scenarios?
Rob: Most of our customers do proof of concepts and pilots. They are putting OpenStack through dev test cycles, they are doing dev workloads. In some cases customers are going with OpenStack into production. In that case they are not using our tool, because Dell Crowbar doesn’t do automatic production deployment yet … that’s something we’re working towards. Anybody who is doing OpenStack in production is building up operational expertise. Most of those customers are OpenStack contributors, they are very active in the community. That’s what it takes today to run OpenStack in production: a certain level of commitment to the OpenStack community.
AT&T is on top of my mind as well as Dreamhost. Both of those are hosting examples. I just recently talked to a communications company that is doing a production deployment for one of their media apps, I know of a financial institution that is running a big data production cluster. I know of very few companies running real OpenStack production clusters, most of them are trying out things right now. Even Rackspace is operating OpenStack in dual mode. Customers can opt in and out. But we will get there.
#7 Takeaway: OpenStack is meant to be an Amazon and VMware alternative, for different reasons though
Rafael: I oftentimes hear two different statements: “Open Stack is an alternative to Amazon.” The other is: “OpenStack is an alternative to VMware … maybe, hopefully in two or three years from now.” Which of both statements is true?
Rob: I think that both statements are accurate, but for different reasons. OpenStack is an alternative to Amazon for public cloud companies like Rackspace, Dreamhost and many others. They want to drive an ecosystem, where they can create a cloud market which is not Amazon’ objective. Their objective is to be THE cloud. For customers who want choice and portability, OpenStack makes a lot of sense, as well as for providers who want to differentiate on service.
On the VMware side customers I talk to do want alternatives to licensed software, where costs do not scale linearly as they grow their infrastructure. They want the transparency of open source, they want flexibility. Those are real concerns for our customers, and they are willing to trade license cost for needing more operational knowledge. Over time their need to know how to run OpenStack operationally will decrease whereas licensing costs would scale linearly.
Having that alternative with OpenStack will definitely help customers and it will drive response both from Amazon and VMware.
#8 Takeway: VMware joining OpenStack is beneficial for the community, rather than a threat
Rafael: How do you view VMware joining OpenStack. Is it a threat to OpenStack or does VMware add value to the project?
Rob: That’s a really loaded question. (laughs) You have to split that question into pieces. From the Nicira acquisition side, which helps VMware coming into OpenStack as a major contributor: Nicira is doing lot of very important work, they are helping drive Quantum forward.
I think VMware has lot to offer in the community and their presence is good. I am not afraid of them coming in and then trying to establish walls and make things proprietary and disrupt the community. There is no indication right now that we need to protect OpenStack from VMware. They are showing leadership in OpenStack and frankly they have a lot of open source projects, which they are managing very robustly. These fears are overblown, and if someone tries to jockey OpenStack, the community pushes back pretty effectively.
#9 Takeaway: OpenStack has a very broad range of early adopters. But mass market adoption won’t happen until upgradability issues get resolved
Rafael: Let us speak about market adoption. Who are the early adopters of OpenStack? And when do you expect OpenStack to hit the tipping point for mass market adoption?
Rob: The early adoption is much more distributed than we have thought. Hosting companies definitely jumped in fast, because they have the operational ability and the financial incentives are very well lined. But I have also seen educational groups coming in, government entities and financial institutions which are known for aggressively adopting new technologies. Some of them are already comfortable with Hadoop … and I believe there are going to be more synergies between OpenStack and Hadoop, we haven’t seen them emerge yet … although I am little biased because my team does both. (laughs) Definitely social media, online retail, gaming companies are amongst early adopters … every single of those segments is already operationally sophisticated. By and large those companies I talk with are already strongly in Linux, DevOps and they are not afraid to get involved in these technologies. Unfortunately many companies I talk with are not in the position where their corporate culture allows them to contribute back to open source. So there is a bit of a shadow effect of companies using open stack but they don’t show up in the community.
We are not going to cross the chasm into mainstream until we solve the upgrade problem. That’s not surprising, but we are working very hard to help customers make adoption easier.
#10 Takeaway: Dell is in the process of building an entire ecosystem of OpenStack solutions for various types of customers
Rafael Rob, for all those interested in Dell’s commercial offering around OpenStack … can you give a brief overview? We already discussed about Dell Crowbar, the deployment mechanism for OpenStack as well as about some OpenStack distributions Dell is supporting. What else do we have in our portfolio?
Rob: We have a very clear vision of what we want to do in OpenStack, and it involves building up what we consider a full taxonomy. We believe there is a need for orchestration at the API level with partners like enStratus. There is need for automated deployment upgrade … what we want to build is a full experience.
We’ve done this to a certain extent with our solution offering and expanding with partners and different capabilities. One of the things we are trying to do is to start with a free open source option and then, as customers get into larger deployments, to bring in license capabilities as well as further free options with support capabilities around them. Dell has already a lot of assets around storage, networking and compute … software products, cloud services. We have a very broad view of enabling the ecosystem.
Rafael: I am in the process of setting up a wiki page at Dell TechCenter that provides customers an overview over our OpenStack offering: Dell Crowbar as our DevOps tool in its various shapes and forms, OpenStack distros we support … cloud services we build around OpenStack … hardware capabilities optimized for OpenStack.
Rob: You just summed up exactly what our challenge is at Dell. We are working with different partners to bring OpenStack to different customers in different ways. It is confusing. Your question about Dell Crowbar was right … it is targeted at a certain class of users, and I don’t want enterprise customers who expect a lot of shiny chrome and zero touch. That’s not the target by now for Dell Crowbar. We definitely need that sort of magic decoder page to help customers understand our commercial offering.
#11 Takeaway: The main challenge for the OpenStack board is defining OpenStack in detail
Rafael: My final question to you Rob is: What are the challenges for the OpenStack Board of Directors?
Rob: There are some really weighty issues in front of the board for this coming year. People should be aware of them because they really shape OpenStack.
We just formed up the Technical Board, we have got the User Committee, and we can actually start discussing the most significant bundle of issues: “What is the core of Open Stack? What defines OpenStack? What makes OpenStack … OpenStack?” For example, a project like Swift, which is an important, widely used part of OpenStack: Is it a required core component of OpenStack? Or is it an optional component?
That is a serious question because the way the OpenStack board defines what’s core and non-core, helps people bringing in new projects understand where they fit in. That also changes the incubation definition. If it’s core, it is going to have a certain incubation pattern. If it’s not, it might have a different incubation option.
That leads us straight into the question: “Is OpenStack an API? Or is OpenStack an implementation?” Today most people don’t realize that it’s really an implementation. If OpenStack is the API, then how do we provide a fitness test to make sure that someone’s implementation of OpenStack complies? We have a lot of these questions around Swift but also in the Nova, Cinder and Quantum pieces. If someone wants to replace the implementation of Swift, which is about how objects are actually stored: How do they know that they comply with the Swift API, so that they can be certified in OpenStack Swift? … even if they didn’t use the RSync implementation of Swift?
The same would be true if you reimplemented the Nova API but back ended it with VMware vCloud … is that OpenStack or is it not?
Defining API is really important. At the same time OpenStack progressed really quickly because we haven’t gone through committees and API use negotiations … and that worked. (laughs) But part of OpenStack’s maturity is to define API and the Board of Directors has to find the resources to back these decisions. It also has to find resources in the community to provide API testing.
The other thing on technical side is that we need to drive upgrade into the code. It’s a really a significant challenge to make sure that upgradeability is a key factor in the development cycle. We need to drive much more operational testing of OpenStack releases earlier in the cycle instead of waiting until the release is done. That’s not a good pattern, we need to do it earlier.
It’s important to understand what very pragmatically the challenges around OpenStack are.
Rafael: Rob, thank you very much for taking time. Be sure I will come back soon with even more questions.
Rob: (laughs) You’re welcome!
Resources
Rob Hirschfeld
Personal Blog: http://robhirschfeld.com/
Personal Youtube Channel: http://www.youtube.com/user/rollingwoodcitizen
Twitter: https://twitter.com/zehicle
OpenStack
OpenStack Foundation: http://www.openstack.org/
OpenStack at Dell: http://dell.com/openstack
Dell Crowbar: https://github.com/dellcloudedge/crowbar
Feedback
Twitter: @RafaelKnuth
Email: rafael_knuth@dellteam.com