Big Data:
Cloudera: Apache Spark: A Delight for Developers
Apache Spark is well known today for its performance benefits over MapReduce, as well as its versatility. However, another important benefit – the elegance of the development experience – gets less mainstream attention. In this post, you’ll learn just a few of the features in Spark that make development purely a pleasure. Read more.
Datameer: Considerations for SQL on Hadoop
SQL on HadoopSQL interfaces for Hadoop have gotten a lot of buzz over the past year because companies are anxious to utilize current skill sets to leverage Hadoop rather than hiring new talent or re-training. However, there are multiple considerations a decision maker must keep in mind when evaluating an interface to Hadoop. In their recent report “Choosing Your SQL Access Strategy for Hadoop”, Gartner analysts Nick Heudecker and Merv Adrian do a good job outlining the differences among the SQL offerings, Hive-based SQL, and direct SQL execution, and offers guidance for those evaluating the space. Read more.
DevOps:
Cloudera: Apache ZooKeeper Resilience at Pinterest
scovery, dynamic configuration management and distributed locking. While it’s versatile and useful, it has failure modes that can be hard to prepare for and recover from, and if used for site critical functionality, can have a significant impact on site availability. It’s important to structure the usage of ZooKeeper in a way that prevents outages and data loss, so it doesn’t become a single point of failure (SPoF). Here, you’ll learn how Pinterest uses ZooKeeper, the problems we’ve dealt with, and a creative solution to benefit from ZooKeeper in a fault-tolerant and highly resilient manner. Read more.
Cloudera: Inside Apache Oozie HA
One of the big new features in CDH 5 for Apache Oozie is High Availability (HA). In designing this feature, the Oozie team at Cloudera had two main goals: 1) Don’t change the API or usage patterns, and 2) the user shouldn’t even have to know that HA is enabled. In other words, we wanted Oozie HA to be as easy and transparent as possible. In this blog post, I’ll explain how Oozie HA works and how it achieves those goals. Read more.
Docker: Introducing private repos, webhooks and more
The Docker.io team has been working hard on a number of new services and we’re excited to roll them out to you today. With this release, we aspired to provide services to help users share repos with others, drill-down into repo contents, and automate and link workflows. Read more.
OpenStack:
Adam Huffman: Simple OpenStack monitoring with Ganglia and Nagios
I’ve been running an OpenStack-based cloud for a while. While the modularity of OpenStack is a strength, helping the fast pace of development, it also means that the interactions between components can be quite complex, with all the possibilities for obscure errors this implies. For instance, upgrades in one component (such as a GlusterFS backend) can cause problems elsewhere. Here’s a description of some simple monitoring I’ve added to ameliorate this. Read more.
Boden Russel: OpenStack nova VM migration (live and cold) call flow
OpenStack nova compute supports two flavors of Virtual Machine (VM) migration: cold migration -- migration of a VM which requires the VM to be powered off during the migrate operation during which time the VM is inaccessible and hot or live migration -- zero down-time migration whereupon the VM is not powered off during the migration and thus remains accessible. Understanding these VM migration operations from an OpenStack internals perspective can be a daunting task. I had the pleasure of digging into these flows in the latter part of 2013 and as part of that effort created a rough outline of the internal flows. Read more.
CERN: CERN Cloud Architecture - Update
In the last OpenStack Design Summit in Hong Kong I presented the CERN Cloud Architecture with the talk “Deep Dive into the CERN Cloud Infrastructure” https://www.openstack.org/summit/openstack-summit-hong-kong-2013/session-videos/presentation/deep-dive-into-the-cern-cloud-infrastructure . Since then the infrastructure grown to a third cell and we enabled ceilometer compute-agent. Because of that we needed to perform some architecture changes to cope with the number of nova-api calls that ceilometer compute-agent generates. Read more.
FOSS Knowledge Base: OpenStack Havana on Ubuntu 12.04 LTS – Single machine setup
A straightforward tutorial on setting up OpenStack havana on the latest LTS release of Ubuntu. Read more.
Matthias Runge: A stable and custom theme for Horizon
During the OpenStack Icehouse development cycle, we got in Horizon the ability to use additional python packages by simply dropping in a file in the openstack_dashboard/enabled dir. In the rest of the article, I'll propose a method to customize the look and feel, without breaking during package updates. It will work, if you have a Horizon package in Icehouse release RC1 or later. Read more.
Mirantis: Trove + Cassandra = Love: NoSQL Database Solutions and the OpenStack Ecosystem
NoSQL databases are systems for data storage and retrieval that do not primarily use the now-dominant RDBMS model: tabular data structures, organized relationally and accessed using Structured Query Language (SQL). Instead, NoSQL databases employ a host of methods, such as schema-free key-value pairs, intended to map better to a growing class of problems that may be difficult to solve with RDBMS. For example, some problems are best approached using data-structures (for example, trees) that are hard to represent with relational tables, algorithms that are difficult to express in SQL, or problems whose efficient solution requires creation and access to very large, unstructured and/or distributed databases using massive parallelism. Read more.
Opensource.com: How Openstack stores data in the cloud with Cinder
Without solid storage, the data of the cloud lives only in the moment. Within OpenStack, storage comes in two flavors: object storage, which stores chunks of information, and block storage, which is more analogous to the traditional idea of a disk drive image. Block storage in OpenStack is handled by Cinder, which provides the infrastructure for managing volumes and sharing them with the other components in OpenStack. We chatted with John Griffin, the Program Technical Lead for the Cinder project, to better understand how it functions within OpenStack, why it is important and what is on the roadmap for the Icehouse release. Read more.
Opensource.com: Get your IT career kickstarted with OpenStack
Whether you love living in the cloud or still cling to your desktop applications whenever possible, it has become increasingly clear in recent years that the cloud is where computing is headed. And if you’re seeking to keep your skills relevant to the IT jobs of today, and tomorrow, understanding the technology that underlies cloud services is critical. Fortunately, the cloud offers many opportunities for using open source software up and down the stack. If being on the cutting edge of cloud infrastructure interests you, it’s probably time to take a look at OpenStack. Read more.
Piston: Piston OpenStack 3.0: Now with Policy-Driven Storage Provisioning
When our CTO, Joshua McKenty, first drew up his vision of Piston OpenStack™ 3.0 enhanced storage for the team, we all thought it was crazy (and likely impossible without completely changing our product). Thankfully, we were wrong. This was not the first time we’ve reacted that way to one of Josh’s ideas, but part of the thrill of doing this job is finding a way to make the seemingly impossible…possible. And in today’s second post in our blog series covering some of our newest product features and enhancements, we’ll be talking about the addition of policy-driven storage provisioning in Piston OpenStack 3.0. Read more.
Rackspace: An Introduction To Developing On OpenStack From DeveloperWeek
At DeveloperWeek Conference & Festival 2014, Rackspace Director of Training and Education Tony Campbell showcased how to develop on OpenStack. In his keynote presentation, “Developing On OpenStack,” Campbell introduced OpenStack the open source cloud platform co-founded by Rackspace in 2010, which has quickly become one of the fastest growing open source projects in history. OpenStack now boasts more than 13,000 individuals from 132 countries. Read more.
RafStack: Docker: Fundamentals & OpenStack Integration w/ Eric Windisch
Docker is an Open Source containerization technology that is not simply an alternative to virtualization. There are various benefits of using Docker with or within OpenStack. Eric Windisch of the Docker team will cover fundamentals of Docker and OpenStack integration. Read more.
SearchTelecom: eNovance on why OpenStack public cloud is the telco platform of choice
As telecom providers look to expand their service offerings and increase revenue, many are looking to public cloud services. But building, deploying and maintaining a public cloud can be complex and costly. That's where French cloud and managed service provider eNovance, which recently expanded to North America, found a niche in helping telecom providers build and deploy public clouds using OpenStack. Read more.