The term “Cloud” is really confusing some times. While the basic concept is becoming more and more clear, what is not is that we have multiple Cloud types to contend with.

1. Datacenter Cloud – This makes perfect sense to me. My VMware experience and all of the VMware/virtualization kool-aid out there jives very well. The Datacenter Cloud is just like what I have in my datacenter. Just with some added layers of management and automation on top. I am cool with that.
2. Application Cloud – This is where I am getting lost… and, I feel like this may be where things are going, especially for environments sized like my Corporate environment. Application Clouds include Google Gmail, Salesforce Database.com, Google Docs, VMware/Salesforce VMforce, and Windows Live.

In my Corporate environment, we are trying to make a conscious decision to move towards Cloud based resources. We figure that if we can simplify the internal infrastructure to commodity components and start leveraging usage based hosted models, we can actually reap some of the benefits. Starting to acknowledge the trend now and make decisions based on the trend makes it easier to grow into a “Cloud” environment.

So, back to this darn crossroads… Datacenter Cloud or Application Cloud??

The biggest issue I am running into is my data in the Application Cloud. Like most applications, all of our applications need the app tier and the data tier. In the Application Cloud model, the database lives in one location and the app in another. Suddenly, not only do I need to worry about access times and experience for the end user getting to the application, but the access times between the various Cloud providers. AKA – things could be significantly slower.

Additionally, what about backing up the data and accessing those backups. We may have documented policies stating retention values, locations, etc… We all know that song and dance. However, each individual component theoretically operates individually.

Security become another issue to address with the Application Cloud environment. I “trust” that my data is secure. However, I am addressing security as credentials. Each service has their own authentication system. So, how do we, as IT professionals, manage these? Existing solutions provide for their own management structure (aka – web console for administration and creation of user-level accounts) or use agents that run on workstations for a pseudo-single sign-on experience. But, what I am looking for is some level of integration between my existing authentication mechanisms and what exists in the Cloud.

One of the final speedbumps in this Cloud crossroad conundrum is how can we ensure that our data is being backed up reliably and that restoration mechanisms are timely and accessible via my company versus needing to hunt down a Cloud provider support person? Many companies have regulations and policies regarding data retention and many Cloud providers cannot deal with those policies. Plus, the business may need to “feel in control” of their data.

Alright… the light is turning green… which way… WHICH WAY…???

I know, I have this awesome SUV, I am going to make my own path. Instead of left or right, I am going to forge straight ahead. With the direction we need to go, we cannot just chose one or another. There are too many advantages for both to ignore them… For those systems with their own authentication methods regardless of being hosted or internal, to the Application cloud with you! For those that we deem important to have more control over, Datacenter Cloud for you!

As long as we make a conscious decision to move towards some kind of Cloud based solution (be it Application Cloud or Datacenter Cloud), we are moving in the right direction. I feel confident that I am not the only one in the IT world with these concerns and the answers will come in good time. By moving towards Cloud infrastructure now, we can adapt when the technology advances and be more agile and lite. The development of policies that handle external authentication systems and data access (backups/recoveries/SLAs/etc…) and business buy in (perhaps with ROI and cost savings over alternatives) will help drive this path home… and perhaps the business will pay to pave this new road I am blazing. Otherwise, these darn bumps are going to kill me.

Happy Clouding!

© Bill for Gestalt IT, 2011. | Cloud Crossroads
Read more posts categorized as All, Cloud Computing, Server Virtualization

The unique Field Day℠ concept brings together as equals two groups that rarely meet: Product vendors and independent thought leaders. The event includes 11 delegates and just five speaking slots. Both vendors and delegates are nominated and selected by an independent team of volunteers based on interest, technology and passion. The resulting discussions are packed with technical information, with feedback and questions flowing both ways across the table.

Presenting Sponsors

Just seven companies will sponsor and present at Tech Field Day 4. Delegates will spend two to four hours with each vendor, learning about their products, technology, and relevance to modern IT practices. The Gestalt IT Tech Field Day events have attracted some of the biggest names in information technology, and this event is no exception.

Tech Field Day 4 begins Thursday, November 11 at the NetApp campus in Sunnyvale. We will spend the morning learning about their unified storage technology and looking into their latest product updates. After lunch, the delegates will move to the PlugandPlay Tech Center to learn about the virtual I/O products developed by Aprius. Avere Systems is up next, presenting their tiered NAS appliances.

Friday begins with data management startup, Actifio. Intel hosts the delegates next, covering converged networking and their latest Xeon server processor technology. Friday afternoon will be hosted by The Hoffman Group in San Jose. There, Field Day veterans, SolarWinds, will cover their management and monitoring software. We will conclude the event with a look at the cloud backup capabilities made possible by Asigra.

Selected Delegates

Only eleven hand-picked delegates will attend this Tech Field Day event. Our selection process focuses on the unique characteristics of thought leaders: Independence, open-mindedness, technical knowledge, and influence are key criteria. The global team of delegates come from a variety of backgrounds, including server virtualization, enterprise storage, networking, security, and systems administration.

Field Day alumni, Brandon Carroll, Matt Simmons, Derek Schauland, and Robin Harris will be joined by seven new delegates: Jon Owings, Edward Aractingi, Frank Owen, Chris Dearden, Bill Hill, Paul Miller, and Joe Onisick. Each has distinguished himself in his community, and each was selected by a team of delegates from past events.

About Gestalt IT and Tech Field Day

Complete delegate and sponsor lists are available at the Gestalt IT web site. We encourage those interested in IT infrastructure topics to follow the delegates on Gestalt IT (gestaltit.com), Twitter (bit.ly/TFD4TL and bit.ly/TFD4TS), Facebook (bit.ly/TFDGFB), LinkedIn (bit.ly/TFDLI), Flickr (bit.ly/TFDpics) and Vimeo (bit.ly/TFDvids).

For the first time, Tech Field Day will be streamed live over the Internet. Visit the special Tech Field Day 4 live stream page during the event to follow along!

The whole is greater than the sum of the parts. This is the concept behind Gestalt IT – that experts in a variety of disciplines are stronger when they work together. As a group, we formed a media organization focused on promoting community among independent thought leaders in IT infrastructure topics. Our productions include the Gestalt IT web site and the series of Tech Field Day events. Gestalt IT Media LLC community organizer, Stephen Foskett, can be contacted at http://GestaltIT.com, sfoskett@gestaltit.com, or (508)451-9532.

© Stephen Foskett for Gestalt IT, 2010. | Announcing Tech Field Day 4: This Week in San Jose!
Read more posts categorized as All, Cloud Computing, Desktop, Events, Exclusive, Featured, Networking, Security, Server Virtualization, Storage, Tech, Top Story

IT is crossing a chasm, and we may not be prepared for the new direction we are taking

I’m an IT revolutionary. I talk all the time about the quaint backwards “state of the art” in enterprise IT, what with its (many) decades old protocols, paradigms, and practices. What we call modern is really just a charade of faked-out old-fashioned open systems infrastructure: Pretend servers talking to fake disks over frankenstein networking technology.

What is Flexible IT?

I’ll be doing a “Twitterview” about Flexible IT today with NetApp. While I’m sure most companies would like to define flexible IT as “what we have in our product catalog,” I’m not going to play that game. IT has to change its ways or The Techie/Business Schismwill get us all.

I see the roadmap ahead as one journey over two paths:

1. Tactically, IT infrastructure groups must immediately begin implementing technologies that are responsive to user needs and focused on improving the performance of the business applications they support. This is what NetApp is calling Flexible IT.
2. Strategically, IT infrastructure must change entirely, implementing revolutionary platforms to support the next-generation apps that are being widely developed. Call this “cloud” if you must, but just about everything we take for granted today will lose significance in this new era.

Flexible IT (as it stands today) is tactical. IT infrastructure must be entirely virtualized to enable “right now” provisioning and reconfiguration. This means embracing enabling technologies like server-side hypervisors, blade computing, converged networking and I/O, and advanced storage technologies.

What Does it Mean to be a Service Provider?

But real IT flexibility will mean higher-level abstraction. The next applications won’t want to run on “a server” – even a very flexible one. These apps will require a programming platform that abstracts away the entire concept of “server.” Although there will always be servers and disks and such, they will fade into the background, hidden under a layer of application services.

You know how some people like to think of themselves as “service providers”? They’re on the right track, though I’m not sure that they (yet) grasp the full significance of that concept.

A service provider has to provide the services required by consumers or it will fail. This means IT infrastructure has to prepare the environment demanded by IT application developers, rather than just shuffling around the chairs and tables and hoping customers will stream in.

What happens when the IT applications group asks the IT infrastructure manager to implement Microsoft’s Azure Services Platform? What if they want to run on App Engine or Force.com? The head-scratching in the VMware community about SpringSource is a sign that IT infrastructure just doesn’t comprehend what’s happening (yet).

Stephen’s Stance

This is a strategic shift, and we have to be ready. Being ready means changing not just what we do but also how we do it. The systems we employ in a decade might look familiar inside, but the process of delivering services on them will be entirely different. Making this shift requires real flexibility from IT, not just another round virtualization fake-outs.

Image credits: Highway construction Guizhou S0404 by sweart, VW engine by Grant Foskett

© Stephen Foskett for Gestalt IT, 2010. | Flexible IT and the Path to the Services Future
Read more posts categorized as All, Cloud Computing, Storage

Think about how you deploy virtual machines. Usually you will deploy one machine at a time, which is a good thing if you only need one server. But usually in larger environments, you will find that applications or application systems are not based on a single server. You will find larger environments that consist of multiple servers that will segregate functions, so for example, your landscape could consist of a DB server, an application server, and one or more proxies that provide access to your application servers.

If you are lucky, the folks installing everything will only request one virtual machine at a time. Usually that isn’t the case though. Now, this is where vCloud Director comes in. This will allow you to roll out a set of virtual machines at a time as a landscape. But it doesn’t stop there, since you can do a lot more because you can pool things like storage, networks and you a tight integration with vShield to secure your environment. But this should give you a very rough idea of what you can do with the vCloud Director. For a more comprehensive overview, take a look at Duncan’s post here.

Anyway, let’s dig in to the technical part.

There are plenty of blog posts that cover how to set up the CentOS installation, so I won’t cover that at great length. If you are looking for that info, take a peek here. If you want to install the Oracle DB on CentOS, take a look here to see how it’s done.

Here are some tips that might come in useful during the install:

• Use the full path to the keytool. There is a slight difference between /usr/bin/keytool, /usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre/bin/keytool and /opt/vmware/cloud-director/jre/bin/keytool. Be sure to use one of those, and if the commands to create and import your self-signed certificates are not working for some reason be sure to try a different one.

If you just simply create a database and browsed through the installation guide, you might have a hard time once you install the binary. Basically you run the “dbca” tool to create an empty database. If you by any chance forget to create the database files and run the installation binary (or the vCD configuration tool for that matter), you will receive an error while running the .sql database initialization scripts under /opt/vmware/cloud-director/db/oracle. The error message will tell you that there was an error creating the database.

Well, if only you had read the installation guide properly. Bascially what you do is start up the database:

sqlplus "/ as sysdba"
startup


Make sure that the path you use in the “create tablespace” command actually exists. If they don’t you need to perform “mkdir $ORACLE_HOME/oradata” first. Then create the tablespaces and corresponding files:

Create Tablespace CLOUD_DATA datafile '$ORACLE_HOME/oradata/cloud_data01.dbf' size 1000M autoextend on;
Create Tablespace CLOUD_INDX datafile '$ORACLE_HOME/oradata/cloud_indx01.dbf' size 500M autoextend on;

Now create a seperate user that we will give right for the database. The password for the user is the thing you type after “identified by”:

create user vcloud identified by vcloud default tablespace CLOUD_DATA;

Make sure that you give the user the correct rights to perform all the DB operations:

grant CONNECT, RESOURCE, CREATE TRIGGER, CREATE TYPE, CREATE VIEW, CREATE MATERIALIZED VIEW, CREATE PROCEDURE, CREATE SEQUENCE, EXECUTE ANY PROCEDURE to vcloud;


Now run the setup script, or run the configure script and you should be set to go.

© Bas for Gestalt IT, 2010. | Shorts: VMware vCloud Director installation tips
Read more posts categorized as Cloud Computing, Server Virtualization

You may not know Caringo, but you have probably heard of cloud storage, EMC Centera, and Dell DX. Read on to learn the link!

Now that the hype of “cloud everything” is subsiding, organizations are getting down to work deploying cloud storage to do actual useful tasks. The march from CAS to cloud to object storage has seen high-profile high-end flare-ups (think EMC Centera and Atmos) but the bulk of work is done by more pedestrian (think lower-cost) hardware and software. Through it all, Paul Carpentier has been at the forefront. Now his company, Caringo, is back in the news, delivering much-needed storage service features like multi-tenancy, named objects, and dynamic caching.

For essential background, check out my article, From CAS to Cloud: Revolutionary Storage

The Back-Story of Caringo

The “Caringo” company name refers to its three founders, CTO Paul Carpentier, President Jonathan Ring, and CEO Mark Goros. Carpentier is the man behind CAS pioneer FilePool, which EMC acquired and markets as Centera. The three formed Caringo and launched the CAStor product in 2006 as a software alternative to Centera.

Although you may not have heard of Caringo, you may have encountered their product in the form of the Dell DX object storage system. Many were puzzled when Dell, known for its EMC-powered storage offerings, embraced Caringo for object storage, but those familiar with the products weren’t surprised. Caringo’s approach is much more in line with Dell’s image of affordability, simplicity, and commodity products, and their relationship with EMC is increasingly shaky due to their recent acquisition strategy.

Caringo’s CAStor is a software product that transforms commodity servers into a scale-out object repository. It is all-inclusive, with compliance, tiering, spin-down, and replication part of the total package. Like most CAS and cloud storage solutions, CAStor uses a simple HTTP interface for client access, with “gateways” available for NAS along with some native support from applications.

What’s New in CAStor 5?

Caringo has set a course for the service provider market, adding essential features like multi-tenancy and flexible permissions to version 5 of CAStor. Although still pitched as an object store, CAStor 5 is close enough to be thought of as a cloud storage platform.

The ability to support and segregate multiple “tenants” is a holy grail for service provider storage systems and a key ingredient of cloud storage solutions. CAStor 5 can be segmented into multiple domains, each with its own security and authentication and each subdivided into “buckets” for different applications. This would be useful both for a public service provider and an internal-only solution, since segmenting applications is relevant in the enterprise as well.

CAStor 5 no longer clings to system-assigned names for objects, allowing users to assign their own names for public consumption. This is a huge advancement for CAS, and was one of the key differentiators of cloud solutions which often directly serve content to web clients. Another “ripped from the cloud” feature is dynamic caching, allowing high performance access to popular content, again useful for direct client access.

Stephen’s Stance

Caringo seems reluctant to wear the “cloud storage” mantle, but their product has been steadily moving in that direction. CAStor 5, with its multi-tenancy, segmented security and authentication, named objects, and caching, looks an awful lot like Amazon S3 and the rest. But the hype around “cloud storage” is dying away. Businesses looking for functionality rather than marketing labels will find a lot to like in CAStor.

© Stephen Foskett for Gestalt IT, 2010. | Caringo Bulks Up CAStor For Cloud Services
Read more posts categorized as All, Cloud Computing, Storage

We need to move beyond fake disks and deploy application-centric storage

Change is not a word normally associated with storage, and revolution is practically unheard of. Today’s modern enterprise storage systems and networks employ massive resources to do one simple thing: Emulate the basic hard disk drives used over three decades ago. But cracks are appearing in our mausoleum of fake disks: Application developers are discovering the value of object storage, and storage systems are appearing to support this need.

I also wrote about this two years ago, proclaiming that We Need a Storage Revolution and forecasting The Techie/Business Schism

The CAS Revolution

Caringo founder and CTO, Paul Carpentier, rose to prominence around 2000 at FilePool, one of the prime movers in the content-addressable storage (CAS) space. I recall a light going off in my head as Paul introduced me to FilePool’s CAS technology back then, imagining the possibilities of the concept. Files would be stored based on “what they were” rather than “where they were” and could be organized according to application needs rather than the conventional “extent of blocks” or tree heirarchy.

CAS discarded decades of filesystem and block storage baggage, introducing a new method for storing and retrieving data that better-matched the burgeoning web and enterprise applications of today. I had seen the failure of the first wave of storage service providers from inside StorageNetworks, and it was this desire for a real storage revolution that led me to dive into cloud storage at Nirvanix almost a decade later. Although I am now on my own, I remain convinced that the future belongs to storage systems that look nothing like today’s SAN and NAS.

Shortly after that 2001 meeting, EMC acquired FilePool and launched it as the Centera product line. But CAS systems quickly ran into a serious roadblock: Conventional applications cannot read and write to unconventional storage systems like Centera. EMC pushed key software vendors (especially in the archiving space) to create special Centera interfaces, and the industry bogged down developing the XAM standard. Other companies, like Seven Ten Storage Software, jumped in to help with the translation from proprietary CAS interfaces, but the transition from legacy files and blocks to object storage has been long and slow.

Cloud Storage: Another Dimension

Meanwhile, in an alternate dimension, web developers realized they had a serious problem. They were developing applications that scaled massively, spanning servers and exhausting conventional filesystems. Conventional systems just wouldn’t cut the mustard.

Since they were soaking in web applications, these developers applied the lessons of web services to storage: Why not just make an HTTP connection and ask for an object by a unique ID rather than walk a filesystem tree? Why not encapsulate the “state” of this request in the request itself rather than make a lasting connection and association between the client and server?

Thus was born cloud storage, and it was bookseller Amazon who opened the floodgates with their 2006 introduction of a “Simple Storage Service” or S3. They allowed anyone to store and retrieve objects from their massive web services infrastructure. S3 and similar services from Rackspace, Nirvanix, and others, are special-purpose web servers, and their simple interfaces are wonderfully attractive to web developers. For example, this WordPress-based blog uses cloud storage to serve images to your browser!

Similarities in CAS and Cloud

Although developed from vastly-differing starting points, CAS and cloud storage are essentially similar: Both reject conventional blocks and files in favor of object storage; both organize data with metadata databases; both multiply and scale out. There is one other major similarity between CAS and cloud storage: Both are attractive to service providers.

Imagine you operate a business that stores data for customers. You would want a flexible infrastructure that would scale with demand and segment each “tenant” from others for security and performance. As we learned at StorageNetworks, conventional SAN and NAS systems just weren’t meant to work in this kind of environment. Whether operating an internal service or a public cloud, service providers require something entirely different.

Cloud storage was designed from the start with service providers in mind, embedding per-object and per-”bucket” security, scalability, and abstraction between hardware and clients. Although quite complex to design, cloud storage is amazingly simple to use, provided an application can interface with it.

CAS wasn’t designed like this. Systems like EMC’s Centera were created for the needs of applications like enterprise archiving, but secure storage of content and extreme scalability are critical here as well. But early CAS systems didn’t need simple web-style interfaces or extreme hardware abstraction. These were enterprise systems, after all.

The CAS/Cloud Colission

CAS wasn’t exactly successful. Although object storage found a niche in enterprise archiving, the enterprise storage world has mostly continued with blocks and files. The major storage vendors all have some kind of object storage, but most are repurposed NAS rather than dedicated CAS like the Centera.

Although much skepticism has been raised about cloud storage in the enterprise, its impact on application development cannot be denied. Indeed, the majority of developers are now focused on programming platforms that abstract both compute and storage from conventional operating systems. The next generation of applications will run in “platform as a service” environments first, and cloud storage is a key component.

Storage vendors are rapidly moving to rework their conventional systems for cloud use. Although block and file systems from 3PAR, NetApp, Isilon, Symantec, HDS, HP, and others are useful in cloud environments, unconventional CAS becomes more valuable here. This is where EMC, Mezeo, and Caringo (with Dell) shine, and why HDS bought Parascale, NetApp bought Bycast, and what Overland could do with MaxiScale. In the mean time, “gateway” products from Nasuni, Cirtas, StorSimple, Twin Strata, and Asigra are awfully interesting.

Stephen’s Stance

The storage revolution is coming, whether we in the industry are ready or not. Developers are voting with their feet, targeting cloud storage and application platforms rather than conventional filesystems. Although the market for cloud storage products is slow to develop, the cloud storage concept will eventually dominate the landscape.

It seems most likely that this revolution will decimate the storage industry as we know it today. Unable to push high-margin storage arrays into the ballooning cloud space, product vendors will see their market share eroded by service providers with no use for these expensive systems. Monolithic file and block will soldier on in the new legacy applications, but the action will inevitably slip away.

The likely winners will be those who can leverage commodity hardware for scale-out cloud storage use. The proliferation of cloud platforms will settle down, with a few gaining traction and the rest discarded. Then we will see companies like HP, Dell, and Oracle rise to lead the storage sales charts with massive volume shipments to service providers.

Disclosure: I used to work for StorageNetworks (which is now defunct) and Nirvanix.

Image credit: Barcelona Graffiti by Aeioux

© Stephen Foskett for Gestalt IT, 2010. | From CAS to Cloud: Revolutionary Storage
Read more posts categorized as All, Cloud Computing, Storage

Much has been said about virtualization platform interoperability. VMware is constantly pushing forward with trying to achieve ANSI standards for common virtualization components and functions. One of which is the most useful… the virtual hard disk format. In theory, I should be able to take a virtual disk from my Hyper-V environment and import into my vSphere environment. I have seen this work on my workstation when I imported my Windows XP Mode drive into my Workstation setup!

However, the non-VMware virtualization providers have been combining their forces into a new cloud computing platform, called OpenStack (slogan: “Open source software to build private and public clouds”). OpenStack is a software layer that supports the operation of multiple hypervisors… including Xen, KVM, QEMU, and UML (User Mode Linux).

Installation (per installation instructions) occurs on a Debian/Ubuntu based OS. So, out of the box, it appears as though we are going to be working with a potential Type 2 hypervisor…or multiple hypervisors on a single physical platform (?!). Resource management between the multiple hypervisors looks to be interesting, then.

Much of the effort has been focused on “avoiding vendor lock-in” in virtualization services. Apparently, people do not like being locked into a single vendor for services. Although, I would reckon that any of the virtualization companies involved with the OpenStack would love to be in the situation VMware finds themselves in… at the forefront of the virtualization market. I would like to go out on a limb and state that VMware loves the success it finds, but it is still working on interoperability between virtualization providers and products. Look at the ANSI work that VMware has gone through for virtualization standards. Additionally, the conscious decision to allow 3rd party providers to interact with the vSphere environment (ex: offloaded VM antivirus, backups, monitoring, etc…). Hyper-V and XenServer do not have this level of flexibility. The VMware ecosystem is such that many other vendor products (ex: System Center Virtual Machine Manager) can manage them. The same cannot be said for the other vendors.

Surprisingly, what is bringing this to the forefront of the virtualization industry is the inclusion of Microsoft into the mix. Suddenly, a known closed source company and one of the most prolific companies that distribute a vendor lock in set of products is trying to get in on the action. Prior to this, it could be said that OpenStack was primarily open source hypervisor products. However, with the inclusion of the Hyper-V hypervisor, that model has changed.

The inclusion of Hyper-V really opens an interesting avenue of discussion, though. What is Microsoft’s intention here and how is it going about it? Are they going to open Hyper-V to the open source community? Will this run natively in the Linux environment the other hypervisors are running in or will any hosting provider need to operate a Windows environment to support this?

Supporting multiple hypervisors can and will lead to nightmares as each hypervisor product is being developed independently of one another. So, it is possible that one hypervisor is being developed at 2x the rate as the others. How are updates going to be handled? What if one requires different libraries than another?

What makes the big three virtualization vendors so useful is the inclusion of sophisticated and intuitive GUI and APIs for management. However, the management of the environment appears to be restricted to the CLI of the host server. Right away, this will drive away any non-savvy customers. So, I doubt that customers will flock to the OpenStack environment en masse. Sure, some customers have the resources to handle this, but not enough to make a difference… Statement: yes. Impact: no.

As far as cloud infrastructure is concerned, what is wrong with vendor lock in, really? The virtualization providers know how to interact with their own products and drastically increase their functionality and power. Sticking with VMware ensures compatibility and functionality across products. What is the need to run virtual machines in a Hyper-V environment locally and UML in the cloud? While the servers could interact with each other, management functions will differ significantly, and the portability is drastically limited. Contrast this with using the VMware vCloud initiative in your local and public clouds. Management and APIs are identical regardless of the location of the virtual machines (either in Public or Private clouds). Portability is not an option because the virtual machines exist on a common platform.

My feeling is that this is very much about some lesser virtualization providers ganging up to try and beat VMware in the cloud infrastructure game. While the intent is great, the complexity is much greater as there are more hypervisors to support and potential for instability and abnormal product growth. Rather, this is more of a proof of concept project for what “the cloud should be like”. However, the OpenStack environment and the other virtualization providers should take note of the VMware vCloud initiative as it is showing how true cloud operation should work.

Links:

OpenStack:http://www.openstack.org/

Installation Instructions: http://wiki.openstack.org/NovaInstall

http://ostatic.com/blog/as-microsoft-and-vmware-square-off-keep-your-eyes-on-openstack

http://gigaom.com/cloud/microsoft-joins-openstack-to-add-hyper-v-support/

http://cloudcomputing.info/en/news/2010/07/vmware-answers-to-rackspace-on-openstack.html

© Bill for Gestalt IT, 2010. | Cloud Fight – OpenStack Steps Up
Read more posts categorized as Cloud Computing, Server Virtualization

Leadership and management can almost be interchangeable when an organisation is in a steady-state and lets be honest, most IT organisations have been in some kind of semi-steady-state for the last decade. IT Delivery became very procedural in a glacial sort of way.

Radical change in many IT organisations is simply shuffling the cards around without adding anything else in; perhaps moving what already is being done to a third party. In fact, arguably the trend to outsource can be pretty much attributed to the steady-state of the organisation. IT became something to be managed as opposed to something to lead. IT leaders who want to change things rapidly and quickly often find themselves at conflict with their own internal organisations; it becomes easier to accept the status-quo as opposed to embracing the change.

Arguably, IT’s success in becoming ubiquitous in many organisations has become it’s biggest blocker to change; IT has become so large and so important that it’s ability to respond to the business has become seriously compromised by this importance. Look at the sheer amount of work it takes to roll out a new version of Microsoft Office to an organisation? Okay, it is a moot point whether upgrading Microsoft Office buys very much to an organisation but lets look at the number of corporates who are still running with huge amount of risk because changing from IE6 to a later version is allegedly so complex.

This apparent complexity has lead to an inward focus on the delivery of IT and paradoxically this focus on the delivery of IT has significantly damaged it’s delivery. It has become more important to the manage the delivery than to deliver.

However, we have already seen changes in the way that applications are being delivered and developed; agile development techniques are probably used for more than 50% of our applications internally. I suspect that this is an industry trend and your mileage will certainly vary, yet I remember when I first ran an agile development team nearly ten years ago that this was considered to be extremely unusual and that the ‘Agile Manifesto’ was some kind of hippie movement.

The move to agile development techniques has been remarkably quick compared to say that of take up of object orientated techniques which can only be attributed to that there is something right and easily implementable about the techniques.  You can find the Agile Manifesto here.

Agile development techniques have some key points that I believe point to the direction that all IT organisations need to take.

• Customer involvement in writing the stories which are to be delivered are key; every successful agile development has customer involvement at it’s heart. Don’t hide your people away from the Business; introduce them slowly if needed but stop making every communication between the IT organisation and the Business a management function.
• Leaders can develop without been forced into some management cookie-cutter mould. A great developer gets to work with many different developers and gets to disseminate their greatness but there is no pressure on them to become managers or team-leaders.
• Teams should be allowed to organise themselves and define their own structures. They will often surprise you and if you allow them to break down organisational silos, you will find that their effectiveness is significantly increased.
• The acceptance that no plan survives contact with the enemy. This does not mean that there should be no plan but there needs to be flexibility and options.
• Simplicity and the art of doing the minimum to deliver is essential. For a long time I have preached the concept of ‘Constructive Laziness’; do nothing which does not need doing and do what needs doing in the easiest manner.

How many organisations are ready to embrace this? I think this where the real difference between Leadership and Management will show, to implement such principles means taking up the mantle of Leader and dropping some of the comforting tropes of the Manager.

© Martin for Gestalt IT, 2010. | Agile IT leads to Agility?
Read more posts categorized as Cloud Computing, Featured

Chuck talks about storage as being software, go read his blog; there’s little I would disagree with there at all; well, until he starts talking about EMC products! However, I would go much further and suggest that we are getting to the stage where all infrastructure at a very real level is becoming software. Although I am not totally enamoured with the Intel-focussed monoculture, it has allowed a common hardware platform and it is flattening the playing field when it comes to hardware differentiation.

In fact, to differentiate your hardware is going to become increasingly expensive and hard, so why bother? Yes, there will be edge cases where hardware differentiation will be a key USP but in 90% of all use-cases; an Intel box assembled from bog-standard off-the-shelf parts will be good enough.

If we then factor in pervasive virtualisation in the data-centre; we have a platform which has become pretty much standardised and commoditised. I would like to see more ‘standardisation’ in the virtualisation arena but it’s not that bad at present and you really do not have that many choices.

So this has some interesting results; it lowers the cost of entry for new players in the market, if they no longer have to spend time developing a hardware platform and packaging their infrastructure product as a device but simply as a ‘soft appliance’; they can get it out there a lot quicker. If they can now rely on the virtualisation layer providing them with a common way of accessing hardware services; they can develop a lot quicker and they can try things much faster. The cost of failure is a lot less; it also allows integration with other types of infrastructure to be tested without a lot of expense; this is a plus for both the developer but also for the interested Infrastructure specialist.

The speed to market is greatly enhanced and they can get it front of idiots like me who will download the appliance and have a play and it’s not just storage appliances. There are products like Traffic Squeezer which do WAN Network Traffic Acceleration; there are more open-source router projects than you can shake a stick at.

I am slowly building a virtual Data Centre out of open source or at least free products; I want to see how far it would be possible to get. But I’m not suggesting that anyone would do this for real today, although I can see some Cloud providers having a really good bash at it. This approach probably would not make sense for most companies as a complete strategy but as part of a strategy, it may well be worth considering. There are large companies out there who invest in start-ups simply to develop stuff for them to use internally; the advent as ‘infrastructure as software’ without a huge investment in tooling up to build hardware means this a very viable approach for the biggest companies.

Google do it, Amazon do it but often you hear the comments, ‘Well they employ very clever people, so it’s easier for them!’ Well, don’t you employ clever people? Or are you saying that all your employees are second rate.

It took me a long time to come round to the idea that commoditisation and standardisation could drive innovation at all levels; I now believe that it could. It’s not just about more and more Web applications; it could drive a new wave of infrastructure innovation as well.

This leaves some interesting conflicts on the horizon for companies like VMware; perhaps VMware might want to get into infrastructure appliances but that would lead them into direct competition with the Mothership. Infrastructure as software; interesting times.

Here’s some links of things worth looking at or playing with, it possibly includes things which are not strictly infrastructure but are interesting anyway. Some are great, some not so great; some show great potential.

• Traffic Squeezer
• Openfiler
• Samba Ctdb
• Vyatta
• Amanda
• OpenDedup

And of course, there is Ubuntu Server; which will let you build your own cloud for free; there are various ZFS-based storage appliances. You can build your own appliances as well, packaging up and integrating components in the way you want.

One area where EMC have shown great foresight and that is investing in the vSpecialist team and building a team out of diverse specialities because as infrastructure becomes software; the cross-over between the infrastructure disciplines will become even more mandated. Now the vSpecialist team may be very focused on the ‘EMC product set’ but if I was an SI or another vendor playing in this space; I would be looking at doing something very similar in the near future.

© Martin for Gestalt IT, 2010. | Infrastructure is Software
Read more posts categorized as Cloud Computing, Networking, Storage

I’d like to take an example without being too specific from my own experience; currently in the race to innovate, many corners are cut. Every screen whether it is a TV screen or the screen on a hand-held games device is now potential target for a product.

Our developers build proof-of-concept systems on servers under their desks at best and often on desktops which might be laying around spare; proof-of-concepts are demonstrated and then proof-of-concepts are products. Cycle times for delivery of new product are often measured in weeks, not the months and years of the past.

No thought is made to non-functional requirements, the capability is all and it’s now a product with paying customers who expect it to be up and running. In fact, the mention of non-functional requirements are enough to send customers running for the hills, they don’t want to know.

But of course, in reality we have to retro-engineer in all those requirements that are required to make it a supportable service. And whilst we are doing this, our people are in a constant struggle to keep this system up. At times, our people do remarkable things in this battle and they should be commended for doing so.

But there must be a better way than PCs under desks and developers acting like MacGyver to get a new service together? I think there is, I think it’s a dynamic, scalable, on-demand infrastructure; call it Cloud, call it what you will. Users should be given the ability to throw-up an environment quickly and easily with almost no thought made to the availability, scalability requirements which will surely be required if the development is a success.

Yes, some use is made of public Cloud but I think that pales into insignificance when compared to use of commodity hardware which is just lying about; this has been going on since the PC reared it’s ugly head but as PC’s have become more powerful with the ability to run server operating systems and especially with the rise of Open Source, every developer has the ability to build a ‘production environment’ without permission.

Are they doing things in Cloud? Yes, they surely are but I suspect it is a more common situation for a infrastructure team to be presented with a skunk-works system built out of commodity and be told to make it live…..tomorrow.

And it is then that the Cloud comes into play, it would be much better to have built an environment which allows developers the flexibility and agility that they require, having them work in an environment which can be promoted to a production environment rapidly. An environment which is as cheap and as flexible as the development teams believe that their PCs are but gives the infrastructure teams the supportability that they require.

Non-functional requirements whether we like it or not are just bolt-ons; after-thoughts. They are seen as the obstacles which stop customers gettting the new services which they require, it’s time to move on. I still believe that developers should pay attention to non-functional requirements, I still believe that systems should be designed with availability, scalability, recoverability etc in mind but I think that this is now needs to provided in such a way that it is transparent to all.

Of course this transparency should be completely transparent and open allowing rapid migration between providers, private and public…but that is another story!

© Martin for Gestalt IT, 2010. | No Longer Functional
Read more posts categorized as Cloud Computing