It’s been nearly five months since my last blog entry – far too long! But the last few months have been really busy. As well as developing Agility, we’ve been talking at conferences and telling analysts about the novel ideas behind Agility. We’ve also been building relationships with early adopters and, if you add to this, the fact that we’re currently raising another finance round you’ll get the picture.

But the good news is that we’ve taken Agility out of the demonstrator and into the customer site. This alpha release is solving real problems for customers. We’ve summarised our initial case study on Rozmic Wireless where Agility manages a SaaS deployment – managing a private cloud and cloudbursting onto the public cloud under the control of automated policy. We’re also contracted to a public sector organisation and we’ll describe how Agility resolves those particular issues soon.

With the alpha release you can manage your own hardware and software resources as a private cloud and then cloudburst out to the public cloud for scalability (and for fault-tolerance). You can insert your own policy to manage how and when you scale your applications, control which resources are used for what purpose, control load-balancing, insert specialist debug and monitoring at run-time, and do a whole lot more! Agility let’s you start small and add resources incrementally to take more and more advantage of cloud technology. All this without impacting upon your legacy systems.

Although the types of resources you can manage is still limited for this early release, and management consoles have relatively simple functionality, you can still use the alpha to get started in the cloud and begin to appreciate the power of the platform that Agility delivers.

If you’d like to explore the benefits of cloud computing using Agility then please contact us. We’d be happy to help you plan a PoC/Pilot using this alpha release. We plan to release a beta in August with full GA in January 2010.

At the moment there’s a clear difference of opinion amongst cloud aficionados on the subject of trust. The Web 2.0 optimists argue that informal trust is good enough. They’ll say “I’ve rarely had a problem with EC2/FlexiScale/Mosso and when I do I just restart my app some time later. Oh, and I get some credits too when that happens”. The business skeptics on the other hand say “How can I expect some third party to take the same care and attention over my critical applications that I do myself? How can I trust someone else not to lose, accidentally expose, or sell my confidential data?”

Trust is often treated in these cloud discussions as if it was a binary property. I either trust ‘the cloud’ or I don’t. But things aren’t as simple in the real world. I might trust James to look after my pint whilst I go to the restroom but not to look after my Porsche (if I had a Porsche, that is). Whereas I’d trust my colleague Barry with my Porsche but I wouldn’t leave him alone for 5 minutes with my pint. Trust between two individuals / organisations is a function of their previous interactions.

In the business world (and in the pre-nuptial arrangements of the very wealthy) trust is codified in legal contracts and in the legal system that supports those contracts. So, when you ask me if I trust my bank to look after my money then I’d say … (no, wait, that’s a bad example). When you ask me if I trust my airline to deliver the seat I’ve booked then I’d say ‘yes, in the main’. But if they don’t, then I know that there is a contract in place and an audit trail and that there are laws that will result in my being compensated for their failure to deliver. This knowledge bolsters my trust and is ultimately what makes my business with the airline, indeed all business, possible. I don’t think we’ll see broad take-up of cloud infrastructure until we can capture the contractual relationships between cloud customers and vendors (and incidentally I believe that in the cloud this distinction will become increasingly blurred).

At Arjuna we think this can be done by allowing service requirements to be clearly defined and then by constructing service agreements (effectively contracts) between independent parties intended to support those service requirements. (Thomas Bittman of Gartner has recently blogged on how potentially complex some of the requirements might be – http://blogs.gartner.com/thomas_bittman/2009/01/08/virtual-cloud-privacy-is-gray/ ). These agreements need to be very dynamic in nature and to be sufficiently flexible so that they are capable of supporting everything from complex, tightly defined business relationships backed by legal documentation, to the very loose and non-contractual relationships. Once an agreement is in place both parties can then build their own audit trail recording their view of how they and the other party have performed. This knowledge can be used to inform further agreements i.e. build trust, and to help to settle (or avoid) disputes between the parties.

Business requires contracts and, if it means business, then so does the cloud.

If you’re attending, drop me a line at steve.caughey -at- arjuna.com.

Last week, Professor Paul Watson presented federation on Arjuna’s behalf at the London CloudCamp to good reviews. Phil Wainewright picked out federation as a core them of the conference (read his blog). I’m currently in the Bay Area talking to a number of companies about this subject. It’s a hot topic.

Federation is seen as an important aspect by a number of companies including Cisco – Doug Gourlay recently outlined Cisco’s thinking around cloud and Phil Dean presented at London CloudCamp (again, see Phil Wainewright’s blog) – it was also covered at the Web 2.0 Summit although sometimes under the term ‘hybrid cloud’, a cloud that somehow spans private IT and the public clouds.

Personally, I don’t like the term ‘hybrid’ – it suggests a solution that is neither one thing nor the other. We prefer to talk about the ‘federation’ of clouds. It’s our belief that enterprises won’t present the public cloud to their internal users in its raw state, but that they’ll access the cloud-hosted services via a proxy/gateway which will represent the cloud to the users, and the enterprise to the cloud. By this means the enterprise will control access and manage the costs. This internal representation of the cloud looks to its users like a cloud – it’s opaque, and it behaves like a cloud. In fact it is a cloud – a ‘private cloud’, albeit one which doesn’t directly manage any resources itself, but a cloud nevertheless. If such a private cloud also had the means to manage internal resources and to make dynamic decisions as to whether service would be delivered by those resources or by the public cloud then we can begin to see the means by which internal IT infrastructure could over time be increasingly subsumed into the cloud.

Our Cloud computing product, Agility, is focused on delivering precisely this functionality. Agility instances can simply broker between the enterprise and one or more public clouds, or can be assigned a set of IT resources, offering users service delivered by one or the other set of resources, or by some combination of both. By this means Agility acts as an on-ramp to the cloud and allows the IT department to begin to experiment with cloud computing in a gradual, incremental way, without any need for disruption to existing service.

The answer of course is that no-one will. People won’t commit to THE cloud – but they would commit to A cloud. They’d commit to a cloud that they could trust, and in the absence of implicit trust, to one they could hold accountable. They would, for example, let their applications execute in their own department’s cloud – one that managed the departmental resources and provided its users with services and supported their service requirements, without the users needing to concern themselves with the IT mechanics involved. They’d do so if they could have clearly defined Service Agreements with that local cloud and insight into the Policy operated by that cloud.

At some point the local cloud might be connected to another department’s cloud within the same enterprise (so that resources can be shared) and this would require the creation of Service Agreements between those departments. If this happened the users wouldn’t care – or rather they wouldn’t care so long as their Service Agreements and the local Policy (the embodiment of the things they DO care about) were maintained. And when the enterprise connected to other enterprises or to some external cloud they still wouldn’t care. If concerns were present, or if new concerns arose, then the cloud user would need to:

• be sure that their Service Agreement (or the Policies enforced by their department, enterprise or government) clearly specified what was, and what was not, acceptable and
• place sufficient trust in their local cloud provider – sufficient that is to believe they could sue, fire or imprison the provider for breach of contract.

Does such a Cloud exist? A Cloud that respects Service Agreements and policy? A Cloud that can be formed from internal resources? One that can be federated with Clouds in other parts of the enterprise, or even with an external Cloud such as Amazon? I’d not found one, and so we at Arjuna have designed and built one – Arjuna Agility.

So, returning to Carr’s metaphor, if you want to use the cloud then you’ll need to tame the dragons. How do you tame a dragon? My advice is to start with a small one and right from the get go, make it clear who’s the boss! Impose a set of rules regarding its general behaviour (Policy) and give it clear instructions (Service Agreements). The dragons may be stirring but if you’ve got one on your side there’s no stopping you.

‘Agility’ will manage all IT resources, software, hardware and data, as virtual resources and (nested) containers which hold those resources. It will support federated organisations (individuals, departments, enterprises or utility computing suppliers) who may each share some part of the total IT infrastructure. Key to the product is a bespoke description language which can be used to express ‘Service Requirements’ in a manner completely decoupled from the underlying IT infrastructure. The language can be transformed through a series of ’Policy-driven’ decisions, potentially across a number of organisations, into an expression (using the same language) of the concrete resources to be utilised in support of the ‘Service Requirements’. Resources will be controlled through ‘Service Agreements’, which can be monitored and modified, and which are created and managed using organisational policies. ‘Agility’ will allow business users to define their requirements in their own terms. It will then allow the IT infrastructure, steered by ‘Policy’, to make dynamic decisions as to how best to support those requirements at any one time – taking into account changing circumstances.

Crucially, unlike other approaches, ‘Agility’ does not force “big bang” changes onto the enterprise’s IT infrastructure and applications. Instead it can be introduced incrementally, as needed. This is because, ‘Agility’ does NOT require:
• new programming models to be introduced
• legacy applications to be modified
• legacy application’s clients to be modified
• existing middleware investments to be written-off
• existing IT infrastructure to be restructured or changed in any way
• infrastructure to be shared – except under conditions as specified and controlled by the owner of the resources

I see ‘Agility’ as a key product for enterprises looking to move towards ‘utility computing’.

So, whilst the ‘internal utility’ is a concept gaining ground, progress will continue to be gradual. PC applications and data will move into shared servers. Individual IT departments will coalesce into resources capable of being shared across the enterprise. Enterprises closely connected in the supply chain will share facilities, and so on. Enterprises and individuals will also take temporary advantage of third party shared facilities to handle high loads or compensate for failures in their own internal IT.

Therefore, whilst some form of unified ‘Computing Utility’ will eventually appear, once the appropriate technologies and business models have been standardised, there are many years of transition ahead. During that time enterprises will continue to experiment with, and utilise, a wide variety of technologies and business models: all of which will promise some of the benefits of utility computing. In our view the key to managing this process of change, and of being able to exploit emerging opportunities, is the deployment of ‘Agile IT Infrastructure’.

‘Agile IT Infrastructure’ is capable of responding, autonomously, to changing requirements and environments. Clearly, this means that it must be capable of changing (i.e. capable of dynamic reconfiguration). However, it also implies that it is capable of understanding what is meant by change so as to be able to make the right choices.

Nicholas Carr continues with a theory that he first outlined in his controversial 2003 article, ‘IT Doesn’t Matter’ (later expanded and published as his first book, ‘Does IT Matter?’) But it’s also a critique of the ‘Brave New World’ of the Web as heralded by the pioneers of the internet, the dotcom boomers and more lately by the Web 2.0 fan clubs.

‘Does IT Matter?’ Well, the answer reiterated in ‘The Big Switch’, is yes it does but not as much as the CEOs of the largest IT suppliers would have you believe. Carr makes the strong case that although IT is an operational requirement it doesn’t necessarily drive profits. Indeed, IT is increasingly becoming a commodity – as any competitive benefits gained by one company can be quickly imitated by rivals. These views have made him unpopular with many in the IT industry and he was once proclaimed “The Technology World’s No. 1 Public Enemy” by Newsweek (a badge he wears with considerable pride).

‘The Big Switch’ elaborates on Carr’s earlier comparison of the Electricity Generating industry, at the start of the last century, and today’s Computing industry. He explains how Edison delivered much of the technology and Insull the business model which eventually persuaded companies to abandon their reliance on internally generated power. Insull persuaded them to connect to an electricity ‘grid’ enabling them to draw power as a ‘Utility’ service. He argues that such a move, with its massive economies of scale, was an economic necessity at the time and that today’s technological advances, combined with the same economic forces, will drive us inevitably towards a ‘Computing Utility’.

Carr goes on to examine the social consequences of concentrating this ‘power’ (read ‘data’ for the Computing Utility) in the hands of the few, given the commoditisation of knowledge and even human relationships. Where ‘The Long Tail’ (by Chris Anderson) presented a glowing economic picture of a world full of almost infinite consumer choice, Carr warns of the dangers of a world full of ‘unbundled’ knowledge where every piece of information is of equal value and is therefore, ultimately, valueless.

I share Carr’s views of the future. Indeed, I believe IT is already in the process of transition. Whilst I don’t have the statistics to hand I’d confidently predict that the proportion of processor cycles initiated by an individual that execute remotely, (rather than on the individual’s local machine), is increasing very significantly year on year, and that the same is true of accessed data. Some of these cycles and data accesses are performed on behalf of applications running in the server room (possibly from thin clients), some are running on outsourced infrastructure, some are SAAS or mobile applications, others are networked applications such as Google search, ‘Second Life’ or ‘YouTube’. All represent small, but significant steps towards a Computing Utility.

Arjuna is working to help ease this transition by building support for Agile IT Infrastructure that recognises that today’s IT is increasingly distributed within and beyond the organisation. I’ll explain some of the concepts behind our work soon.

This will be the first and last entry about me – to set the scene around the content of this blog. I’ve been a blog cynic for many years now, wincing at the content of most but occasionally reaping great insight from some. The good ones are passionate, relevant and insightful, ‘Rough Type’ is a great example. I won’t offer examples of bad ones!

I hope to share some of the passion Arjuna folks have about technology through this blog. I’ll talk about what we are doing and why we are doing it. I hope to encourage some discussion around our ideas and I’m ready for criticism as well as compliments.

Right now however, I’m off to read ‘The Big Switch’. Providing infrastructure to smooth the move towards utility computing is closely related to what we are doing here in Arjuna. I’ll post my thoughts on the book soon and start sharing our ideas around Agile IT infrastructure.