Archive for the ‘Cloud Computing’ Category

Cloud Federation

November 17, 2008

We’ve been talking to a lot of people about federation lately.

Last week, Professor Paul Watson presented federation on Arjuna’s behalf at the London CloudCamp to good reviews. Phil Wainewright picked out federation as a core them of the conference (read his blog). I’m currently in the Bay Area talking to a number of companies about this subject. It’s a hot topic.

Federation is seen as an important aspect by a number of companies including Cisco – Doug Gourlay recently outlined Cisco’s thinking around cloud and Phil Dean presented at London CloudCamp (again, see Phil Wainewright’s blog) – it was also covered at the Web 2.0 Summit although sometimes under the term ‘hybrid cloud’, a cloud that somehow spans private IT and the public clouds.

Personally, I don’t like the term ‘hybrid’ – it suggests a solution that is neither one thing nor the other. We prefer to talk about the ‘federation’ of clouds. It’s our belief that enterprises won’t present the public cloud to their internal users in its raw state, but that they’ll access the cloud-hosted services via a proxy/gateway which will represent the cloud to the users, and the enterprise to the cloud. By this means the enterprise will control access and manage the costs. This internal representation of the cloud looks to its users like a cloud – it’s opaque, and it behaves like a cloud. In fact it is a cloud – a ‘private cloud’, albeit one which doesn’t directly manage any resources itself, but a cloud nevertheless. If such a private cloud also had the means to manage internal resources and to make dynamic decisions as to whether service would be delivered by those resources or by the public cloud then we can begin to see the means by which internal IT infrastructure could over time be increasingly subsumed into the cloud.

Our Cloud computing product, Agility, is focused on delivering precisely this functionality. Agility instances can simply broker between the enterprise and one or more public clouds, or can be assigned a set of IT resources, offering users service delivered by one or the other set of resources, or by some combination of both. By this means Agility acts as an on-ramp to the cloud and allows the IT department to begin to experiment with cloud computing in a gradual, incremental way, without any need for disruption to existing service.

Taming the Dragon

June 9, 2008

Nicholas Carr writes of the dangers of Cloud Computing. ‘Beyond Here There Be Dragons’ he warns, pointing at the empty space on the map marked THE cloud. And he’s right. Bill Thompson notes that ‘in the real world national borders, commercial rivalries and political imperatives all come into play’. Who is going to send their applications, and more importantly their data, into this uncharted territory?


The answer of course is that no-one will. People won’t commit to THE cloud – but they would commit to A cloud. They’d commit to a cloud that they could trust, and in the absence of implicit trust, to one they could hold accountable. They would, for example, let their applications execute in their own department’s cloud – one that managed the departmental resources and provided its users with services and supported their service requirements, without the users needing to concern themselves with the IT mechanics involved. They’d do so if they could have clearly defined Service Agreements with that local cloud and insight into the Policy operated by that cloud.


At some point the local cloud might be connected to another department’s cloud within the same enterprise (so that resources can be shared) and this would require the creation of Service Agreements between those departments. If this happened the users wouldn’t care – or rather they wouldn’t care so long as their Service Agreements and the local Policy (the embodiment of the things they DO care about) were maintained. And when the enterprise connected to other enterprises or to some external cloud they still wouldn’t care. If concerns were present, or if new concerns arose, then the cloud user would need to:

  • be sure that their Service Agreement (or the Policies enforced by their department, enterprise or government) clearly specified what was, and what was not, acceptable and
  • place sufficient trust in their local cloud provider – sufficient that is to believe they could sue, fire or imprison the provider for breach of contract.

Does such a Cloud exist? A Cloud that respects Service Agreements and policy? A Cloud that can be formed from internal resources? One that can be federated with Clouds in other parts of the enterprise, or even with an external Cloud such as Amazon? I’d not found one, and so we at Arjuna have designed and built one – Arjuna Agility.


So, returning to Carr’s metaphor, if you want to use the cloud then you’ll need to tame the dragons. How do you tame a dragon? My advice is to start with a small one and right from the get go, make it clear who’s the boss! Impose a set of rules regarding its general behaviour (Policy) and give it clear instructions (Service Agreements). The dragons may be stirring but if you’ve got one on your side there’s no stopping you.

Arjuna’s Agility Infrastructure

February 25, 2008

Since inception as a research group towards the end of the 1980’s and throughout our commercial existence, the Arjuna team has been building support for reliable, large-scale distributed systems (with much of our activity centered on ‘transactioning’ i.e. coordinating distributed activities). Those of you who already know of Arjuna may have been wondering what we’ve been doing since selling the Arjuna JTS to JBoss 18 months ago. Now, I can announce that Arjuna is building a prototype for a future product, to be known as ‘Agility’, capable of supporting agile IT infrastructures.

‘Agility’ will manage all IT resources, software, hardware and data, as virtual resources and (nested) containers which hold those resources. It will support federated organisations (individuals, departments, enterprises or utility computing suppliers) who may each share some part of the total IT infrastructure. Key to the product is a bespoke description language which can be used to express ‘Service Requirements’ in a manner completely decoupled from the underlying IT infrastructure. The language can be transformed through a series of ’Policy-driven’ decisions, potentially across a number of organisations, into an expression (using the same language) of the concrete resources to be utilised in support of the ‘Service Requirements’. Resources will be controlled through ‘Service Agreements’, which can be monitored and modified, and which are created and managed using organisational policies. ‘Agility’ will allow business users to define their requirements in their own terms. It will then allow the IT infrastructure, steered by ‘Policy’, to make dynamic decisions as to how best to support those requirements at any one time – taking into account changing circumstances.

Crucially, unlike other approaches, ‘Agility’ does not force “big bang” changes onto the enterprise’s IT infrastructure and applications. Instead it can be introduced incrementally, as needed. This is because, ‘Agility’ does NOT require:
• new programming models to be introduced
• legacy applications to be modified
• legacy application’s clients to be modified
• existing middleware investments to be written-off
• existing IT infrastructure to be restructured or changed in any way
• infrastructure to be shared – except under conditions as specified and controlled by the owner of the resources

I see ‘Agility’ as a key product for enterprises looking to move towards ‘utility computing’.

‘Internal Utility’

February 14, 2008

At Arjuna, we’ve been believers in the inevitability of ‘Utility Computing’ for many years. I share Nicholas Carr’s views that the case for moving much of an enterprise’s hardware infrastructure, and a considerable amount of its software infrastructure, out of siloed IT Departments and into shared facilities, is economically compelling. Initially, this will occur within the enterprise, or shared between enterprises (which we refer to as ‘Internal Utility’), but ultimately the move will be into an external Utility. The savings to be made by sharing computing load (rather than having to over-provision internally for peak load) are just too significant to be ignored, given that current IT utilisation levels are commonly lower than 25%! Indeed William Fellows (451 Group) wrote late in 2006 that ‘shared, internal utility’ was “today’s hotspot”.

So, whilst the ‘internal utility’ is a concept gaining ground, progress will continue to be gradual. PC applications and data will move into shared servers. Individual IT departments will coalesce into resources capable of being shared across the enterprise. Enterprises closely connected in the supply chain will share facilities, and so on. Enterprises and individuals will also take temporary advantage of third party shared facilities to handle high loads or compensate for failures in their own internal IT.

Therefore, whilst some form of unified ‘Computing Utility’ will eventually appear, once the appropriate technologies and business models have been standardised, there are many years of transition ahead. During that time enterprises will continue to experiment with, and utilise, a wide variety of technologies and business models: all of which will promise some of the benefits of utility computing. In our view the key to managing this process of change, and of being able to exploit emerging opportunities, is the deployment of ‘Agile IT Infrastructure’.

‘Agile IT Infrastructure’ is capable of responding, autonomously, to changing requirements and environments. Clearly, this means that it must be capable of changing (i.e. capable of dynamic reconfiguration). However, it also implies that it is capable of understanding what is meant by change so as to be able to make the right choices.

‘The Big Switch’ Rewiring the World, from Edison to Google

January 30, 2008

So I enjoyed reading ‘The Big Switch’ – well written, sobering, and intelligent is my summary.

Nicholas Carr continues with a theory that he first outlined in his controversial 2003 article, ‘IT Doesn’t Matter’ (later expanded and published as his first book, ‘Does IT Matter?’) But it’s also a critique of the ‘Brave New World’ of the Web as heralded by the pioneers of the internet, the dotcom boomers and more lately by the Web 2.0 fan clubs.

‘Does IT Matter?’ Well, the answer reiterated in ‘The Big Switch’, is yes it does but not as much as the CEOs of the largest IT suppliers would have you believe. Carr makes the strong case that although IT is an operational requirement it doesn’t necessarily drive profits. Indeed, IT is increasingly becoming a commodity – as any competitive benefits gained by one company can be quickly imitated by rivals. These views have made him unpopular with many in the IT industry and he was once proclaimed “The Technology World’s No. 1 Public Enemy” by Newsweek (a badge he wears with considerable pride).

‘The Big Switch’ elaborates on Carr’s earlier comparison of the Electricity Generating industry, at the start of the last century, and today’s Computing industry. He explains how Edison delivered much of the technology and Insull the business model which eventually persuaded companies to abandon their reliance on internally generated power. Insull persuaded them to connect to an electricity ‘grid’ enabling them to draw power as a ‘Utility’ service. He argues that such a move, with its massive economies of scale, was an economic necessity at the time and that today’s technological advances, combined with the same economic forces, will drive us inevitably towards a ‘Computing Utility’.

Carr goes on to examine the social consequences of concentrating this ‘power’ (read ‘data’ for the Computing Utility) in the hands of the few, given the commoditisation of knowledge and even human relationships. Where ‘The Long Tail’ (by Chris Anderson) presented a glowing economic picture of a world full of almost infinite consumer choice, Carr warns of the dangers of a world full of ‘unbundled’ knowledge where every piece of information is of equal value and is therefore, ultimately, valueless.

I share Carr’s views of the future. Indeed, I believe IT is already in the process of transition. Whilst I don’t have the statistics to hand I’d confidently predict that the proportion of processor cycles initiated by an individual that execute remotely, (rather than on the individual’s local machine), is increasing very significantly year on year, and that the same is true of accessed data. Some of these cycles and data accesses are performed on behalf of applications running in the server room (possibly from thin clients), some are running on outsourced infrastructure, some are SAAS or mobile applications, others are networked applications such as Google search, ‘Second Life’ or ‘YouTube’. All represent small, but significant steps towards a Computing Utility.

Arjuna is working to help ease this transition by building support for Agile IT Infrastructure that recognises that today’s IT is increasingly distributed within and beyond the organisation. I’ll explain some of the concepts behind our work soon.