Microsoft has been touting its “software plus services” strategy for some time, but if you want see some of the most creative thinking about how to meld cloud services with traditional PC software you’d do well to look not at Microsoft but at Mathematica. Wolfram Research, which makes Mathemetica, a heavy-duty and widely used program for computation and modeling, announced last week that it will build “the cloud” into the latest version of the application, delivering utility computing as, in essence, a seamless feature of its software.
Wolfram is working with two partners, Nimbis Services and R Systems, who specialize in supercomputing – or “high performance computing” (HPC) as it’s often called today – to “enable the Mathematica cloud service to access many diverse HPC systems, including TOP500 supercomputers and the Amazon Elastic Compute Cloud [EC2].” The “service will provide flexible and scalable access to HPC from within Mathematica, simplifying the transition from desktop technical computing to HPC,” the company says.
A representative from Amazon Web Services explains how Mathematica will tap into EC2 and the benefits it can provide to users:
Mathematica is a true cloud service offering. They connect to Amazon’s Cloud from within Mathematica. So you can simply use all the powerful features of Mathematica and ask it to run it in the cloud. For example, you don’t need to buy a Digital Image Processing package to do image processing in-the-Cloud. It’s all bundled in.
The workflow is very simple to understand and it takes very few clicks to deploy your code in the cloud. A typical Mathematica user develops code in their standard notebook interface, a programming concept that defines their input code and output results, including graphics. The user specifies input cells, output cells and other parameters. Mathematica will evaluate one input cell at a time so evaluation could take a lot of time to process on one machine. Now, with the new Cloud service, users can evaluate the entire notebook in one shot by pushing it to the cloud.
The HPC Cloud Service lets users take the entire notebook, click a few buttons in the HPC Cloud Service GUI and ask it to run it in the cloud. The HPC Cloud Service evaluates the code, runs it in parallel Mathematica sessions, bundles up the results and notifies the user. In other words, a user can test the code (a Mathematica Notebook) with a small amount of input and then increase size of the input to a more realistic size, push it to the cloud so it runs on hundreds and even thousands of nodes in parallel, and get notified when its done.
We certainly need a nifty abbreviation for the merging of cloud services into traditional PC software, so let me suggest CaaF, for Cloud-as-a-Feature. Wolfram is by no means the first company to roll out a CaaF offering – Apple’s MobileMe incorporates lots of cloud features, and even Google Earth is kind of CaaFy – but Mathematica’s example shows most clearly how the cloud can be used to build powerful new features into existing programs. It’s a model that is sure to expand as programmers think more creatively about the cloud. “Imagine,” says the Amazon rep, “if you could do this with any software and simply click the ‘Run it in-the-Cloud’ button, run everything in parallel and get your results faster.”
I recently suggested that the cloud is most interesting as a means of doing new things with computers, rather than just doing old things in a new environment. By radically changing the economics of high-performance computing, the cloud democratizes the supercomputer. What Wolfram is doing points to one way that software companies can exploit those new economics.
I’m sure this is of value to many Mathematica users. However I do not subscribe to the view that high performance computing is an underutilised source of competitive differentiation. Outside of a few niche areas it’s essentially irrelevant to most businesses.
Well, Greg, you never know. Change the economics, and you tend to change the uses. But I see your point. The broadest immediate benefits of CaaF probably lie more in facilitating collaboration than in tapping into a supercomputing grid.
As a key member of the team that put together the Mathematica for the Cloud initiative, I am very interested in your perspective. We would be interested in hearing more from techical developers and particularly native application desktop users regarding how they plan on using Cloud computing. Please send mail to me at joy_costa@wolfram.com. By the way, CaaF isn’t a bad idea!
It’s interesting to contrast this with other grid computing schemes like SETI@Home, which use peers on the network. Has Mathematica ever done that?
Bandwidth constraints and software complications also limit the applications of this approach. I use After Effects and 3D tools, and it’d be nice to speed up renders this way, but transferring frames of film/video across the internet isn’t practical (and even LANs can limit the effectiveness of local grids). And these tools use software plug-ins and fonts that don’t necessarily port easily to a cloud/grid environment. (Are plug-ins for Mathematica written in Java, or strictly in Mathematica’s own language?)
I can see value in this offering. I’m not a Mathematica user but I do use SAS, R and WPS for analytics. When you have a desktop machine that only has two to four GB’s of RAM and you have a large model to solve, the cloud offers the perfect solution. Sure beats the heck out of buying another PC or spending $100 to upgrade your RAM just for a few models.
So, here’s where that goes:
Right now, Mathematica has a “client” part that runs locally and an “expandable, cloudable” optional part that runs on the web and in computing utilities.
Later, the “client” part will also be able to run on the web and in computing utilities.
Then or later, the former-“client” part will be robust against partitioning and merging. I.e., you can copy an image of the client part on your local PC *or* run it on the net *or* do both at once with the two copies syncing up *or* have the two copies be unable to sync up for a while and then be able to sync them up (like your “contacts list” vs. your “devices” and web services and multiple desktops).
It’s just a kind of “process virtualization” and “process migration” and it’s pretty well understood (though not extensively practiced) how to make it all work, in theory. One point of “how to make it work” is to use higher-level, more lisp-ish programming languages as the main architectural theme — which helps to explain in retrospect why Mathematica is an early entrant example of a programming becoming virtual and migratable this way. (“Just so!” :-)
Think how sweet it will be for other kinds of program.
For example, consider a huge, multi-user double entry book-keeping system — a kind of “CFO Wiki”. You could put part or all of the accumulated financial models on a detached laptop and have a stand-alone snapshot to play with. You could also let the same models run on the cloud and have an organization-wide real-time set of “tills” to view. No special hardware purchase required and, if the software if free software — nothing but maintenance costs for your deployment.
Or word processing and layout: people currently use “phased” approaches (first word process, then do the page design). That’s because a tiny little change in the word processing part can imply a huge change on the pages — and that huge change takes a lot of cycles to compute. But a lot of parallelization is possible for the page setting — it could fit swell on a cloud cluster and if such utilities are cheap….
But, again: Wolfram’s out front here because of their systems programming research. They’re weirdos using a very disciplined architectural approach in their software emphasizing a thoughtful form of high level language programming — the word processor, page setting software, and double entry bookkeeping system need to be rewritten in a similar style before they can catch up.
-t
It’s definitely a step in the right direction. Up until now, most uses of EC2 look a lot like the old mainframe model using things like Job Control Language: set up data sets, upload program, schedule the job or etc.. One would hope that ultimately the migration to cloud computing will have a ubiquitous kind of scripting layer that hides and encapsulates the distribution of that it is much like data processing on the console of a machine. How about EC2 shell?
Keynote Systems is a mobile and internet apps testing company, and is offering a “desktop-software-to-cloud” testing product. It’s downloadable from http://kite.keynote.com (internet) and http://mite.keynote.com. Both these products have CaaF – you can run the tests from your desktop PC, but then you can push the tests to a global network of PCs or mobile phones, and execute the tests from Hong Kong, London or San Francisco, or from mobile phones connected to aircards.
Tim, Great post! We will definitely see a lot more “nifty abbreviations”, it’s inherent in this business.
How about “CaC” enabled applications for Cloud-assisted-Computing? Developers would provide CaC enabler plug-ins allowing end users to send intensive compute workloads into the cloud for processing. An end user would simply click the CaC button, then, encrypted meta data, data and process commands would be sent to the cloud for processing. Results would be returned in minutes rather than hours. There could also be an optimizer built into the app giving the end user the ability to compare runtimes locally versus in the cloud with estimated pricing to aid in the decision process. Let’s call that the “COPP” button, for the Cloud-Optimizer-for-Performance-and-Pricing button.
Sorry, I meant Nick!
@ Linuxguru1968
EC2 does have a shell, if you are firing up a Linux instance. Do you mean some other further abstracted shell? I can see that, but I’m not sure it is a huge boost over what you can get already.
BJ Upton:
I was kind of proposing a shell type language that would sit at a higher level and allow for control of multiple VMs – not just the shells that come with the VM OSs- perhaps something with IPC/semiphores that would allow applications running on differnt VMs to communicate and co-ordinate data exchange. Imagine piping the output of one program in one VM into the input of another on a differnet VM. But, you are right that might not be a big boost over just running it all in a single VM.
I see the second wave of the cloud computing. I have a detailed post on my blog at:
http://cloudcomputing.blogspot.com/2008/12/de-coupled-cloud-runtime-and-demand.html