Python And Grid Computing

Seems to be buzz word that best applies to the PyWacket project.

In fact, I seem to be using the term 'Grid Computing' as a stand-in for CloudComputing. Is the Cloud just another part of the my Grid ? Is it more like a ' Federated Architecture' maybe ? How about just calling it 'Globular Computing', sounds more like it. My use of the term needs better definition.

See Servers#GridComputing, PythonMultiprocessing

These platforms seem to be long on discussion and short on actual software ...

Most seem to have peaked out in 2005 ... another example of how to turn a simple thing into a big complex thing ?

A number of Python-related libraries exist for the programming of solutions either employing multiple CPUs or multicore CPUs in a symmetric multiprocessing (SMP) or shared memory environment, or potentially huge numbers of computers [ like 3 or 4 ] in a cluster or grid environment.

Many dead links ...


Ganga is an easy-to-use frontend for job definition and management, implemented in Python. It has been developed to meet the needs of the ATLAS and LHCb for a Grid user interface ...

Some sort of CERN thing ... a bit mysterious ... seems to assume Linux.


Minimum intrusion Grid (MiG) is an attempt to design a new platform for Grid computing which is driven by a stand-alone approach to Grid, rather than integration with existing systems. The goal of the MiG project is to provide Grid infrastructure where the requirements on users and resources alike is as small as possible (minimum intrusion).

Listed as Python, but is Java ?

The Minimum intrusion Grid (commonly MiG) is an effort to design and develop a modern grid computing system that has minimal requirements for usage yet still maintains a security for all participants in the system ...

Everything within MiG must be implemented in Python unless another language is absolutely required

Interest seems to have peaked about 2005-2007 ...


Acquired by DropBox in 2013 ... still no release.

PiCloud is a cloud-computing platform that integrates into the Python Programming Language. It enables you to leverage the computing power of Amazon Web Services without having to manage, maintain, or configure virtual servers.

Cloud Mesh

A tool to simplify managing multiple clouds

Introduction to Cloud Computing with Cloudmesh -

Cloudmesh is an important component to deliver a software-defined system – encompassing virtualized and bare-metal infrastructure, networks, application, systems and platform software – with a unifying goal of providing Cloud Testbeds as a Service (CTaaS).

Cloudmesh federates a number of resources from academia and industry. This includes existing FutureGrid infrastructure (4704 cores used by more than 355 projects), Amazon Web Services, Azure, HP Cloud, Karlsruhe using various technologies.

See PythonTools#CMD3

Doesn't say so, but apparently is Linux only.

Open Stack

It is really "Cloud Computing", but classifying as "Grid Computing". In my case, the distinction may be minimal.

In any case, the OpenStack API is fully implemented in Python so ... it's geared for commerical Rack Space accounts.

Rackspace's official SDK for Python is pyrax, an Apache licensed SDK for Open Stack clouds.

Pyrax supports these Rackspace services directly

Cloud Block Storage

Cloud DNS

Cloud Databases

Cloud Files

Cloud Images

Cloud Load Balancers

Cloud Monitoring

Cloud Servers

A unified OpenStack SDK for Python is in the works.

Using python-novaclient with the Rackspace Cloud

Python-novaclient is a command-line client for controlling Open Stack instances. Since the next generation of Cloud Servers is based on Open Stack, python-novaclient will work with Cloud Servers as well.

The client provides a bit more functionality than the control panel without needing to delve into the API directly.

We have installation instructions for Windows, Mac OS X, and Linux

Python 2.7 only

pyrax should work with most OpenStack-based cloud deployments,though it specifically targets the Rackspace public cloud.

For example,the code for cloudfiles contains the ability to publish your content on Rackspace's CDN network, even though CDN support is not part ofOpenStack Swift. But if you don't use any of the CDN-related code, your app will work fine on any standard Swift deployment.

To sign up for a Rackspace Cloud account, go to:

Managed Infrastructure - $.005/GB/hr service level rate - $50 minimum

Also See

Topic :: System :: Distributed Computing -



Last modified 2 years ago Last modified on 12/24/2015 03:08:40 PM