Performing intricate calculations using high-performance computing

In September of 2008, a cluster of Ontario computers known as SHARCNET broke into the world's news consciousness when, after 16 days of relentless calculating and recalculating, it verified that yes, 243,112,609-1 was the largest number human beings had ever found that could be divided only by 1 and itself; that is, a prime number.

To give you a metaphorical idea of how truly massive this number was, Jeff Gilchrist, the Carleton University doctoral student who ran the calculation through SHARCNET, computed that if you printed all 12.9 million digits of 243,112,609-1 in a book, that book would have to be 3,200 pages long. If stretched out in 12-point type, the number would have been 50 kilometres long.

While readers of the planet's blogosphere "oohed" and "wowed" and marvelled over the number and its implications for computer encryption, what wasn't making the headlines were some things almost as amazing that SHARCNET had done without any publicity. When the computers spread over 17 institutions weren't verifying prime numbers, they were helping Matt Davison at the University of Western Ontario develop financial computer models that may tell investors the best time to buy and sell their stocks.

They were assisting Vesselin Stoilov of the University of Windsor in his quest for the perfect frictionless surface. This surface would allow car manufacturers, among others, to do away completely with the need for lubricants. It's a search that required simulations of surfaces from the atomic level up. So without SHARCNET's number-crunching power, "We might as well be looking in the dark," says Stoilov.

And let's not forget the work of Chris Bauch, a mathematics professor at the University of Guelph who has been evaluating the cost-effectiveness of giving every child in the country a hepatitis A vaccination.

But perhaps the biggest contribution that SHARCNET - the acronym stands for Shared Hierarchical Academic Research Computing Network - has made to the world is the vision that gave it birth. It is a made-in-Ontario, perfected-in-Canada, adopted-worldwide model for how university computer-based research can be organized to be most effective.

Let's go back to a moment in 1999 when five Ontario universities decided that they wanted to ramp up their computer research capacities. At that time, they found themselves scraping up against the prevailing organizational paradigm.

The buzzword of the day was "supercomputing centres." This was a model in which a massive amount of computing power was placed in a central location, in what Michael Bauer, the Western computer science professor who is now associate director of SHARCNET, calls "a great big barn." At that time, scientists arrived at the supercomputing centres to work on their problems or sent their data there to be processed.

What Guelph, Western, Windsor, Laurier and McMaster scientists decided they wanted instead was a place that was in a sense everywhere and nowhere at the same time.

ORION (Ontario Research and Innovation Optical Network) had been established a few years earlier to provide the high speed connectivity, but it was SHARCNET that was going to provide the real computer power.

"Rather than building a single large facility, we chose to build facilities at multiple institutions and connect them together through a high-speed network," remarks Bauer. "We were 'The Cloud' before 'The Cloud' existed."

The Cloud he is referring to is a 21st-century vision of the future of computing that sees all computers hooked together not through individual servers, but through a kind of Internet-linked cloud of anonymous machine locations.

In retrospect, SHARCNET's version of The Cloud seems to be a totally logical linkage between the emerging high-speed Internet and the relentlessly faster, more powerful and smaller modern computer. But at the time, there was much skepticism about the plan, primarily because there was no existing model for SHARCNET to imitate.

"We had an expert review panel, numbers of whom worked at supercomputing centres, and they asked, 'Why don't you just put all the equipment in one place? Why are you doing this?'" says Bauer.

The psychology of the user was one answer.

"You need to have a local presence" is how Cindy Munro, SHARCNET's director of finance and administration, bluntly puts it.

The local presence was designed to give researchers a sense that SHARCNET belonged to them in a way that the supercomputing centres never did. Turning supercomputing into something that was "ours and not theirs" was particularly important, says Bauer, when it came to convincing non-traditional users - such as researchers working outside of the data-intensive areas of physics and engineering - that they too should reconfigure their research agendas to presume they could deconstruct data mountains.

Not to mention that if scientists bought into the notion of a cloud-like network, they had a real inducement to train their students to feel comfortable using the high-performance computing grid's almost effortless-to-access capacities.

Davison's stock-buying conundrum is a classic example of how this all works.

Trying to figure the best time to buy one of 10 specified stocks over the period of a year is enormously complex and "ends up looking a bit like the problem of heat flow on a 10-dimensional object," he says. "Although the math can be written down in a concise way, it is impossible to find an analytic expression for the problem's solution even when just one asset is in the mix."

So Davison and his students have turned to SHARCNET to simulate a number of market conditions and conduct what you might call a multiplicity of stock-buying experiments.

It was also necessary for the founding universities to understand that while they might have to compete to have access to various pieces of the cloud-like supercomputing system, they also had to buy into the notion of collaboration. "We've developed a new word to describe this," says Munro. "We call it 'coop-petition' to signify that you both cooperate and compete together."

Coop-petition might also describe how the computers themselves are configured. They are not the same machines simply installed in different places, but rather, something else. "We have different people with different needs and we can put together different packages of computers for them to access," says Bauman. "As we have grown, what we have come to realize is that you need differences almost as much as sameness."

In a larger sense, the SHARCNET cloud-like model has become a prototype for other collaborative efforts. The West- Grid Consortium, which links four Western provinces, and the Atlantic Computational Excellence Network (ACEnet), which links together seven universities in the Atlantic provinces, are examples.

Perhaps nothing says more about the integration of high-speed computing with the practice of modern science than the fact that there are two other high-performance computing consortia in Ontario that have thrived with a different organizational model. You might describe them as "The Node" rather than "The Cloud" because they are massive, centrally located centres participants can access over the Internet.

The University of Toronto's SciNet consortium, which services U of T and its associated research hospitals, is such a node. SciNet has just announced that, in conjunction with IBM, it will build Canada's most powerful supercomputer - one that can do jaw-dropping a 360 trillion calculations per second. What could you do with that, you might wonder. How about sift through the Alps of particle data coming from the brand new Large Hadron Collider based in Geneva, searching for a sign of the as yet undiscovered particle that theory says should give all things in the universe mass.

In a similar node vein, Carleton, the University of Ottawa, Queen's, the Royal Military College, Ryerson University, Seneca and Loyalist College are linked via the High Performance Computing Virtual Laboratory, based at Queen's.

Doug Mewhort, a professor of psychology at Queen's, is working with graduate student Mike Jones on developing a computer program that can appreciate linguistic variance with such refinement that Mewhort says, "It can tell the difference between a good writer and a bad writer." Even better for a Google universe, the program may be able to accurately parse the context in which a word has been used and thus narrow the often impossibly large results of a keyword search.

Which approach is better, The Cloud or The Node, you might well ask.

"Our applications are memory hogs, but don't need that many processors," says Mewhort. "If I tried to do our analysis on SHARCNET machines I might fail because I can't get enough memory. Other applications, like many used in physics, require more processors than memory. So they need to farm out applications to many slave machines like SHARCNET uses."

And thus, maybe the ultimate Ontario contribution to how high-performance university research computing should be organized expresses itself as a mantra of contradiction.

Success means you are ever flexible in your approach and, accordingly, how you best configure your computers should be written as - with emphasis - it depends.