link to homepage

Navigation and service

A DEEP Insight

Spring 2015: only a few months and the DEEP project will be finished. But for now, we are working in “burst” mode, fully dedicated to meet all our objectives. Find an update below on what we have achieved in the last couple of months.

Body & Mind

Frankly speaking, the last couple of months have been pretty tough. Developing and qualifying complex hardware definitely takes its toll – we probably do not have to tell you that! Problems do arise unexpectedly, and some delays are kind of unavoidable. Yet we worked hard to face and overcome all challenges and decided to go for a short project extension of three months until end of August 2015 to limit the impact to the project and achieve all objectives in full. The good news is: We are out of the woods by now and in the midst of the final project phase, assembling the DEEP prototype system. But let’s quickly rewind and see what happened since our last update:

Hardware = Hard Work Paying Off

The testing and validation work with the first small Booster system at Jülich Supercomputing Centre (JSC) has made good progress: the team has handed over half of the 32 Booster nodes to the system and application software developers, while final hardware validation is being performed on the other half. The all-important Cluster/Booster protocol is running with expected performance, and first application work is underway.

First DEEP Booster Nodes populating the Booster rackFirst DEEP Booster Nodes populating the Booster rack
Copyright: DEEP Partner Eurotech

The final Booster prototype will have 384 Booster nodes, each one with a top of the line Intel® Xeon PhiTM co-processor and an FPGA-based EXTOLL network controller. These nodes are implemented as 192 Booster Node Cards arranged in 12 chassis and connected via a custom backplane. Eurotech’s direct liquid cooling technology enables very dense packaging – the prototype will fill a full rack, and 32 Booster Interface Nodes will bridge between the EXTOLL torus network of the Booster and the Infiniband network of the 128-node DEEP Cluster.

This prototype will support all applications as foreseen. On top of that, a second Booster based on the EXTOLL ASIC implementation will be built to assess the interconnect performance gains and the benefits to application performance.
The team is confident that the scale-up will be finished by June 2015, which gives the system and application software teams sufficient time for application tuning and testing.

Efficient Job on Energy Efficiency

Simultaneously, the colleagues from the energy efficiency team installed the Energy Efficiency Evaluator (EEE) at Leibniz Supercomputing Centre – a smaller copy of the system at JSC (16 Booster nodes, 4 Cluster nodes). The idea is to have a separate system to carry out the necessary tests without interfering the fine-tuning and optimization work done by the system and application software teams towards the end of the project. Furthermore it will utilize LRZ's elaborated environmental and power sensor infrastructure for in-detail evaluation of the system's energy efficiency. The EEE is up and running now, the sensors are calibrated, and the team is excited to start testing.

Software = Running Smoothly

In the meantime, the software team has been extremely productive: The system software has been successfully implemented, and installation and fine-tuning on the Booster system is making good progress. The Cluster-Booster-Protocol – that establishes the communication between the Infiniband-based Cluster and the Extoll-based Booster – is already running on the Booster and measurements show promising performance. Last but not least, the implementation of the programming model has been completed. At the moment, the team implements mechanisms for a dynamic assignment of hardware resources to MPI processes that will enable flexible and highly efficient use of the Cluster-Booster system.

Applications: Getting Ready for The Final Prototype

Finally, the applications team is all set to work with the final DEEP prototype: The division of the applications into highly scalable and less scalable code parts has been done. Optimizing the applications for Xeon Phi in terms of threading and vectorisation has made excellent progress. Application partners have been running preliminary scalability tests on the DEEP Cluster and on the Barcelona Supercomputer Centre’s MareNostrum, JSC’s Blue Gene/Q and TACC’s Stampede systems. Applications are about to get onto the DEEP Booster and to test, for the first time, the final DEEP hardware. The results achieved on the DEEP prototype will be systematically compared with the performance the six applications are able to achieve on current HPC systems.
The to-do list until the end of August still includes quite a number of entries. But things do look promising and we are starting into the final project phase with the same motivation we felt right at the beginning of DEEP.

We’ll stay in touch,
Your DEEP Project Members