A new type of "cluster booster architecture" has been designed for the DEEP system.
The final DEEP prototype system will consist of a 128 node Eurotech Aurora Cluster and a 512 node Booster.
DEEP takes the concept of compute acceleration to a new level: it combines a standard, InfiniBand™ Cluster using Intel® Xeon® nodes (Cluster Nodes) with an innovative, highly scalable Booster constructed of Intel® Xeon Phi™ co-processors (Booster Nodes) and the EXTOLL high-performance 3D torus network. Both interconnects are coupled by Intel® Core™ Booster Interface Nodes. This combination provides maximum throughput and scalability on the Booster side, matching the requirements of highly scalable code parts, and supports proven HPC programming models like MPI and OmpSs. Code parts with limited scalability (e.g. because of complex control flow or data dependencies) run with high efficiency on the Cluster side, and the transparent bridging of both interconnects facilitates high-speed data transfer between the two sides.
The DEEP Cluster consists of 128 Intel® Xeon™ compute nodes with Infiniband interconnect.
The DEEP Booster consists of 512 Intel® Xeon Phi™ Co-Processors with an EXTOLL 3D Torus interconnect network.
Prototypes of all system components are now available, and integration into small, self-contained systems for system and application SW development is making good progress. The prototype Booster Nodes developed by Eurotech contain two Altera Stratix V FPGAs, a very capable board management controller, various sensors for fine-grained power and temperature measurements, and Gigabit Ethernet and SMBus endpoints for maintenance and debugging. They connect via PCI Express to two Intel® Xeon Phi™ cards. Like in the final hardware, cooling in the prototype is already provided through a custom engineered cold plate, with hot pluggable connectors to the backplane. The Booster Interface Node uses a COM Express® Module with an Intel® Core™ i7 CPU, a Mellanox ConnectX®-3 host adapter and an Altera Stratix V FPGA connected by a PCI Express bridge, and it provides SATA connectors for SSD storage. On both kinds of nodes, the EXTOLL NIC is implemented on the FPGAs.
On the network front, the EXTOLL ASIC implementation has made impressive progress, with first samples available in the SC13 timeframe. After test & validation, DEEP will proceed to design the final hardware components, replacing the FPGAs by the EXTOLL ASICs and introducing compact form factor Intel® Xeon Phi™ boards providing a full suite of sensors and actuators for advanced energy monitoring and management. From these components, the final DEEP Booster prototype will be assembled and made available for application development and optimization.