Christoph Raisch

Overview of IBM zEnterprise 196 I/O subsystem with focus on new PCI express infrastructure

IBM zEnterprise® 196 introduces a new input/output (I/O) s5ubsystem, including a new I/O drawer that is largely based on a greatly expanded exploitation of industry-standard high-volume PCI Express® (PCIe®) links and switches. The System z® qualities of reliability, availability, and serviceability (RAS) are preserved and enhanced by combining the PCIe RAS capabilities with new System z capabilities. PCIe ports connecting the processor book to the I/O drawer are provided by a new IBM-designed PCIe fan-out card. This fan-out card and its firmware (Licensed Internal Code) support both traditional System z I/O and new I/O paradigms. In the new PCIe I/O drawer, PCIe switches provide fan-out and the well-established System z I/O failover function referred to as redundant I/O interconnect. This is the third generation of the I/O drawer/cage to be used in System z platforms. The PCIe I/O drawer design is extremely compact and provides enhanced I/O port granularity and density. It has been designed to provide performance extendibility for future I/O advancements. Traditional I/O such as FICONA, Fibre Channel Protocol, and Ethernet are provided with enhanced functionality and are packaged in this new PCIe I/O drawer. The advent of this new infrastructure opens up the possibility of attaching native PCIe adapters while allowing them to be controlled by system firmware or by the operating systems directly.

Improving Communication Performance on InfiniBand by Using Efficient Data Placement Strategies

Despite using high-speed network interconnection systems like InfiniBand, the communication overhead for parallel applications is still high. In this paper we show, how such costs can be reduced by choosing appropriate data placement strategies. For large buffers, we propose a transparent placing in hugepages as it can dramatically decrease memory registration overhead and may increase network bandwidth. Thus, we developed a new library that can be preloaded for applications at load time and cares about drawbacks of using hugepages. So we believe that it is the most suitable one in the HPC area for Linux today. But we do not only refer to large buffers as small communication buffers also play a significant role for application behaviour. We show that transfer latencies vary, depending on data placement. All current communication library implementations for InfiniBand do not utilize scatter-gather lists for send and receive operations, but we show that this feature can have a positive impact on latency for small buffers and data aggregation can perform better. Our results show that communication performance of applications may improve more than 10 % using the presented improvements