10–12 Dec 2013
DESY
Europe/Berlin timezone

Upgrade to 16 PCIexpress lanes in MTCA.4

11 Dec 2013, 11:40
8m
DESY Lecture Hall (DESY)

DESY Lecture Hall

DESY

Notkestraße 85 22607 Hamburg
New products New products

Speaker

Mr Vollrath Dirksen (N.A.T. GmbH)

Description

This presentation shows a solution to overcome the bottleneck in the data path of the main CPU (root complex), if all IO boards transfer their data via 4 lane PCIexpress DMA. By default all AMC slots offer 4 lanes of PCIexpress transfers. By extending the data path to the local CPU of the NAT-MCH-PHYS64 no modification in the backplane of the MTCA.4 crates is needed to offer a 16 lane PCIexpress path to the main processing CPU.

Summary

MicroTCA.4 systems offer 4 lanes of PCIexpress to all 12 AMC slots. At the beginning IO boards only used 1 lane of PCIexpress (2.5 GBaud) or maximum 4 lane of PCIexpress (10 GBaud). At that time the main CPU (root complex) read the data. By using FPGA in IO boards, the IO boards are able to write per DMA directly in the main memory of CPU. This offers an increase of factor 3 compared to reading data via CPU and frees up CPU performance for processing. To overcome the bootleneck, if 12 IO boards DMA their data, the CPU boards offer PCIexpress Gen3 (32 GBaud) to increase data bandwidth. Benchmarks show, that this is no enough data bandwidth. Therefore a solution offering 16 PCIexpress lanes (offering 128 GBaud) is requested.

In this presentation a solution will be shown, which can be used without any modification in standard MicroTCA.4 systems. This solution is based on the NAT-MCH-PHYS64 and the local CPU on the RTM of the MCH. Also the connection to external PCs via PCIexpress x16 cable will be presented. With this two solutions enough data bandwidth and processing power is available to process all data of high speed IO boards.

Primary author

Mr Vollrath Dirksen (N.A.T. GmbH)

Presentation materials