Release Notes v7.3.0

The BeeGFS 7.3.0 release includes support for NVIDIA Magnum IO GPUDirect Storage (GDS), multi-rail RDMA networking on the client, Linux 5.10 and the Arm architecture and other smaller improvements.

General Changes/Improvements

  • Added support for NVIDIA Magnum IO GPUDirect Storage (GDS). More information about the technology can be found in this blog post. For a guide on how to enable the new feature on the client and how to set up the system to benefit from GDS please have a look at the GDS documentation. GDS will become an Enterprise Feature in the future, but can be used with the Community Edition until further notice. We are looking forward to receiving your feedback on functionality and performance.

  • Added support for multi-rail RDMA networking to the client. Please refer to the RDMA documentation page for more information on the setup.

  • Added support for Linux 5.10 and the Arm architecture by redisigning the client I/O path to avoid the now defunct get_fs() and set_fs() kernel APIs.

  • Added a new direct I/O mode to StorageBench. Direct I/O on the storage servers can be enabled with the --odirect flag. This is useful in situations where the page cache on the storage servers becomes a performance bottleneck, which has been observed on systems that use multiple very fast NVMe drives.

Known Issues and Limitations

  • The Metadata daemon does not work reliably on RHEL/CentOS 8 and SLES 15.1 and 15.2 due to a problem in the versions of glibc. The problem was fixed in RHEL/CentOS 8.1 and SLES 15.3.

  • The client module might hang if the IBV device(s) used are taken offline during operation.

  • The client module does not compile on SLES 15.2 with Mellanox OFED 5.2

  • Attempting to mount the filesystem with RDMA over RoCE will crash the kernel on RHEL 8.5 (and Rocky, Alma 8.5). RDMA over IB with and without Mellanox OFED works fine.

Supported Linux Distributions and Kernels

Packages are provided for the x86_64 architecture and the following distributions:

  • RHEL 8 (packages can also be used on RockyLinux and Alma Linux)

  • SLES 15

  • Debian 10 and 11

  • Ubuntu 18.04 and 20.04

Packages for the Arm64 architecture will be published via our repositories within the next days.

The following Mellanox OFED driver versions are supported: 5.5, 5.4, 5.3, 5.2, 5.1, 5.0, 4.9

The full integration test suite was run on Rocky 8.5, RHEL 8.4, Rocky 8.4, Alma 8.4, CentOS 8.3, OpenSUSE 15 SP3 and Debian 10 and 11.

Client build testing:

  • RHEL 8.3: no OFED, OFED 4.9, 5.0, 5.1, 5.2, 5.3, 5.4

  • AlmaLinux 8.4: no OFED, OFED 5.3, 5.4

  • AlmaLinux 8.5: no OFED, OFED 5.3, 5.4, 5.5

  • Rocky Linux 8.4: no OFED, OFED 5.3, 5.4

  • Rocky Linux 8.5: no OFED, OFED 5.5

  • SLES 15.1: no OFED, OFED 5.0

  • SLES 15.2: no OFED, OFED 5.1, 5.4

  • SLES 15.3: no OFED, OFED 5.4, 5,5

  • Debian 10: no OFED, OFED 5.2, 5.3, 5.4

  • Ubuntu 18.04: no OFED

  • Ubuntu 20.04: no OFED, OFED 5.4

Version Interoperability

BeeGFS v7.3.0 is not compatible with older versions of BeeGFS. Upgrades from older versions of BeeGFS require a shutdown of the filesystem.

Upgrading from Older Versions

To upgrade from an older version, please refer to the Upgrade Guide.