btrzx2 (2016,deprecated)

General remarks

This cluster will be be decommissioned on 10.02.2023.

The cluster btrzx2 went into operation in December 2016. Besides two login nodes, it features five different node types with the properties compute8, compute20, compute40, phi, and cuda which are connected by an Intel Omni-Path network. The compute8 and compute20 nodes are equipped with two 4-core or 10-core Intel Xeon CPUs, respectively, while the compute40 nodes contain four of the 10-core CPUs. The phi and cuda nodes, on the other hand, are based on the compute20 nodes but have an additional Intel Xeon phi (phi) or Nvidia Tesla GPU (cuda) attached. As a peculiarity of btrzx2, hyperthreading is enabled which doubles the logical cores per node. Consequently, to request complete compute20 (compute8) nodes, one must use ppn=40 instead of ppn=20 (ppn=16 instead of ppn=8). However, in view of the performance of parallel programs, it can make sense to use physical cores only, e.g., by dropping every second entry in the host file (see hyperthreading).

Login nodes

btrzx2-1.rz.uni-bayreuth.de
btrzx2-2.rz.uni-bayreuth.de

Compute nodes (state June 2018)

112 nodes (compute20)
2x Intel Xeon E5-2630 v4 @ 2.20GHz (Broadwell) with 2x 32GB RAM
(in total 20 physical cores, hyperthreading enabled)
28 nodes (compute8)
2x Intel Xeon E5-2623 v4 @ 2.60GHz (Broadwell) with 2x 32GB RAM
(in total 8 physical cores, hyperthreading enabled)
1 node (phi)
2x Intel Xeon E5-2630 v4 @ 2.20GHz (Broadwell) with 2x 32GB RAM
(in total 20 physical cores, hyperthreading enabled) and
1x Intel Xeon Phi 7120P @ 1.24 GHz (16GB, 61 Kerne)
1 node (cuda)
2x Intel Xeon E5-2630 v4 @ 2.20GHz (Broadwell) with 2x 32GB RAM
(in total 20 physical cores, hyperthreading enabled) and
1x Nvidia Tesla K40M (12GB)
2 nodes (compute40)
4x Intel Xeon E5-4620 v4 @ 2.10GHz (Broadwell) with 4x 128 GB RAM
(in total 40 physical Kerne, hyperthreading enabled)

Queues

route (default queue)
Routes jobs to the default or long queue depending on the requested wall time
default
Default job priority
Wall time limit: 24 hours
Restrictions: no
long
Default job priority
Wall time limit: 100 hours
Restrictions: privileged groups
dev
High job priority
Wall time limit: 2 hours
Max. nodes per job: 3
Max. running jobs per user: 3
Max. occupied nodes by queue: 32

Network

Intel Omni-Path
2-level Fat Tree (Blocking factor 2)

User file space (network and local)

ITS home directories (/home)
Panasas file system (/data & /scratch)
Local disk (/tmp, 884 GByte)

Commissioning & Extension

Basic equipment (Dec. 2016): 78x compute20, 28x compute8, 1x phi, 1x cuda, 2x compute40
Stepwise extension: 34x compute20
HPC extension btrzx4 (Apr. 2018)

Resource Manager & Scheduler

PBS Torque
Maui

Operating system

CentOS 7.9

Node topology (likwid-topology -g)

compute20: 2x Intel Xeon E5-2630 v4 Einklappen

--------------------------------------------------------------------------------
CPU name:	Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
CPU type:	Intel Xeon Broadwell EN/EP/EX processor
CPU stepping:	1
********************************************************************************
Hardware Thread Topology
********************************************************************************
Sockets:		2
Cores per socket:	10
Threads per core:	2
--------------------------------------------------------------------------------
HWThread	Thread		Core		Socket		Available
0		0		0		0		*
1		0		1		0		*
2		0		2		0		*
3		0		3		0		*
4		0		4		0		*
5		0		5		0		*
6		0		6		0		*
7		0		7		0		*
8		0		8		0		*
9		0		9		0		*
10		0		10		1		*
11		0		11		1		*
12		0		12		1		*
13		0		13		1		*
14		0		14		1		*
15		0		15		1		*
16		0		16		1		*
17		0		17		1		*
18		0		18		1		*
19		0		19		1		*
20		1		0		0		*
21		1		1		0		*
22		1		2		0		*
23		1		3		0		*
24		1		4		0		*
25		1		5		0		*
26		1		6		0		*
27		1		7		0		*
28		1		8		0		*
29		1		9		0		*
30		1		10		1		*
31		1		11		1		*
32		1		12		1		*
33		1		13		1		*
34		1		14		1		*
35		1		15		1		*
36		1		16		1		*
37		1		17		1		*
38		1		18		1		*
39		1		19		1		*
--------------------------------------------------------------------------------
Socket 0:		( 0 20 1 21 2 22 3 23 4 24 5 25 6 26 7 27 8 28 9 29 )
Socket 1:		( 10 30 11 31 12 32 13 33 14 34 15 35 16 36 17 37 18 38 19 39 )
--------------------------------------------------------------------------------
********************************************************************************
Cache Topology
********************************************************************************
Level:			1
Size:			32 kB
Cache groups:		( 0 20 ) ( 1 21 ) ( 2 22 ) ( 3 23 ) ( 4 24 ) ( 5 25 ) ( 6 26 ) ( 7 27 ) ( 8 28 ) ( 9 29 ) ( 10 30 ) ( 11 31 ) ( 12 32 ) ( 13 33 ) ( 14 34 ) ( 15 35 ) ( 16 36 ) ( 17 37 ) ( 18 38 ) ( 19 39 )
--------------------------------------------------------------------------------
Level:			2
Size:			256 kB
Cache groups:		( 0 20 ) ( 1 21 ) ( 2 22 ) ( 3 23 ) ( 4 24 ) ( 5 25 ) ( 6 26 ) ( 7 27 ) ( 8 28 ) ( 9 29 ) ( 10 30 ) ( 11 31 ) ( 12 32 ) ( 13 33 ) ( 14 34 ) ( 15 35 ) ( 16 36 ) ( 17 37 ) ( 18 38 ) ( 19 39 )
--------------------------------------------------------------------------------
Level:			3
Size:			25 MB
Cache groups:		( 0 20 1 21 2 22 3 23 4 24 5 25 6 26 7 27 8 28 9 29 ) ( 10 30 11 31 12 32 13 33 14 34 15 35 16 36 17 37 18 38 19 39 )
--------------------------------------------------------------------------------
********************************************************************************
NUMA Topology
********************************************************************************
NUMA domains:		2
--------------------------------------------------------------------------------
Domain:			0
Processors:		( 0 20 1 21 2 22 3 23 4 24 5 25 6 26 7 27 8 28 9 29 )
Distances:		10 21
Free memory:		31009.2 MB
Total memory:		32673.2 MB
--------------------------------------------------------------------------------
Domain:			1
Processors:		( 10 30 11 31 12 32 13 33 14 34 15 35 16 36 17 37 18 38 19 39 )
Distances:		21 10
Free memory:		31824.1 MB
Total memory:		32768 MB
--------------------------------------------------------------------------------


********************************************************************************
Graphical Topology
********************************************************************************
Socket 0:

+---------------------------------------------------------------------------------------------------------------+
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| |  0 20  | |  1 21  | |  2 22  | |  3 23  | |  4 24  | |  5 25  | |  6 26  | |  7 27  | |  8 28  | |  9 29  | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +-----------------------------------------------------------------------------------------------------------+ |
| |                                                   25 MB                                                   | |
| +-----------------------------------------------------------------------------------------------------------+ |
+---------------------------------------------------------------------------------------------------------------+

Socket 1:

+---------------------------------------------------------------------------------------------------------------+
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| | 10 30  | | 11 31  | | 12 32  | | 13 33  | | 14 34  | | 15 35  | | 16 36  | | 17 37  | | 18 38  | | 19 39  | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +-----------------------------------------------------------------------------------------------------------+ |
| |                                                   25 MB                                                   | |
| +-----------------------------------------------------------------------------------------------------------+ |
+---------------------------------------------------------------------------------------------------------------+

compute8: 2x Intel Xeon E5-2623 v4 Einklappen

--------------------------------------------------------------------------------
CPU name:	Intel(R) Xeon(R) CPU E5-2623 v4 @ 2.60GHz
CPU type:	Intel Xeon Broadwell EN/EP/EX processor
CPU stepping:	1
********************************************************************************
Hardware Thread Topology
********************************************************************************
Sockets:		2
Cores per socket:	4
Threads per core:	2
--------------------------------------------------------------------------------
HWThread	Thread		Core		Socket		Available
0		0		0		0		*
1		0		1		0		*
2		0		2		0		*
3		0		3		0		*
4		0		4		1		*
5		0		5		1		*
6		0		6		1		*
7		0		7		1		*
8		1		0		0		*
9		1		1		0		*
10		1		2		0		*
11		1		3		0		*
12		1		4		1		*
13		1		5		1		*
14		1		6		1		*
15		1		7		1		*
--------------------------------------------------------------------------------
Socket 0:		( 0 8 1 9 2 10 3 11 )
Socket 1:		( 4 12 5 13 6 14 7 15 )
--------------------------------------------------------------------------------
********************************************************************************
Cache Topology
********************************************************************************
Level:			1
Size:			32 kB
Cache groups:		( 0 8 ) ( 1 9 ) ( 2 10 ) ( 3 11 ) ( 4 12 ) ( 5 13 ) ( 6 14 ) ( 7 15 )
--------------------------------------------------------------------------------
Level:			2
Size:			256 kB
Cache groups:		( 0 8 ) ( 1 9 ) ( 2 10 ) ( 3 11 ) ( 4 12 ) ( 5 13 ) ( 6 14 ) ( 7 15 )
--------------------------------------------------------------------------------
Level:			3
Size:			10 MB
Cache groups:		( 0 8 1 9 2 10 3 11 ) ( 4 12 5 13 6 14 7 15 )
--------------------------------------------------------------------------------
********************************************************************************
NUMA Topology
********************************************************************************
NUMA domains:		2
--------------------------------------------------------------------------------
Domain:			0
Processors:		( 0 8 1 9 2 10 3 11 )
Distances:		10 21
Free memory:		30136.8 MB
Total memory:		32673.2 MB
--------------------------------------------------------------------------------
Domain:			1
Processors:		( 4 12 5 13 6 14 7 15 )
Distances:		21 10
Free memory:		31611.5 MB
Total memory:		32768 MB
--------------------------------------------------------------------------------


********************************************************************************
Graphical Topology
********************************************************************************
Socket 0:
+---------------------------------------------+
| +--------+ +--------+ +--------+ +--------+ |
| |  0 8   | |  1 9   | |  2 10  | |  3 11  | |
| +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ |
| |  32 kB | |  32 kB | |  32 kB | |  32 kB | |
| +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ |
| | 256 kB | | 256 kB | | 256 kB | | 256 kB | |
| +--------+ +--------+ +--------+ +--------+ |
| +-----------------------------------------+ |
| |                  10 MB                  | |
| +-----------------------------------------+ |
+---------------------------------------------+
Socket 1:
+---------------------------------------------+
| +--------+ +--------+ +--------+ +--------+ |
| |  4 12  | |  5 13  | |  6 14  | |  7 15  | |
| +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ |
| |  32 kB | |  32 kB | |  32 kB | |  32 kB | |
| +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ |
| | 256 kB | | 256 kB | | 256 kB | | 256 kB | |
| +--------+ +--------+ +--------+ +--------+ |
| +-----------------------------------------+ |
| |                  10 MB                  | |
| +-----------------------------------------+ |
+---------------------------------------------+

compute40: 4x Intel Xeon E5-4620 v4 Einklappen

--------------------------------------------------------------------------------
CPU name:	Intel(R) Xeon(R) CPU E5-4620 v4 @ 2.10GHz
CPU type:	Intel Xeon Broadwell EN/EP/EX processor
CPU stepping:	1
********************************************************************************
Hardware Thread Topology
********************************************************************************
Sockets:		4
Cores per socket:	10
Threads per core:	2
--------------------------------------------------------------------------------
HWThread	Thread		Core		Socket		Available
0		0		0		0		*
1		0		1		0		*
2		0		2		0		*
3		0		3		0		*
4		0		4		0		*
5		0		5		0		*
6		0		6		0		*
7		0		7		0		*
8		0		8		0		*
9		0		9		0		*
10		0		10		1		*
11		0		11		1		*
12		0		12		1		*
13		0		13		1		*
14		0		14		1		*
15		0		15		1		*
16		0		16		1		*
17		0		17		1		*
18		0		18		1		*
19		0		19		1		*
20		0		20		2		*
21		0		21		2		*
22		0		22		2		*
23		0		23		2		*
24		0		24		2		*
25		0		25		2		*
26		0		26		2		*
27		0		27		2		*
28		0		28		2		*
29		0		29		2		*
30		0		30		3		*
31		0		31		3		*
32		0		32		3		*
33		0		33		3		*
34		0		34		3		*
35		0		35		3		*
36		0		36		3		*
37		0		37		3		*
38		0		38		3		*
39		0		39		3		*
40		1		0		0		*
41		1		1		0		*
42		1		2		0		*
43		1		3		0		*
44		1		4		0		*
45		1		5		0		*
46		1		6		0		*
47		1		7		0		*
48		1		8		0		*
49		1		9		0		*
50		1		10		1		*
51		1		11		1		*
52		1		12		1		*
53		1		13		1		*
54		1		14		1		*
55		1		15		1		*
56		1		16		1		*
57		1		17		1		*
58		1		18		1		*
59		1		19		1		*
60		1		20		2		*
61		1		21		2		*
62		1		22		2		*
63		1		23		2		*
64		1		24		2		*
65		1		25		2		*
66		1		26		2		*
67		1		27		2		*
68		1		28		2		*
69		1		29		2		*
70		1		30		3		*
71		1		31		3		*
72		1		32		3		*
73		1		33		3		*
74		1		34		3		*
75		1		35		3		*
76		1		36		3		*
77		1		37		3		*
78		1		38		3		*
79		1		39		3		*
--------------------------------------------------------------------------------
Socket 0:		( 0 40 1 41 2 42 3 43 4 44 5 45 6 46 7 47 8 48 9 49 )
Socket 1:		( 10 50 11 51 12 52 13 53 14 54 15 55 16 56 17 57 18 58 19 59 )
Socket 2:		( 20 60 21 61 22 62 23 63 24 64 25 65 26 66 27 67 28 68 29 69 )
Socket 3:		( 30 70 31 71 32 72 33 73 34 74 35 75 36 76 37 77 38 78 39 79 )
--------------------------------------------------------------------------------
********************************************************************************
Cache Topology
********************************************************************************
Level:			1
Size:			32 kB
Cache groups:		( 0 40 ) ( 1 41 ) ( 2 42 ) ( 3 43 ) ( 4 44 ) ( 5 45 ) ( 6 46 ) ( 7 47 ) ( 8 48 ) ( 9 49 ) ( 10 50 ) ( 11 51 ) ( 12 52 ) ( 13 53 ) ( 14 54 ) ( 15 55 ) ( 16 56 ) ( 17 57 ) ( 18 58 ) ( 19 59 ) ( 20 60 ) ( 21 61 ) ( 22 62 ) ( 23 63 ) ( 24 64 ) ( 25 65 ) ( 26 66 ) ( 27 67 ) ( 28 68 ) ( 29 69 ) ( 30 70 ) ( 31 71 ) ( 32 72 ) ( 33 73 ) ( 34 74 ) ( 35 75 ) ( 36 76 ) ( 37 77 ) ( 38 78 ) ( 39 79 )
--------------------------------------------------------------------------------
Level:			2
Size:			256 kB
Cache groups:		( 0 40 ) ( 1 41 ) ( 2 42 ) ( 3 43 ) ( 4 44 ) ( 5 45 ) ( 6 46 ) ( 7 47 ) ( 8 48 ) ( 9 49 ) ( 10 50 ) ( 11 51 ) ( 12 52 ) ( 13 53 ) ( 14 54 ) ( 15 55 ) ( 16 56 ) ( 17 57 ) ( 18 58 ) ( 19 59 ) ( 20 60 ) ( 21 61 ) ( 22 62 ) ( 23 63 ) ( 24 64 ) ( 25 65 ) ( 26 66 ) ( 27 67 ) ( 28 68 ) ( 29 69 ) ( 30 70 ) ( 31 71 ) ( 32 72 ) ( 33 73 ) ( 34 74 ) ( 35 75 ) ( 36 76 ) ( 37 77 ) ( 38 78 ) ( 39 79 )
--------------------------------------------------------------------------------
Level:			3
Size:			25 MB
Cache groups:		( 0 40 1 41 2 42 3 43 4 44 5 45 6 46 7 47 8 48 9 49 ) ( 10 50 11 51 12 52 13 53 14 54 15 55 16 56 17 57 18 58 19 59 ) ( 20 60 21 61 22 62 23 63 24 64 25 65 26 66 27 67 28 68 29 69 ) ( 30 70 31 71 32 72 33 73 34 74 35 75 36 76 37 77 38 78 39 79 )
--------------------------------------------------------------------------------
********************************************************************************
NUMA Topology
********************************************************************************
NUMA domains:		4
--------------------------------------------------------------------------------
Domain:			0
Processors:		( 0 40 1 41 2 42 3 43 4 44 5 45 6 46 7 47 8 48 9 49 )
Distances:		10 21 31 21
Free memory:		110175 MB
Total memory:		130960 MB
--------------------------------------------------------------------------------
Domain:			1
Processors:		( 10 50 11 51 12 52 13 53 14 54 15 55 16 56 17 57 18 58 19 59 )
Distances:		21 10 21 31
Free memory:		110780 MB
Total memory:		131072 MB
--------------------------------------------------------------------------------
Domain:			2
Processors:		( 20 60 21 61 22 62 23 63 24 64 25 65 26 66 27 67 28 68 29 69 )
Distances:		31 21 10 21
Free memory:		106367 MB
Total memory:		131072 MB
--------------------------------------------------------------------------------
Domain:			3
Processors:		( 30 70 31 71 32 72 33 73 34 74 35 75 36 76 37 77 38 78 39 79 )
Distances:		21 31 21 10
Free memory:		103963 MB
Total memory:		131072 MB
--------------------------------------------------------------------------------


********************************************************************************
Graphical Topology
********************************************************************************
Socket 0:

+---------------------------------------------------------------------------------------------------------------+
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| |  0 40  | |  1 41  | |  2 42  | |  3 43  | |  4 44  | |  5 45  | |  6 46  | |  7 47  | |  8 48  | |  9 49  | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +-----------------------------------------------------------------------------------------------------------+ |
| |                                                   25 MB                                                   | |
| +-----------------------------------------------------------------------------------------------------------+ |
+---------------------------------------------------------------------------------------------------------------+

Socket 1:

+---------------------------------------------------------------------------------------------------------------+
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| | 10 50  | | 11 51  | | 12 52  | | 13 53  | | 14 54  | | 15 55  | | 16 56  | | 17 57  | | 18 58  | | 19 59  | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +-----------------------------------------------------------------------------------------------------------+ |
| |                                                   25 MB                                                   | |
| +-----------------------------------------------------------------------------------------------------------+ |
+---------------------------------------------------------------------------------------------------------------+

Socket 2:

+---------------------------------------------------------------------------------------------------------------+
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| | 20 60  | | 21 61  | | 22 62  | | 23 63  | | 24 64  | | 25 65  | | 26 66  | | 27 67  | | 28 68  | | 29 69  | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +-----------------------------------------------------------------------------------------------------------+ |
| |                                                   25 MB                                                   | |
| +-----------------------------------------------------------------------------------------------------------+ |
+---------------------------------------------------------------------------------------------------------------+

Socket 3:

+---------------------------------------------------------------------------------------------------------------+
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| | 30 70  | | 31 71  | | 32 72  | | 33 73  | | 34 74  | | 35 75  | | 36 76  | | 37 77  | | 38 78  | | 39 79  | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |  32 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | | 256 kB | |
| +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ +--------+ |
| +-----------------------------------------------------------------------------------------------------------+ |
| |                                                   25 MB                                                   | |
| +-----------------------------------------------------------------------------------------------------------+ |
+---------------------------------------------------------------------------------------------------------------+

Node-level performance (likwid-bench)

Parallel data transfer rate - compute20 Einklappen: Parallel data transfer rate on btrzx2 compute20 nodes. In the 2-socket data set, both sockets are balanced.
Vector triad - compute20 Einklappen: Performance of the vector triad on btrzx2 compute20 nodes.
Parallel data transfer rate - compute8 Einklappen: Parallel data transfer rate on btrzx2 compute8 nodes. In the 2-socket data set, both sockets are balanced.
Vector triad - compute8 Einklappen: Performance of the vector triad on btrzx2 compute8 nodes.

MPI benchmarks (OSU)

MPI Bandwidth Einklappen: P2P MPI Bandwidth between processes on the same socket (by core), the same node but different sockets (by socket), and different nodes (by node) using ofi or shm:ofi fabric interfaces.
MPI Latency Einklappen: Single-point measurements of the point-2-point MPI latency (limit: size 0) between processes on the same socket (by core), the same node but different sockets (by socket), and different nodes in the same or different racks (by node).

Statistics

Note: The workload is broken down into group contributions. However, the colors were chosen randomly for each single graph.

2017 - Job size [compute20]Einklappen: Spent computation time by job size and runtime.
2017 - Job size [compute8]Einklappen: Spent computation time by job size and runtime.
2017 - Job size [compute40]Einklappen: Spent computation time by job size and runtime.
2018 - Job size [compute20]Einklappen: Spent computation time by job size and runtime.
2018 - Job size [compute8]Einklappen: Spent computation time by job size and runtime.
2018 - Job size [compute40]Einklappen: Spent computation time by job size and runtime.