Dell R640 server

This page contains information out Dell PowerEdge R640 could servers which we have deployed in our Niflheim cluster.

Dell OpenManage

Download the OpenManage software ISO image from the R640_downloads page in the Systems Management download category.

Download the Dell EMC OpenManage Deployment Toolkit (Linux) DTK ISO file and mount it on /mnt.

Dell EMC System Update (DSU)

Dell EMC System Update (DSU) is a script optimized update deployment tool for applying Dell Update Packages (DUP) to Dell EMC PowerEdge servers. See the DSU manuals.

The DSU may also be configured as a Yum repository, see the DSU page. The commands are:

curl -O https://linux.dell.com/repo/hardware/dsu/bootstrap.cgi
bash bootstrap.cgi

Alternatively, download the Systems-Management_Application_* file and execute it.

This will create the Yum repository file:

/etc/yum.repos.d/dell-system-update.repo

Install RPM packages including iDRAC tools:

yum install dell-system-update syscfg srvadmin-idracadm7

Using DSU to preview Dell upgrades:

/usr/sbin/dsu -n -p

To apply Dell upgrades:

/usr/sbin/dsu -u

Systems Management Managed Node Core and CLI

Install the package:

yum install srvadmin-omacore

Disk reports:

omreport storage vdisk                      # List of Virtual Disks in the System
omreport storage pdisk controller=1         # List of Physical Disks on Controller 1
omreport storage pdisk controller=1 vdisk=0 # List of Physical Disks belonging to Virtual Disk0

syscfg command

The SYSCFG command is located in:

/opt/dell/toolkit/bin/syscfg

racadm command

Make a soft link for the racadm command:

ln -s /opt/dell/srvadmin/bin/idracadm7 /usr/local/bin/racadm

Get Health LED status:

racadm getled

Get system and version information:

Service Tag: racadm getsvctag
System info: racadm getsysinfo -s
Versions:    racadm getversion
BIOS:        racadm getversion -b
CPLD:        racadm getversion -c
iDRAC:       racadm getversion -f idrac

Get system logs:

SEL Event Log: racadm getsel
Lifecycle Log: racadm lclog view

PERC H330 RAID controller

The R640 comes with a PERC H330 RAID controller.

By default the installed disks are unallocated, and you have to configure their usage.

Press F2 during start-up to enter the setup menus. Go to the Device Settings menu.

Configure the H330 via the menu item Device Settings and select the RAID controller item:

  • In the RAID controller Main Menu select the Configuration Management item.
  • Change the disk setup into Convert to Non-RAID.
  • In the Controller Management menu item Select Boot Device define the non-RAID disk as the boot device.

Press Finish to save all settings.

raidcfg tool

The OpenManage tool raidcfg can be installed from the above mentioned Dell EMC OpenManage Deployment Toolkit (Linux) folder /mnt/RPMs/rhel7/x86_64/:

yum install raidcfg*rpm

See raidcfg quick reference.

To list installed RAID controllers:

/opt/dell/toolkit/bin/raidcfg controller

perccli tool

The perccli tool for Linux. Install the RPM:

tar xzf perccli_linux_NF8G9_A07_7.529.00.tar.gz
cd perccli_7.5-007.0529_linux/
yum install perccli-007.0529.0000.0000-1.noarch.rpm
ln -s /opt/MegaRAID/perccli/perccli64 /usr/local/bin

Se the Reference Guide at https://topics-cdn.dell.com/pdf/dell-sas-hba-12gbps_reference-guide_en-us.pdf

Booting and BIOS configuration

Press F2 during start-up to enter the BIOS and firmware setup menus. Go to the BIOS Settings menu.

Minimal configuration of a new server or motherboard

At our site the following minimal settings are required for a new server or a new motherboard. Remaining settings will be configured by SYSCFG (see below).

The Dell iDRAC9 (BMC) setup is accessed via the System Setup menu item iDRAC Settings:

  • In the System Summary page read the NIC iDRAC MAC Address from this page for configuring the DHCP server.
  • In the Network page set the Enable IPMI over LAN to Enabled.

Go to the System Setup menu item Device Settings and select the Integrated NIC items:

  • In the NIC Main Configuration Page select NIC Configuration. We use NIC port 3 (1 Gbit) as the system's NIC.
  • Read the NIC Ethernet MAC Address from this page for configuring the DHCP server.
  • Select the Legacy Boot Protocol item PXE.

Boot Sequence menu:

  • Click the Boot Sequence item to move PXE boot up above the hard disk boot.

SYSCFG For BIOS Configuration

The above described SYSCFG uses BIOS interfaces to manipulate boot order and BIOS settings. It configures all options available on the BIOS setup screen, including boot order, embedded device configuration, and asset tag management.

Record all settings to a file:

/opt/dell/toolkit/bin/syscfg -o r640.ini

Read all settings from a file:

/opt/dell/toolkit/bin/syscfg -i r640.ini

During the next boot the server will run a Lifecycle Controller task reconfiguring the server, and then it will reboot.

Boot settings menu

  • Boot Mode = BIOS.
  • In the Boot Sequence menu:
    • Click the Boot Sequence item to move PXE boot up above the hard disk boot (if desired).
    • Verify that the correct devices are selected in Boot Option Enable/Disable.

Memory settings menu

  • Memory Operating Mode = Optimizer Mode.
  • Node interleaving = Disabled.
  • Opportunistic Self-Refresh = Disabled.

Processor settings menu

  • Disable Hyperthreading by Logical Processor = Disabled.
  • Virtualization Technology = Disabled.
  • Dell Controlled Turbo = Disabled.
  • Sub NUMA Cluster = Disabled.

System Profile Settings menu

  • System Profile = Performance.

System Security menu

  • AC Power Recovery = Last state.

Miscellaneous Settings menu

  • Keyboard NumLock = Off.

PXE boot setup

Go to the System Setup menu item Device Settings and select the Integrated NIC items:

  • In the NIC Main Configuration Page select NIC Configuration. We use NIC port 3 (1 Gbit) as the system's NIC.
  • Read the NIC Ethernet MAC Address from this page for configuring the DHCP server.
  • Select the Legacy Boot Protocol item PXE.
  • Set Wake On LAN to Enabled.
  • Set the Boot Retry Count = 3 if desired.
  • Disable PXE boot for all unused NICs (port 1).

Press Finish to save all settings.

iDRAC (BMC) setup

The Dell iDRAC9 (BMC) setup is accessed via the System Setup menu item iDRAC Settings:

  • In the System Summary page read the NIC iDRAC MAC Address from this page for configuring the DHCP server.
  • In the Network page set the Enable IPMI over LAN to Enabled.
  • In the User Configuration page set the User 2 (root) Administrator user name and change the password.
  • Optional: In the Thermal page set Thermal: Maximum Performance.

Press Finish to save all settings.

SMTP alerts from iDRAC

First you must configure the DNS name of the iDRAC, see https://www.dell.com/support/article/us/en/04/sln309388/dell-idrac-how-to-configure-the-email-notifications-for-system-alerts-on-idrac-7-8-and-9?lang=en

In the iDRAC web GUI go to iDRAC Settings->Connectivity->Common Settings and configure the DNS domain name and hostname.

Then configure alerts in Configuration->System Settings->Alert Configuration->Alerts. Then go to the SMTP (Email) Configuration sub-menu and set up SMTP alerts.

TSR reports from iDRAC

TSR system reports for Dell Support cases are normally generated using the iDRAC web interface.

It is also possible to generate TSR reports using the racadm techsupreport subcommand:

racadm techsupreport collect

Check the progress of the report generation with:

racadm jobqueue view

After some minutes export the completed TSR report to a local ZIP file:

racadm techsupreport export -f <filename>.zip

iDRAC server power management

The server power can be managed from the iDRAC web interface under the Dashbord pull-down menu Graceful shutdown.

The iDRACC9 CLI can also be used to manage server power. Use SSH to login to the CLI, and the Help menu states this:

/admin1-> racadm help serveraction
serveraction -- perform system power management operations
Usage:
racadm serveraction <action>
<action>:  server power management operation to perform.  Must be one of:
           graceshutdown   : perform a graceful shutdown of server
           powerdown       : power server off
           powerup         : power server on
           powercycle      : perform server power cycle
           hardreset       : force hard server power reset
           powerstatus     : display current power status of server
           nmi             : Genarate Non-Masking Interrupt to halt system operation

To hard power cycle the server:

racadm serveraction hardreset

iDRAC or LifeCycle Controller errors

If the iDRAC controller seems frozen, or if the LifeCycle Controller (LCC) has errors, one should try to perform a deep power drain.

We have seen the R640 LCC going into a Recovery Mode preventing the setting of BIOS parameters using SYSCFG, and an error message on the console:

Couldn't locate device handle for MAS001.. System rebooting

This error was resolved by a deep power drain of the server.

Deep power drain procedure
  • Pull both power cables from the server
  • Hold down the power button for 30 seconds
  • Plug the power cables back in
  • Wait for 30-60 seconds before powering the server on. This will drain the residing power from the capacitors and waiting 30-60 seconds before powering on will allow the iDRAC to complete post.
  • Connect via the idrac and follow the boot process via the virtual or physical console.

iDRAC Easy Restore

See the iDRAC9 User's Guide:

After you replace the motherboard on your server, Easy Restore allows you to automatically restore the following data:

  • System Service Tag
  • Asset Tag
  • Licenses data
  • UEFI Diagnostics application
  • System configuration settings—BIOS, iDRAC, and NIC

Easy Restore uses the Easy Restore flash memory to back up the data. When you replace the motherboard and power on the system, the BIOS queries the iDRAC and prompts you to restore the backed-up data. The first BIOS screen prompts you to restore the Service Tag, licenses, and UEFI diagnostic application. The second BIOS screen prompts you to restore system configuration settings. If you choose not to restore data on the first BIOS screen and if you do not set the Service Tag by another method, the first BIOS screen is displayed again. The second BIOS screen is displayed only once.

IT-wiki: Dell_R640 (last edited 2020-04-02 08:46:48 by OleHolmNielsen)