.. _Huawei_server: =================== Huawei server setup =================== .. Contents:: This page contains information for selected Huawei servers: XH620_ and RH2288H_. Documentation and software ========================== XH620 V3 server --------------- * XH620_ product information.From * XH620_Documentation_. * XH620_Downloads_. * iBMC_ management processor. .. _XH620_Documentation: http://support.huawei.com/enterprise/productNewOffering?idAbsPath=7919749|9856522|21782478|21149952|21456501&pid=21456501&tab=doc&docType=DOCTYPE0&productname=XH620%20V3 .. _XH620_Downloads: http://support.huawei.com/enterprise/en/server/xh620-v3-pid-21456501/software .. _XH620: http://e.huawei.com/en/products/cloud-computing-dc/servers/x-series/xh620-v3 RH2288H V3 server ----------------- * RH2288H_User_Guide_. * RH2288H_Software_downloads_ are at the bottom of the RH2288H_ page *Technical Support* section. The latest software (topmost on the list) *Version and Patch Number* should be used. The software includes drivers for supported OSes, and BIOS and iBMC_ firmware updates. .. _RH2288H: http://e.huawei.com/en/products/cloud-computing-dc/servers/rh-series/rh2288h-v3 .. _RH2288H_User_Guide: http://support.huawei.com/enterprise/docinforeader.action?contentId=DOC1000054727&partNo=10072 .. _RH2288H_Software_downloads: http://support.huawei.com/enterprise/productNewOffering?tab=software&pid=9901881&lang=en X6800 chassis ------------- * FusionServer X6800_ Data Center Server. * X6800_White_paper_. * X6800_Documentation_. * X6800_Software_ downloads. Rack cabinet requirements from the X6800_White_paper_: * The depth of an X6800 chassis is 898 mm, and therefore the cabinet depth must be no less than 1200 mm. .. _X6800: http://e.huawei.com/en/products/cloud-computing-dc/servers/x-series/x6800 .. _X6800_Documentation: http://support.huawei.com/enterprise/en/server/x6800-pid-21149487/ .. _X6800_Software: http://support.huawei.com/enterprise/en/server/x6800-pid-21149487/software/ .. _X6800_White_paper: http://e.huawei.com/en/marketing-material/onLineView?MaterialID={FA9AF2F4-FB8A-4157-95B5-4B4AD20A6157} SUMMARY: How to manage a Huawei server ====================================== This is a summary of the steps required to configure a factory default Huawei server so that it becomes manageable. The most important steps are to reconfigure the BIOS and the iBMC_ management processor. The sections below provide full details. Assumptions: * Your server as well as management networks use a DHCP_ server to manage IP addresses. The following steps must be taken in numerical order: 1. Attach a console screen and keyboard and boot the server into BIOS Setup mode (press DEL). 2. Remove the useless BIOS setup (supervisor) password installed by Huawei. 3. Read the server's port 1 Ethernet MAC address. 4. Go into the iBMC_ Configuration menu and read the iBMC_ Ethernet MAC address. - Configure the iBMC_ Ethernet for DHCP_ IP address assignment (a static IP address is the factory default). - For a X6800_ chassis only: Configure the *Aggregation* chassis management Ethernet port. 5. In BIOS Setup press F10 to save the changes and exit to reset the server. 6. Connect Ethernet cables to the server port 1 and the iBMC_ *Mgmt* ports. - For a X6800_ chassis only: Connect an Ethernet cable to the chassis management *Aggregation* Ethernet port in stead of the server's *Mgmt* port. 7. Configure your network's DHCP_ server with the MAC addresses learned in steps 3 and 4. - Make sure that the iBMC_ responds to network ping_ packets. 8. Use the Huawei *uMate* tool (from the FusionServer_tools_ web page) to reconfigure BIOS settings as required (*uMate* works through the iBMC_). - It is a good idea to update also the BIOS and iBMC_ firmware. - Note: The *uMate* is unable to set **all** required BIOS parameters, so some parameters may have to be set manually in the BIOS Setup. 9. Reboot the server and perform a PXE_ network installation of operating system. .. _DHCP: https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol .. _ping: https://en.wikipedia.org/wiki/Ping_%28networking_utility%29 .. _PXE: https://en.wikipedia.org/wiki/Preboot_Execution_Environment Booting and BIOS configuration ============================== A boot menu will be presented: * F5 to select keyboard: Only US English and French are available! * DEL to enter BIOS setup * F11 to Boot Manager * F12 to PXE boot The server like RH2288H will first spend some time configuring storage connected to the LSI MegaRAID_ controller, then proceed with the booting. .. _MegaRAID: http://www.avagotech.com/products/server-storage/raid-controllers/ BIOS keyboard layout -------------------- Using F5 at startup, the BIOS can configured to accept one of the following national keyboard layouts: * US English, French. In order to enter the BIOS setup initially, you have to type the default password: Huawei12#$ However, this won't work with many national keyboards because the characters # and $ will be placed on different keys. The trick for national keyboards is to type this default password as: * Huawei12(shift)3(shift)4 For newer servers such as XH321 V5 the Administrator password is: Admin@9000 The trick for national keyboards is to type this default password with (shift)2 for "@" as: * Admin(shift)2(un-shift)9000 Since BIOS thinks the keyboard uses a US layout, this will work correctly. Configure BIOS settings ----------------------- Press DELETE to enter the BIOS setup. The (silly) Huawei password **Huawei12#$** is required to enter BIOS setup! Press F9 to set defaults, F10 to Save and Exit. Consider changing the defaults: * Go to the **Security** submenu and select **Clear supervisor password** (see below). * Go to the **Boot** submenu and select appropriate boot devices, for example: * **Boot Type** = **Legacy Boot Type**. * Go to the **Boot Type Order** submenu and select appropriate boot order: * **PXE, CD, Hard Disk** (use F5/F6 to move down/up). * Go to the **Advanced** submenu: * In **Intel RC Group**: * Go to **Advanced Power Management Configuration** and set **Power Policy Select** to **Performance** for HPC computing. **Notice:** Setting this will (unexpectedly) reset other unrelated parameters such as Hyper-Threading! * Go to *Processor Configuration*. Set **Hyper-Threading** to **Disabled** for HPC computing. * Go to **PXE configuration** to write down the on-board **Ethernet MAC address**. Also verify the PXE enabled/disabled settings. Note: On the XH620_ V3 server, the Ethernet MAC address seems to be defined by the small mini-board containing the LAN ports at the server's front. * Go to **IPMI iBMC configuration** to update the iBMC_ management controller configuration: * Write down the **iBMC Ethernet MAC address**. * Set **Restore on AC Power Loss** to **Last state**. * Verify the *Set iBMC services* which actually means the SSH_ login service (enabled by default). * In **iBMC Configuration** configure: - The iBMC_ network port **iBMC&NCSI Select** is set to *Dedicated* by default. Select the appropriate value for your installation: - **Dedicated**: Use the server's front port. - **Aggregation**: Use the X6800 chassis consolidated management port (cannot be selected with *Auto*). - **Shared-LOM**: Share the server's Ethernet port. - **Auto**: If you select this option, the iBMC automatically selects a network port as the iBMC management network port based on the connection status of optional network ports. You need to specify optional network ports by selecting check boxes. If multiple network ports are connected, the iBMC selects a network port as the iBMC management network port based on the following priority: dedicated network port > LOM network port > PCIe extern port. The aggregation network port cannot be automatically selected. - You *may* change the **iBMC password**. (**Note:** special characters will be used - remember that BIOS uses US keyboard layout!) - Set the IPv4/IPv6 *IP Source* configuration for **DHCP**. * In *Misc configuration* you should set **Wake on LAN** to **enabled**. .. _iBMC: http://e.huawei.com/en/products/cloud-computing-dc/servers/accessories/ibmc .. _SSH: https://en.wikipedia.org/wiki/Secure_Shell Missing BIOS configuration items ................................ The following desired BIOS settings are unavailable (as of BIOS V350): * Keyboard NUMLOCK = Enabled/Disabled * There does not seem to be any way to display the iBMC_ Event Log within the BIOS setup. In stead you may use the iBMC_ GUI, or the command ``ipmitool sel elist`` in Linux. BIOS settings for Intel OmniPath network adapter ................................................ If you install an Intel OmniPath network adapter, there is an Intel recommendation (BIOS dependent) to set the PCIe bus speed to **Gen2**, see our `OmniPath page `_. The default BIOS setting may be that the PCIe speed is set to Auto (may vary with BIOS). For a PCIe Gen3 x16 adapter the PCIe bus speed should be 8 GT/s, whereas Gen2 speed would only be 5 GT/s. Please verify your adapter's speed. In the Huawei BIOS configure this in the *Advanced->Intel RC Group->IIO Config -> IIO1*: * Select the correct PCIe slot, it may be the Port3A x16 port. * Set the link speed to **Auto**. Optimized BIOS settings ....................... Download the manual *Intel® Omni-Path Performance Tuning User Guide*. See Chapter **2.0 BIOS Settings** about recommended settings, which are likely important for any type of network fabric. For the Huawei server BIOS configuration please first perform the above standard configurations for HPC servers. Then go to the **Advanced->Intel RC Group** and configure: * **Advanced Power Management Configuration**: - EIST Support (Enhanced Intel SpeedStep_ Technology) = **Enabled**. - Turbo Mode = **Enabled**. - CPU C-State = **Enabled**. - Processor C3 report = **Disabled**. - Processor C6 report = **Enabled**. * **IIO Configuration**: - **IIO1 Configuration**: - All ports: Link Speed = **Auto** - All ports: PCI-E Port Max Payload Size = **Auto** - IOU Non-posted Prefetch = **Disabled** (available with BIOS V350 or later). - Intel VT for Directed I/O (VT-d) = **Disabled**. * **QPI Configuration:** - Snoop Mode Select = **HomeSnoop+OSB** (improved memory bandwidth) * **Memory Configuration:** Numa (NUMA_ Optimized) = **Enabled**. .. _SpeedStep: https://en.wikipedia.org/wiki/SpeedStep .. _NUMA: https://en.wikipedia.org/wiki/Non-uniform_memory_access To save the settings and reboot: * Press F10 **Save and Exit**. BIOS password protection ------------------------ Many BIOS functions are password protected, making normal server operation quite cumbersome. For example, PXE booting is only permitted after typing the BIOS password! Huawei servers seem to have a factory default BIOS as well as iBMC_ password which is printed on a label on top of the chassis:: Huawei12#$ which you can easily find by this `Google search `_ or with some effort look up in the system *User Guide*. system In the BIOS setup you should select *Clear supervisor password* in order to disable this annoying password. iBMC network configuration -------------------------- Go to the XH620_Documentation_ page and find the document entitled *X6800 Server Node V100R003 iBMC User Guide*, this contains the full iBMC_ documentation. The server's factory default is for the iBMC_ to connect to the *Dedicated port* on the server cabinet (label: *Mgmt*). You can connect a network cable to that port. The factory default iBMC_ network uses a fixed IP address. The factory default static IP address of each iBMC_ is 172.31.1.128 + slot-number, where slot-number=1..8. The netmask is 255.255.255.0. You may change the IP configuration to DHCP_, provided you have configured the iBMC_'s MAC-address in your DHCP_ server. This is described above under BIOS configuration. Alternatively, from the Linux OS you can configure the iBMC_ to use DHCP_:: ipmitool lan set 1 ipsrc dhcp If you want to use the X6800_ chassis consolidated management port (label: *Mgmt* on the chassis "ears"), you have to change the iBMC configuration. This is described above under BIOS configuration. Alternatively, connect a cable temporarily to the *Dedicated* port so that the iBMC_ connects to the network and uses DHCP_. Now you can use SSH_ to login to the iBMC_ IP address as user *root* with the (silly) Huawei password. Read the *iBMC User Guide* sections: * Setting the Network Port Mode (netmode) * Setting the active port (activeport) Query the iBMC_ network port mode:: iBMC:/->ipmcget -d ipinfo Change the iBMC_ port configuration:: ipmcset -d netmode -v 1 ipmcset -d activeport -v 3 If you have many nodes to configure, this is a very time-consuming process. You can reconfigure many iBMC's using a script ibmc_consolidated_port.sh_, it requires the command sshpass_ to be installed with this RPM:: yum install sshpass .. _ibmc_consolidated_port.sh: ftp://ftp.fysik.dtu.dk/pub/Huawei/ibmc_consolidated_port.sh .. _sshpass: https://sourceforge.net/projects/sshpass/ .. _DHCP: https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol iBMC security warning --------------------- If the iBMC_ is connected to a network (see above), anyone with access to login using SSH_ or the web GUI (port 443) as the default user *root* and the default Huawei password can take over the iBMC_ controller and thereby control the server remotely! This is documented in the system *User Guide* section *Accessing the CLI of the iBMC Management Software*. It is strongly recommended to: * Make sure the iBMC_ network is completely private with no possibility of any unauthorized access by SSH_. * Change the default iBMC_ password (see above). Obviously, Huawei ought to configure random iBMC_ passwords from the factory and print them on labels attached to the servers. SSH login to iBMC ----------------- The integrated management module (referred to as the **iMana**) is a control unit used to manage servers. When the iBMC_ network interface has been configured correctly, it is possible to login to the iBMC_ CLI using SSH. **Note:** If you have several SSH authentication key files ($HOME/.ssh/id_*) they will be tried in turn, and since the iBMC_ accepts a maximum of 3 login attempts, SSH logins may fail with the error:: Received disconnect from 10.x.x.x port 22:2: Too many authentication failures Workaround: Specify only 1 of the keys to the SSH command, for example:: ssh -i $HOME/.ssh/id_rsa iBMC operation ============== See the HUAWEI Server iMana_200_User_Guide_ which explains iBMC_ GUI and CLI operations. .. _iMana_200_User_Guide: http://support.huawei.com/enterprise/docinforeader.action?contentId=DOC1000038843&partNo=10072 For example, iBMC_ and BIOS upgrades can be performed using the iBMC_ CLI interface via SSH login. The guide instructions in *3.4.11 Upgrading the Software (upgrade)* are:: iBMC:/->ipmcset -d upgrade -v /tmp/*.hpm [option] option: 0: Do not restart the iMana. You need to upload the upgrade files to the tmp directory on the target server over file transmit tool (just as SFTP), and then upgrade the iMana software in CLI mode. Display the iBMC_ LAN configuration:: iBMC:/->ipmcget -d ipinfo EthGroup ID : 1 Net Mode : Manual Net Type : Dedicated IPv4 Information : IP Mode : dhcp IP Address : 10.5.135.94 Subnet Mask : 255.255.0.0 Default Gateway : scp MAC Address : 20:3d:b2:20:bf:e5 ... Modify the LAN configuration to *Manual* and *Aggregated*:: ipmcset -d netmode -v 1 ipmcset -d activeport -v 3 where the modes are: * 1: manual * 2: adaptive and the network ports are: * 0: indicates dedicated network port * 1: indicates LOM * 2: indicates PCIe extern port * 3: indicates aggregation network port IRC: iBMC remote console ------------------------ In the iBMC_ web page you can start a system console as a Java client. Alternatively install the **Independent Remote Console** (IRC_) which is a remote control tool developed by Huawei based on the server management software iBMC_. The IRC_ offers the same functions as the Remote Virtual Control function of the iBMC_ WebUI. With the IRC_, you can access and manage the server in real time. The IRC_ does not depend on the browser or JRE version, but comes with its own JRE environment. .. _IRC: http://support.huawei.com/onlinetoolsweb/itexpress/kvmclient/en.html Download IRC_ Linux package ``kvm_client_linux.zip`` from the FusionServer_tools_ page. Unpack the files to a new directory:: mkdir kvm_client_linux cd kvm_client_linux unzip .../kvm_client_linux.zip The script ``KVM.sh`` contains the KVM console JRE application. iBMC power control ------------------ The server power may be controlled using the iBMC CLI:: ipmcset -d powerstate -v