Supported configurations for 8 Gigabit Fibre Channel - IBM BladeCenter H (7989, 8852)

Source

RETAIN tip: H202210

Symptom

• Unexpected Link Negotiation Errors when starting from Storage Area Network (SAN). The blade could fail to start, or the fiber connection may negotiate to a speed less than 8 Gigabit (Gb).
• Excessive Cyclical Redundancy Check (CRC) or Encode errors are seen in switch logs during runtime. These errors can cause link drops, poor performance, or failover in a Multipath environment.

See RETAIN Tip H202550 (MIGR-5087270) for help with G-port log in symptoms.

Affected Configurations

The system can be any of the following IBM servers:

• BladeCenter H, type 7989, any model
• BladeCenter H, type 8852, any model

The system is configured with one or more of the following IBM options:

• Brocade 8 Gigabit Ten-Port SAN Switch Module for IBM BladeCenter, option part number 44X1921, replacement part number 44X1927
• Brocade 8 Gigabit Twenty-Port SAN Switch Module for IBM BladeCenter, option part number 44X1920, replacement part number 44X1926
• Emulex 8 Gigabit Fibre Channel Expansion Card (CIOv) for IBM BladeCenter, option part number 46M6140, replacement part number 46M6138
• QLogic 8 Gigabit Twenty-Port SAN Switch Module for IBM BladeCenter, option part number 44X1905, replacement part number 44X1914
• QLogic 8 Gigabit Fibre Channel Expansion Card (CIOv) for IBM BladeCenter, option part number 44X1945, replacement part number 44X1948
• QLogic 8 Gigabit Intelligent Pass-thru Module for IBM BladeCenter, option part number 44X1907, replacement part number 44X1916
• QLogic Ethernet and 8 Gigabit Fibre Channel Expansion Card (CFFh) for IBM BladeCenter, option part number 44X1940, replacement part number 44X1943

This tip is not software specific.

The system has the symptom described above.

Solution

Configure the Fibre Channel Host Bus Adapter (HBA) and switch module as indicated in the Additional Information section.

Additional Information

Select one of the following five supported configurations for 8 Gigabit Fibre Channel:

Configuration 1:

Emulex 8 Gigabit Host Bus Adapter CIOv configuration:

Firmware Boot Code EDC SERDES Update Speed Setting

2.01a9 5.12a10 3.22 (see Note 1) (see Note 1) Auto (see Note 5)

Brocade 8 Gigabit FCSM switch configuration:

Firmware FPGA Internal Port Settings: Speed Setting Fill Word

6.4.2b (see Note 10) 97 (see Note 7) Auto (see Note 5, Note 8) 1 (see Note 6)

Configuration 2:

Emulex 8 Gigabit CIOv with 8 Gigabit QLogic IPTM/FCSM

Emulex 8 Gigabit Host Bus Adapter CIOv configuration

Firmware Boot Code EDC SERDES Update Speed Setting: Local Boot Boot from SAN

2.01a9 5.12a10 3.22 (see Note 1) (see Note 1) Auto Auto

QLogic 8 Gigabit IPTM/FCSM switch configuration:

Firmware Internal Port Speed Setting: Local Boot Boot from SAN

7.10.1.28 (see Note 8) Auto Auto

Configuration 3:

QLogic 8 Gigabit CIOv/CFFh with 8 Gigabit Brocade FCSM (see Note 9)

QLogic 8 Gigabit Host Bus Adapter CIOv/CFFh configuration:

Update Driver and/or Bundled Firmware EDC level based on Switch type Firmware Speed Setting

(see Note 2) (see Note 3, Note 4) 05.07.01 (see Note 3) Fixed

Brocade 8 Gigabit FCSM switch configuration:

Firmware FPGA Internal Port Settings: Speed Setting Fill Word

6.4.2b (see Note 10) 97 (see Note 7) (see Note 8) 1 (see Note 6)

Configuration 4:

QLogic 8 Gigabit CIOv/CFFh with QLogic 8 Gigabit IPTM/FCSM (see Note 9)

QLogic 8 Gigabit CIOv/CFFh Host Bus Adapter configuration:

Update Driver and/or Bundled Firmware EDC level based on Switch type Firmware Speed Setting: Local Boot Boot from SAN

(see Note 2) (see Note 3, Note 4) 05.07.01 (see Note 3) Auto Fixed

QLogic 8 Gigabit IPTM/FCSM switch configuration:

Firmware Internal Port Speed Setting: Local Boot Boot from SAN

7.10.1.28 (see Note 8) Fixed Fixed

Configuration 5:

With any configuration that includes XIV storage models prior to 8 Gb Gen3 (4 Gb machine types 2810-A14 or 2812-A14), XIV firmware level 10.2.4a or later is required.

In all configurations, ensure users are using the latest driver for the operating system.

If the issues continue after using one (1) of the configurations listed, set the speed on both the HBA and the IPTM/FCSM to 4 Gb fixed.

If the issue is seen with QLogic IPTM/FCSM in bay-3 or bay-4 using the supported configuration, try running QLogic IPTM/FCSM in slot-8 or slot-10.

Notes:

1. EDC refers to Electronic Dispersion Compensation and is required for optimal Signal Integrity. SERDES refers to SERializer/DESerializer and is required for optimal signal transmit settings. Use 'Offline ISO for x86 - VS' for Intel and AMD based systems and 'Offline ISO for Power PC - VS' for IBM BladeCenter Power Systems. This update includes EDC 3.22, SERDES settings, 2.01a9 Firmware, and 5.12a10 Universal Start Code.

   Upgrade Utility:

   http://www.emulex.com/downloads/ibm/fc-HBA-FW-and-boot-code/lpe1205-ciov-for-ibm-BladeCenter/edc-updates.html

2. Depending on the operating system running on the blade, the driver and bundled firmware will need to be updated, or for operating systems where the inbox driver is the supported driver, only the bundled firmware will need to be updated.
   

   To determine the appropriate update for your operating system, reference Table 3-1 in the Application Note for the update:

   http://filedownloads.qlogic.com/files/Driver/82210/AppNote_qlogic_8Gb_FC_ExpCard_EDC_FW_Update_CU0251401-00A.pdf

   If the operating system requires both a driver and bundled firmware update, download the specified driver package and install it according to the instructions provided. The driver install package will update both the driver and the bundled firmware.

   If theoperating system specifies the inbox driver, download the specified RPM file to update the bundled firmware and run the RPM. The update files can be downloaded from IBM Fix Central at the following URL:

   http://www.ibm.com/support/fixcentral/

3. The EDC code level will be determined by the switch type being used. The EDC updates can be found on IBM Fix Central. Select the update for the operating system (Linux, Windows, or VMware), and the correct switch manufacturer being used (Brocade or QLogic).

   The blade must be restarted after the EDC update is performed. It is critical that the system not be interrupted or powered off during this restart. This restart will take slightly longer than a normal restart. The EDC update file also includes the latest multiboot code containing firmware 05.06.03. When the EDC update is performed, all code on the HBA will be updated at the same time.

4. EDC code is applicable only to the CIOv card.
   
   
5. For the majority of environments, auto/auto negotiates to 8 Gigabit speed, as expected. However, in some implementations we have seen speeds negotiate to less than 8 Gigabit. If this behavior is observed in your environment, we would recommend fixed port speeds as a viable, supported configuration.
   
   
6. Fill word settings can be verified using the 'portcfgshow' command (also shows the port speed settings). The fill word for internal blade facing ports always should be set to 1 (which is the switch default). The fill word for external ports might need to be modified based on the requirements of your upstream device.
   
   
7. For the Brocade 8 Gigabit switch module, the FPGA version number can be verified using the Command Line Interface (CLI), command 'fpga_update'.
   
   
8. The speed setting for internal blade facing ports should be set as specified in this document. Speed settings for external ports might need to be modified based on the requirements of your upstream device.
   
   
9. The QLogic 8 Gigabit CIOv adapter cannot have mixed switch vendors in the chassis switch bay-3 and bay-4.
   
   
10. An IBM BladeCenter H chassis configured with midplane option part number 46C9700 requires a Brocade firmware level of 6.4.2b4.