Ubuntu 16.04 and Brocade 1010/1020/1007/1741 CNA: task bfad_worker:952 blocked (incompatible firmware)

by

After upgrading Ubuntu LTS 14.04 to 16.04 system stopped booting and proposed entering maintenance shell to solve a problem. There are two problem indicators in logs:

[  721.242651] INFO: task bfad_worker:952 blocked for more than 120 seconds.
[  721.272889]       Not tainted 4.4.0-31-generic #50-Ubuntu
[  721.302231] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  721.332218] bfad_worker     D ffff880c45767ea8     0   952      2 0x00000000 
[  721.332220]  ffff880c45767ea8 ffff8818443939c0 ffff881848f5cb00 ffff880c43d59900
[  721.332222]  ffff880c45768000 ffff881842780000 ffffffffc00df450 0000000000000000
[  721.332223]  0000000000000000 ffff880c45767ec0 ffffffff81829a25 ffff880c48aa5b40
[  721.332225] Call Trace:
[  721.332231]  [<ffffffffc00df450>] ? bfad_read_firmware+0xd0/0xd0 [bfa]
[  721.332233]  [<ffffffff81829a25>] schedule+0x35/0x80
[  721.332235]  [<ffffffff810a07e3>] kthread+0xb3/0xf0
[  721.332236]  [<ffffffff810a0730>] ? kthread_create_on_node+0x1e0/0x1e0
[  721.332238]  [<ffffffff8182decf>] ret_from_fork+0x3f/0x70
[  721.332239]  [<ffffffff810a0730>] ? kthread_create_on_node+0x1e0/0x1e0

and (the more important):

[    6.070145] bfa 0000:15:00.0: Running firmware version is incompatible with the driver version
[    6.295256] bfa 0000:15:00.0: bfa init failed

According to this https://bugzilla.kernel.org/show_bug.cgi?id=104191 bug report, bfa driver loads incorrect (old) firmware. Simple diagnostic command confirms it:

# modinfo bfa
filename:       /lib/modules/4.4.0-31-generic/kernel/drivers/scsi/bfa/bfa.ko
version:        3.2.23.0
author:         Brocade Communications Systems, Inc.
description:    Brocade Fibre Channel HBA Driver fcpim
license:        GPL
firmware:       ct2fw-3.2.3.0.bin
firmware:       ctfw-3.2.3.0.bin
firmware:       cbfw-3.2.3.0.bin
[...]

So it’s version 3.2.3.0 instead of 3.2.5.1. The fast workaround for it could be:

  modprobe -r bfa
  cd /lib/firmware
  mv ct2fw-3.2.3.0.bin ct2fw-3.2.3.0.bin-ORIG
  mv ctfw-3.2.3.0.bin ctfw-3.2.3.0.bin-ORIG
  mv cbfw-3.2.3.0.bin cbfw-3.2.3.0.bin-ORIG
  ln -s ct2fw-3.2.5.1.bin ct2fw-3.2.3.0.bin 
  ln -s ctfw-3.2.5.1.bin ctfw-3.2.3.0.bin 
  ln -s cbfw-3.2.5.1.bin cbfw-3.2.3.0.bin
  modprobe bfa
  update-initramfs -u

2 thoughts on “Ubuntu 16.04 and Brocade 1010/1020/1007/1741 CNA: task bfad_worker:952 blocked (incompatible firmware)”

  1. This fixed the problem indeed. I confirm that this also works on ubuntu 16.10.

    Be aware that you have to redo this fix each time the linux-firmware package gets updated. You can avoid this by setting the package to hold (sudo apt-mark hold linux-firmware)

    This is the kernel bug for this issue. Not much has happened.
    https://bugzilla.kernel.org/show_bug.cgi?id=104191

Comments are closed.