Quantcast
Channel: Intel Communities: Message List - Wired Ethernet
Viewing all articles
Browse latest Browse all 9952

Re: 82599ES SR-IOV on Ubuntu 14.04 - VF behavior

$
0
0

Hi Patrick,

 

Thanks so much for your reply.  I hope you enjoyed your vacation time and I'm really sorry you have to deal with me upon your return!

 

So, I wouldn't normally assign MAC addresses to the VFs, I only went that route because of the random behavior of the VFs after starting the VM that the hostdev interfaces were assigned to.

 

For this reply, I changed my guest from FreeBSD to CentOS to perhaps simplify things but what I was originally seeing and the reason I even set them was because I was arbitrarily getting an all zeros MAC randomly on the VMs interfaces.

 

First, in response to your recommended steps.

 

1)  ixgbevf is being blacklisted from within /etc/modprobe.d/blacklist.conf

          blacklist ixgbevf

2)  I am loading ixgbe and VFs from within /etc/modprobe.d/ixgbe.conf

          options ixgbe max_vfs=16

          options loop max_loop=128

3)  Now begging with my new CentOS VM, I have not assigned MAC addresses to the VFs.

     root@ubuntu:~# ip link show dev eth4

     5: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000

    link/ether 90:e2:ba:47:2c:30 brd ff:ff:ff:ff:ff:ff

    vf 0 MAC 00:00:00:00:00:00, vlan 100, spoof checking on, link-state auto

    vf 1 MAC 00:00:00:00:00:00, vlan 200, spoof checking on, link-state auto

    vf 2 MAC 00:00:00:00:00:00, vlan 300, spoof checking on, link-state auto

    vf 3 MAC 00:00:00:00:00:00, vlan 100, spoof checking on, link-state auto

    vf 4 MAC 00:00:00:00:00:00, vlan 200, spoof checking on, link-state auto

    vf 5 MAC 00:00:00:00:00:00, vlan 300, spoof checking on, link-state auto

    vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 7 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 8 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 9 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 10 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 11 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 12 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 13 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 14 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 15 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

 

4)  VLANs are assigned (see output from 3)

 

5)  Here is the applicable snippet from my guest's XML configuration

<interface type='hostdev' managed='yes'>
  <mac address='52:54:00:58:5f:0c'/>
  <source>
    <address type='pci' domain='0x0000' bus='0x81' slot='0x10' function='0x0'/>
  </source>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
<interface type='hostdev' managed='yes'>
  <mac address='52:54:00:d4:63:a7'/>
  <source>
    <address type='pci' domain='0x0000' bus='0x81' slot='0x10' function='0x1'/>
  </source>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</interface>
<interface type='hostdev' managed='yes'>
  <mac address='52:54:00:51:5c:0a'/>
  <source>
    <address type='pci' domain='0x0000' bus='0x81' slot='0x10' function='0x2'/>
  </source>
  <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</interface>

 

6)  And the result from starting the guest which still reflects bizarre behavior.  Notice from the log output below that a MAC was set to VF0 two times, a MAC was set for VF1 but VF2 was never set.

 

     root@ubuntu:~# ip link show dev eth4

     5: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000

    link/ether 90:e2:ba:47:2c:30 brd ff:ff:ff:ff:ff:ff

    vf 0 MAC 52:54:00:58:5f:0c, spoof checking on, link-state auto

    vf 1 MAC 52:54:00:51:5c:0a, spoof checking on, link-state auto

    vf 2 MAC 00:00:00:00:00:00, vlan 300, spoof checking on, link-state auto

    vf 3 MAC 00:00:00:00:00:00, vlan 100, spoof checking on, link-state auto

    vf 4 MAC 00:00:00:00:00:00, vlan 200, spoof checking on, link-state auto

    vf 5 MAC 00:00:00:00:00:00, vlan 300, spoof checking on, link-state auto

    vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 7 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 8 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 9 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 10 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 11 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 12 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 13 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 14 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

    vf 15 MAC 00:00:00:00:00:00, spoof checking on, link-state auto

 

Jul 22 12:53:40 ubuntu kernel: [ 3138.012846] pci-stub 0000:81:10.0: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.012936] pci-stub 0000:81:10.2: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.012951] pci-stub 0000:81:10.4: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.012972] pci-stub 0000:81:10.6: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.012988] pci-stub 0000:81:11.0: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013002] pci-stub 0000:81:11.2: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013017] pci-stub 0000:81:11.4: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013031] pci-stub 0000:81:11.6: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013045] pci-stub 0000:81:12.0: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013057] pci-stub 0000:81:12.2: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013068] pci-stub 0000:81:12.4: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013080] pci-stub 0000:81:12.6: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013091] pci-stub 0000:81:13.0: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013102] pci-stub 0000:81:13.2: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013113] pci-stub 0000:81:13.4: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013125] pci-stub 0000:81:13.6: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013136] pci-stub 0000:81:10.1: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013149] pci-stub 0000:81:10.3: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013166] pci-stub 0000:81:10.5: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013178] pci-stub 0000:81:10.7: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013190] pci-stub 0000:81:11.1: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013201] pci-stub 0000:81:11.3: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013213] pci-stub 0000:81:11.5: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013224] pci-stub 0000:81:11.7: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013235] pci-stub 0000:81:12.1: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013246] pci-stub 0000:81:12.3: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013258] pci-stub 0000:81:12.5: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013269] pci-stub 0000:81:12.7: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013280] pci-stub 0000:81:13.1: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013291] pci-stub 0000:81:13.3: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013302] pci-stub 0000:81:13.5: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.013314] pci-stub 0000:81:13.7: claimed by stub

Jul 22 12:53:40 ubuntu kernel: [ 3138.033103] ixgbe 0000:81:00.0: setting MAC 52:54:00:58:5f:0c on VF 0

Jul 22 12:53:40 ubuntu kernel: [ 3138.033109] ixgbe 0000:81:00.0: Reload the VF driver to make this change effective.

Jul 22 12:53:40 ubuntu kernel: [ 3138.033112] ixgbe 0000:81:00.0: The VF MAC address has been set, but the PF device is not up.

Jul 22 12:53:40 ubuntu kernel: [ 3138.033115] ixgbe 0000:81:00.0: Bring the PF device up before attempting to use the VF device.

Jul 22 12:53:40 ubuntu kernel: [ 3138.045072] ixgbe 0000:81:00.1: setting MAC 52:54:00:d4:63:a7 on VF 0

Jul 22 12:53:40 ubuntu kernel: [ 3138.045078] ixgbe 0000:81:00.1: Reload the VF driver to make this change effective.

Jul 22 12:53:40 ubuntu kernel: [ 3138.045081] ixgbe 0000:81:00.1: The VF MAC address has been set, but the PF device is not up.

Jul 22 12:53:40 ubuntu kernel: [ 3138.045084] ixgbe 0000:81:00.1: Bring the PF device up before attempting to use the VF device.

Jul 22 12:53:40 ubuntu kernel: [ 3138.053024] ixgbe 0000:81:00.0: setting MAC 52:54:00:51:5c:0a on VF 1

Jul 22 12:53:40 ubuntu kernel: [ 3138.053030] ixgbe 0000:81:00.0: Reload the VF driver to make this change effective.

Jul 22 12:53:40 ubuntu kernel: [ 3138.053033] ixgbe 0000:81:00.0: The VF MAC address has been set, but the PF device is not up.

Jul 22 12:53:40 ubuntu kernel: [ 3138.053036] ixgbe 0000:81:00.0: Bring the PF device up before attempting to use the VF device.

Jul 22 12:53:40 ubuntu kernel: [ 3138.305853] type=1400 audit(1406058820.760:42): apparmor="STATUS" operation="profile_load" profile="unconfined" name="libvirt-d3724bdc-fc97-37d0-437c-33f5d54fb811" pid=2795 comm="apparmor_parser"

Jul 22 12:53:40 ubuntu kernel: [ 3138.419792] pci-stub 0000:81:10.0: enabling device (0000 -> 0002)

Jul 22 12:53:42 ubuntu kernel: [ 3139.873334] pci-stub 0000:81:10.0: kvm assign device

Jul 22 12:53:42 ubuntu kernel: [ 3139.874950] pci-stub 0000:81:10.1: enabling device (0000 -> 0002)

Jul 22 12:53:42 ubuntu kernel: [ 3139.978660] pci-stub 0000:81:10.1: kvm assign device

Jul 22 12:53:42 ubuntu kernel: [ 3139.981161] pci-stub 0000:81:10.2: enabling device (0000 -> 0002)

Jul 22 12:53:42 ubuntu kernel: [ 3140.082788] pci-stub 0000:81:10.2: kvm assign device

Jul 22 12:53:49 ubuntu kernel: [ 3146.897520] kvm [2797]: vcpu0 disabled perfctr wrmsr: 0xc1 data 0xabcd

Jul 22 12:53:49 ubuntu kernel: [ 3146.967250] kvm: zapping shadow pages for mmio generation wraparound

Jul 22 12:53:51 ubuntu kernel: [ 3149.537229] pci-stub 0000:81:10.0: irq 193 for MSI/MSI-X

Jul 22 12:53:51 ubuntu kernel: [ 3149.537245] pci-stub 0000:81:10.0: irq 194 for MSI/MSI-X

Jul 22 12:53:52 ubuntu kernel: [ 3149.557331] pci-stub 0000:81:10.0: irq 193 for MSI/MSI-X

Jul 22 12:53:52 ubuntu kernel: [ 3149.557372] pci-stub 0000:81:10.0: irq 194 for MSI/MSI-X

Jul 22 12:53:52 ubuntu kernel: [ 3149.577287] pci-stub 0000:81:10.1: irq 195 for MSI/MSI-X

Jul 22 12:53:52 ubuntu kernel: [ 3149.577329] pci-stub 0000:81:10.1: irq 196 for MSI/MSI-X

Jul 22 12:53:52 ubuntu kernel: [ 3149.605431] pci-stub 0000:81:10.1: irq 195 for MSI/MSI-X

Jul 22 12:53:52 ubuntu kernel: [ 3149.605471] pci-stub 0000:81:10.1: irq 196 for MSI/MSI-X

Jul 22 12:53:52 ubuntu kernel: [ 3149.621296] pci-stub 0000:81:10.2: irq 197 for MSI/MSI-X

Jul 22 12:53:52 ubuntu kernel: [ 3149.621327] pci-stub 0000:81:10.2: irq 198 for MSI/MSI-X

Jul 22 12:53:52 ubuntu kernel: [ 3149.645417] pci-stub 0000:81:10.2: irq 197 for MSI/MSI-X

Jul 22 12:53:52 ubuntu kernel: [ 3149.645458] pci-stub 0000:81:10.2: irq 198 for MSI/MSI-X

 

I'm not ruling out yet that I could be doing something wrong.

 

Thanks for your assistance.

 

JC


Viewing all articles
Browse latest Browse all 9952

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>