Xenial SRU - [Hyper-V] Missing PCI patches breaking SR-IOV hot remove

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Xenial SRU - [Hyper-V] Missing PCI patches breaking SR-IOV hot remove

Tim Gardner-2
These are the same clean cherry-picks as Yakkety, but with slightly
different context, hence the second set.

https://bugs.launchpad.net/bugs/1670518

[PATCH 1/4] PCI: hv: Fix hv_pci_remove() for hot-remove
[PATCH 2/4] PCI: hv: Delete the device earlier from hbus->children
[PATCH 3/4] PCI: hv: Make unnecessarily global IRQ masking functions
[PATCH 4/4] PCI: hv: Allocate physically contiguous hypercall params

rtg

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[PATCH 1/4] PCI: hv: Fix hv_pci_remove() for hot-remove

Tim Gardner-2
From: Dexuan Cui <[hidden email]>

BugLink: http://bugs.launchpad.net/bugs/1670518

1. We don't really need such a big on-stack buffer when sending the
teardown_packet: vmbus_sendpacket() here only uses sizeof(struct
pci_message).

2. In the hot-remove case (PCI_EJECT), after we send PCI_EJECTION_COMPLETE
to the host, the host will send a RESCIND_CHANNEL message to us and the
host won't access the per-channel ringbuffer any longer, so we needn't send
PCI_RESOURCES_RELEASED/PCI_BUS_D0EXIT to the host, and we shouldn't expect
the host's completion message of PCI_BUS_D0EXIT, which will never come.

3. We should send PCI_BUS_D0EXIT after hv_send_resources_released().

Signed-off-by: Dexuan Cui <[hidden email]>
Signed-off-by: Bjorn Helgaas <[hidden email]>
Reviewed-by: Jake Oshins <[hidden email]>
Acked-by: K. Y. Srinivasan <[hidden email]>
CC: Haiyang Zhang <[hidden email]>
CC: Vitaly Kuznetsov <[hidden email]>
(cherry picked from commit 17978524a636d007e6b929304ae3eb5ea0371019)
Signed-off-by: Tim Gardner <[hidden email]>
---
 drivers/pci/host/pci-hyperv.c | 53 +++++++++++++++++++++++++++----------------
 1 file changed, 33 insertions(+), 20 deletions(-)

diff --git a/drivers/pci/host/pci-hyperv.c b/drivers/pci/host/pci-hyperv.c
index 2bbfdfa..a290e79 100644
--- a/drivers/pci/host/pci-hyperv.c
+++ b/drivers/pci/host/pci-hyperv.c
@@ -2285,24 +2285,32 @@ free_bus:
  return ret;
 }
 
-/**
- * hv_pci_remove() - Remove routine for this VMBus channel
- * @hdev: VMBus's tracking struct for this root PCI bus
- *
- * Return: 0 on success, -errno on failure
- */
-static int hv_pci_remove(struct hv_device *hdev)
+static void hv_pci_bus_exit(struct hv_device *hdev)
 {
- int ret;
- struct hv_pcibus_device *hbus;
- union {
+ struct hv_pcibus_device *hbus = hv_get_drvdata(hdev);
+ struct {
  struct pci_packet teardown_packet;
- u8 buffer[0x100];
+ u8 buffer[sizeof(struct pci_message)];
  } pkt;
  struct pci_bus_relations relations;
  struct hv_pci_compl comp_pkt;
+ int ret;
 
- hbus = hv_get_drvdata(hdev);
+ /*
+ * After the host sends the RESCIND_CHANNEL message, it doesn't
+ * access the per-channel ringbuffer any longer.
+ */
+ if (hdev->channel->rescind)
+ return;
+
+ /* Delete any children which might still exist. */
+ memset(&relations, 0, sizeof(relations));
+ hv_pci_devices_present(hbus, &relations);
+
+ ret = hv_send_resources_released(hdev);
+ if (ret)
+ dev_err(&hdev->device,
+ "Couldn't send resources released packet(s)\n");
 
  memset(&pkt.teardown_packet, 0, sizeof(pkt.teardown_packet));
  init_completion(&comp_pkt.host_event);
@@ -2317,7 +2325,19 @@ static int hv_pci_remove(struct hv_device *hdev)
        VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
  if (!ret)
  wait_for_completion_timeout(&comp_pkt.host_event, 10 * HZ);
+}
 
+/**
+ * hv_pci_remove() - Remove routine for this VMBus channel
+ * @hdev: VMBus's tracking struct for this root PCI bus
+ *
+ * Return: 0 on success, -errno on failure
+ */
+static int hv_pci_remove(struct hv_device *hdev)
+{
+ struct hv_pcibus_device *hbus;
+
+ hbus = hv_get_drvdata(hdev);
  if (hbus->state == hv_pcibus_installed) {
  /* Remove the bus from PCI's point of view. */
  pci_lock_rescan_remove();
@@ -2327,17 +2347,10 @@ static int hv_pci_remove(struct hv_device *hdev)
  hbus->state = hv_pcibus_removed;
  }
 
- ret = hv_send_resources_released(hdev);
- if (ret)
- dev_err(&hdev->device,
- "Couldn't send resources released packet(s)\n");
+ hv_pci_bus_exit(hdev);
 
  vmbus_close(hdev->channel);
 
- /* Delete any children which might still exist. */
- memset(&relations, 0, sizeof(relations));
- hv_pci_devices_present(hbus, &relations);
-
  iounmap(hbus->cfg_addr);
  hv_free_config_window(hbus);
  pci_free_resource_list(&hbus->resources_for_children);
--
2.7.4


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[PATCH 2/4] PCI: hv: Delete the device earlier from hbus->children for hot-remove

Tim Gardner-2
In reply to this post by Tim Gardner-2
From: Dexuan Cui <[hidden email]>

BugLink: http://bugs.launchpad.net/bugs/1670518

After we send a PCI_EJECTION_COMPLETE message to the host, the host will
immediately send us a PCI_BUS_RELATIONS message with
relations->device_count == 0, so pci_devices_present_work(), running on
another thread, can find the being-ejected device, mark the
hpdev->reported_missing to true, and run list_move_tail()/list_del() for
the device -- this races hv_eject_device_work() -> list_del().

Move the list_del() in hv_eject_device_work() to an earlier place, i.e.,
before we send PCI_EJECTION_COMPLETE, so later the
pci_devices_present_work() can't see the device.

Signed-off-by: Dexuan Cui <[hidden email]>
Signed-off-by: Bjorn Helgaas <[hidden email]>
Reviewed-by: Jake Oshins <[hidden email]>
Acked-by: K. Y. Srinivasan <[hidden email]>
CC: Haiyang Zhang <[hidden email]>
CC: Vitaly Kuznetsov <[hidden email]>
(cherry picked from commit e74d2ebdda33b3bdd1826b5b92e9aa45bdf92bb3)
Signed-off-by: Tim Gardner <[hidden email]>
---
 drivers/pci/host/pci-hyperv.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/host/pci-hyperv.c b/drivers/pci/host/pci-hyperv.c
index a290e79..10bc3a0 100644
--- a/drivers/pci/host/pci-hyperv.c
+++ b/drivers/pci/host/pci-hyperv.c
@@ -1600,6 +1600,10 @@ static void hv_eject_device_work(struct work_struct *work)
  pci_unlock_rescan_remove();
  }
 
+ spin_lock_irqsave(&hpdev->hbus->device_list_lock, flags);
+ list_del(&hpdev->list_entry);
+ spin_unlock_irqrestore(&hpdev->hbus->device_list_lock, flags);
+
  memset(&ctxt, 0, sizeof(ctxt));
  ejct_pkt = (struct pci_eject_response *)&ctxt.pkt.message;
  ejct_pkt->message_type.type = PCI_EJECTION_COMPLETE;
@@ -1608,10 +1612,6 @@ static void hv_eject_device_work(struct work_struct *work)
  sizeof(*ejct_pkt), (unsigned long)&ctxt.pkt,
  VM_PKT_DATA_INBAND, 0);
 
- spin_lock_irqsave(&hpdev->hbus->device_list_lock, flags);
- list_del(&hpdev->list_entry);
- spin_unlock_irqrestore(&hpdev->hbus->device_list_lock, flags);
-
  put_pcichild(hpdev, hv_pcidev_ref_childlist);
  put_pcichild(hpdev, hv_pcidev_ref_pnp);
  put_hvpcibus(hpdev->hbus);
--
2.7.4


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[PATCH 3/4] PCI: hv: Make unnecessarily global IRQ masking functions static

Tim Gardner-2
In reply to this post by Tim Gardner-2
From: Tobias Klauser <[hidden email]>

BugLink: http://bugs.launchpad.net/bugs/1670518

Make hv_irq_mask() and hv_irq_unmask() static as they are only used in
pci-hyperv.c

This fixes a sparse warning.

Signed-off-by: Tobias Klauser <[hidden email]>
Signed-off-by: Bjorn Helgaas <[hidden email]>
Acked-by: K. Y. Srinivasan <[hidden email]>
(cherry picked from commit 542ccf4551fa019a8ae9dfb7c8cd7e73a3d7e614)
Signed-off-by: Tim Gardner <[hidden email]>
---
 drivers/pci/host/pci-hyperv.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/host/pci-hyperv.c b/drivers/pci/host/pci-hyperv.c
index 10bc3a0..7ff8a48 100644
--- a/drivers/pci/host/pci-hyperv.c
+++ b/drivers/pci/host/pci-hyperv.c
@@ -758,7 +758,7 @@ static int hv_set_affinity(struct irq_data *data, const struct cpumask *dest,
  return parent->chip->irq_set_affinity(parent, dest, force);
 }
 
-void hv_irq_mask(struct irq_data *data)
+static void hv_irq_mask(struct irq_data *data)
 {
  pci_msi_mask_irq(data);
 }
@@ -773,7 +773,7 @@ void hv_irq_mask(struct irq_data *data)
  * is built out of this PCI bus's instance GUID and the function
  * number of the device.
  */
-void hv_irq_unmask(struct irq_data *data)
+static void hv_irq_unmask(struct irq_data *data)
 {
  struct msi_desc *msi_desc = irq_data_get_msi_desc(data);
  struct irq_cfg *cfg = irqd_cfg(data);
--
2.7.4


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[PATCH 4/4] PCI: hv: Allocate physically contiguous hypercall params buffer

Tim Gardner-2
In reply to this post by Tim Gardner-2
From: Long Li <[hidden email]>

BugLink: http://bugs.launchpad.net/bugs/1670518

hv_do_hypercall() assumes that we pass a segment from a physically
contiguous buffer.  A buffer allocated on the stack may not work if
CONFIG_VMAP_STACK=y is set.

Use kmalloc() to allocate this buffer.

Reported-by: Haiyang Zhang <[hidden email]>
Signed-off-by: Long Li <[hidden email]>
Signed-off-by: Bjorn Helgaas <[hidden email]>
Acked-by: K. Y. Srinivasan <[hidden email]>
(cherry picked from commit 0de8ce3ee8e38cc66683438f715c79a2cc69539e)
Signed-off-by: Tim Gardner <[hidden email]>
---
 drivers/pci/host/pci-hyperv.c | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/drivers/pci/host/pci-hyperv.c b/drivers/pci/host/pci-hyperv.c
index 7ff8a48..166e217 100644
--- a/drivers/pci/host/pci-hyperv.c
+++ b/drivers/pci/host/pci-hyperv.c
@@ -380,6 +380,8 @@ struct hv_pcibus_device {
  struct msi_domain_info msi_info;
  struct msi_controller msi_chip;
  struct irq_domain *irq_domain;
+ struct retarget_msi_interrupt retarget_msi_interrupt_params;
+ spinlock_t retarget_msi_interrupt_lock;
 };
 
 /*
@@ -777,34 +779,40 @@ static void hv_irq_unmask(struct irq_data *data)
 {
  struct msi_desc *msi_desc = irq_data_get_msi_desc(data);
  struct irq_cfg *cfg = irqd_cfg(data);
- struct retarget_msi_interrupt params;
+ struct retarget_msi_interrupt *params;
  struct hv_pcibus_device *hbus;
  struct cpumask *dest;
  struct pci_bus *pbus;
  struct pci_dev *pdev;
  int cpu;
+ unsigned long flags;
 
  dest = irq_data_get_affinity_mask(data);
  pdev = msi_desc_to_pci_dev(msi_desc);
  pbus = pdev->bus;
  hbus = container_of(pbus->sysdata, struct hv_pcibus_device, sysdata);
 
- memset(&params, 0, sizeof(params));
- params.partition_id = HV_PARTITION_ID_SELF;
- params.source = 1; /* MSI(-X) */
- params.address = msi_desc->msg.address_lo;
- params.data = msi_desc->msg.data;
- params.device_id = (hbus->hdev->dev_instance.b[5] << 24) |
+ spin_lock_irqsave(&hbus->retarget_msi_interrupt_lock, flags);
+
+ params = &hbus->retarget_msi_interrupt_params;
+ memset(params, 0, sizeof(*params));
+ params->partition_id = HV_PARTITION_ID_SELF;
+ params->source = 1; /* MSI(-X) */
+ params->address = msi_desc->msg.address_lo;
+ params->data = msi_desc->msg.data;
+ params->device_id = (hbus->hdev->dev_instance.b[5] << 24) |
    (hbus->hdev->dev_instance.b[4] << 16) |
    (hbus->hdev->dev_instance.b[7] << 8) |
    (hbus->hdev->dev_instance.b[6] & 0xf8) |
    PCI_FUNC(pdev->devfn);
- params.vector = cfg->vector;
+ params->vector = cfg->vector;
 
  for_each_cpu_and(cpu, dest, cpu_online_mask)
- params.vp_mask |= (1ULL << vmbus_cpu_number_to_vp_number(cpu));
+ params->vp_mask |= (1ULL << vmbus_cpu_number_to_vp_number(cpu));
+
+ hv_do_hypercall(HVCALL_RETARGET_INTERRUPT, params, NULL);
 
- hv_do_hypercall(HVCALL_RETARGET_INTERRUPT, &params, NULL);
+ spin_unlock_irqrestore(&hbus->retarget_msi_interrupt_lock, flags);
 
  pci_msi_unmask_irq(data);
 }
@@ -2205,6 +2213,7 @@ static int hv_pci_probe(struct hv_device *hdev,
  INIT_LIST_HEAD(&hbus->resources_for_children);
  spin_lock_init(&hbus->config_lock);
  spin_lock_init(&hbus->device_list_lock);
+ spin_lock_init(&hbus->retarget_msi_interrupt_lock);
  sema_init(&hbus->enum_sem, 1);
  init_completion(&hbus->remove_event);
 
--
2.7.4


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

ACK: Xenial SRU - [Hyper-V] Missing PCI patches breaking SR-IOV hot remove

Marcelo Henrique Cerri
In reply to this post by Tim Gardner-2
--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team

signature.asc (484 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

ACK: Xenial SRU - [Hyper-V] Missing PCI patches breaking SR-IOV hot remove

Joseph Salisbury-3
In reply to this post by Tim Gardner-2
We now have good testing results from Microsoft.

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

Applied: Xenial SRU - [Hyper-V] Missing PCI patches breaking SR-IOV hot remove

brad.figg
In reply to this post by Tim Gardner-2

Applied to the master-next branch of Xenial

--
Brad Figg [hidden email] http://www.canonical.com

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team