[d/azure][PATCH 0/2] LP: #1837661 - [linux-azure] CRI-RDOS | Live migration only takes 10 seconds, but the VM was unavailable for 2 hours

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[d/azure][PATCH 0/2] LP: #1837661 - [linux-azure] CRI-RDOS | Live migration only takes 10 seconds, but the VM was unavailable for 2 hours

Marcelo Henrique Cerri
BugLink: https://bugs.launchpad.net/bugs/1837661

It's still not clear if the fixes are really needed for 4.15. So I'm
submitting the fixes for 5.0 so it can make into the next SRU
cycle. The fixes for 4.15 will be handled separately.

Dexuan Cui (2):
  PCI: hv: Fix a use-after-free bug in hv_eject_device_work()
  UBUNTU: SAUCE: PCI: hv: Fix panic by calling hv_pci_remove_slots()
    earlier

 drivers/pci/controller/pci-hyperv.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

--
2.20.1


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[d/azure][PATCH 1/2] PCI: hv: Fix a use-after-free bug in hv_eject_device_work()

Marcelo Henrique Cerri
From: Dexuan Cui <[hidden email]>

BugLink: https://bugs.launchpad.net/bugs/1837661

Fix a use-after-free in hv_eject_device_work().

Fixes: 05f151a73ec2 ("PCI: hv: Fix a memory leak in hv_eject_device_work()")
Signed-off-by: Dexuan Cui <[hidden email]>
Signed-off-by: Lorenzo Pieralisi <[hidden email]>
Reviewed-by: Michael Kelley <[hidden email]>
Cc: [hidden email]
(cherry picked from commit 4df591b20b80cb77920953812d894db259d85bd7)
Signed-off-by: Marcelo Henrique Cerri <[hidden email]>
---
 drivers/pci/controller/pci-hyperv.c | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index 42af7f6a7c4c..ae91e708796f 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -1888,6 +1888,7 @@ static void hv_pci_devices_present(struct hv_pcibus_device *hbus,
 static void hv_eject_device_work(struct work_struct *work)
 {
  struct pci_eject_response *ejct_pkt;
+ struct hv_pcibus_device *hbus;
  struct hv_pci_dev *hpdev;
  struct pci_dev *pdev;
  unsigned long flags;
@@ -1898,6 +1899,7 @@ static void hv_eject_device_work(struct work_struct *work)
  } ctxt;
 
  hpdev = container_of(work, struct hv_pci_dev, wrk);
+ hbus = hpdev->hbus;
 
  WARN_ON(hpdev->state != hv_pcichild_ejecting);
 
@@ -1908,8 +1910,7 @@ static void hv_eject_device_work(struct work_struct *work)
  * because hbus->pci_bus may not exist yet.
  */
  wslot = wslot_to_devfn(hpdev->desc.win_slot.slot);
- pdev = pci_get_domain_bus_and_slot(hpdev->hbus->sysdata.domain, 0,
-   wslot);
+ pdev = pci_get_domain_bus_and_slot(hbus->sysdata.domain, 0, wslot);
  if (pdev) {
  pci_lock_rescan_remove();
  pci_stop_and_remove_bus_device(pdev);
@@ -1917,9 +1918,9 @@ static void hv_eject_device_work(struct work_struct *work)
  pci_unlock_rescan_remove();
  }
 
- spin_lock_irqsave(&hpdev->hbus->device_list_lock, flags);
+ spin_lock_irqsave(&hbus->device_list_lock, flags);
  list_del(&hpdev->list_entry);
- spin_unlock_irqrestore(&hpdev->hbus->device_list_lock, flags);
+ spin_unlock_irqrestore(&hbus->device_list_lock, flags);
 
  if (hpdev->pci_slot)
  pci_destroy_slot(hpdev->pci_slot);
@@ -1928,7 +1929,7 @@ static void hv_eject_device_work(struct work_struct *work)
  ejct_pkt = (struct pci_eject_response *)&ctxt.pkt.message;
  ejct_pkt->message_type.type = PCI_EJECTION_COMPLETE;
  ejct_pkt->wslot.slot = hpdev->desc.win_slot.slot;
- vmbus_sendpacket(hpdev->hbus->hdev->channel, ejct_pkt,
+ vmbus_sendpacket(hbus->hdev->channel, ejct_pkt,
  sizeof(*ejct_pkt), (unsigned long)&ctxt.pkt,
  VM_PKT_DATA_INBAND, 0);
 
@@ -1937,7 +1938,9 @@ static void hv_eject_device_work(struct work_struct *work)
  /* For the two refs got in new_pcichild_device() */
  put_pcichild(hpdev);
  put_pcichild(hpdev);
- put_hvpcibus(hpdev->hbus);
+ /* hpdev has been freed. Do not use it any more. */
+
+ put_hvpcibus(hbus);
 }
 
 /**
--
2.20.1


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

[d/azure][PATCH 2/2] UBUNTU: SAUCE: PCI: hv: Fix panic by calling hv_pci_remove_slots() earlier

Marcelo Henrique Cerri
In reply to this post by Marcelo Henrique Cerri
From: Dexuan Cui <[hidden email]>

BugLink: https://bugs.launchpad.net/bugs/1837661

The slot must be removed before the pci_dev is removed, otherwise a panic
can happen due to use-after-free.

Fixes: 15becc2b56c6 ("PCI: hv: Add hv_pci_remove_slots() when we unload the driver")
Signed-off-by: Dexuan Cui <[hidden email]>
Cc: [hidden email]
Signed-off-by: Marcelo Henrique Cerri <[hidden email]>
---
 drivers/pci/controller/pci-hyperv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index ae91e708796f..3bd94149b9d2 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -2714,8 +2714,8 @@ static int hv_pci_remove(struct hv_device *hdev)
  /* Remove the bus from PCI's point of view. */
  pci_lock_rescan_remove();
  pci_stop_root_bus(hbus->pci_bus);
- pci_remove_root_bus(hbus->pci_bus);
  hv_pci_remove_slots(hbus);
+ pci_remove_root_bus(hbus->pci_bus);
  pci_unlock_rescan_remove();
  hbus->state = hv_pcibus_removed;
  }
--
2.20.1


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

ACK: [d/azure][PATCH 1/2] PCI: hv: Fix a use-after-free bug in hv_eject_device_work()

Sultan Alsawaf
In reply to this post by Marcelo Henrique Cerri
On Thu, Aug 08, 2019 at 12:34:52PM -0300, Marcelo Henrique Cerri wrote:

> From: Dexuan Cui <[hidden email]>
>
> BugLink: https://bugs.launchpad.net/bugs/1837661
>
> Fix a use-after-free in hv_eject_device_work().
>
> Fixes: 05f151a73ec2 ("PCI: hv: Fix a memory leak in hv_eject_device_work()")
> Signed-off-by: Dexuan Cui <[hidden email]>
> Signed-off-by: Lorenzo Pieralisi <[hidden email]>
> Reviewed-by: Michael Kelley <[hidden email]>
> Cc: [hidden email]
> (cherry picked from commit 4df591b20b80cb77920953812d894db259d85bd7)
> Signed-off-by: Marcelo Henrique Cerri <[hidden email]>
> ---
>  drivers/pci/controller/pci-hyperv.c | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index 42af7f6a7c4c..ae91e708796f 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -1888,6 +1888,7 @@ static void hv_pci_devices_present(struct hv_pcibus_device *hbus,
>  static void hv_eject_device_work(struct work_struct *work)
>  {
>   struct pci_eject_response *ejct_pkt;
> + struct hv_pcibus_device *hbus;
>   struct hv_pci_dev *hpdev;
>   struct pci_dev *pdev;
>   unsigned long flags;
> @@ -1898,6 +1899,7 @@ static void hv_eject_device_work(struct work_struct *work)
>   } ctxt;
>  
>   hpdev = container_of(work, struct hv_pci_dev, wrk);
> + hbus = hpdev->hbus;
>  
>   WARN_ON(hpdev->state != hv_pcichild_ejecting);
>  
> @@ -1908,8 +1910,7 @@ static void hv_eject_device_work(struct work_struct *work)
>   * because hbus->pci_bus may not exist yet.
>   */
>   wslot = wslot_to_devfn(hpdev->desc.win_slot.slot);
> - pdev = pci_get_domain_bus_and_slot(hpdev->hbus->sysdata.domain, 0,
> -   wslot);
> + pdev = pci_get_domain_bus_and_slot(hbus->sysdata.domain, 0, wslot);
>   if (pdev) {
>   pci_lock_rescan_remove();
>   pci_stop_and_remove_bus_device(pdev);
> @@ -1917,9 +1918,9 @@ static void hv_eject_device_work(struct work_struct *work)
>   pci_unlock_rescan_remove();
>   }
>  
> - spin_lock_irqsave(&hpdev->hbus->device_list_lock, flags);
> + spin_lock_irqsave(&hbus->device_list_lock, flags);
>   list_del(&hpdev->list_entry);
> - spin_unlock_irqrestore(&hpdev->hbus->device_list_lock, flags);
> + spin_unlock_irqrestore(&hbus->device_list_lock, flags);
>  
>   if (hpdev->pci_slot)
>   pci_destroy_slot(hpdev->pci_slot);
> @@ -1928,7 +1929,7 @@ static void hv_eject_device_work(struct work_struct *work)
>   ejct_pkt = (struct pci_eject_response *)&ctxt.pkt.message;
>   ejct_pkt->message_type.type = PCI_EJECTION_COMPLETE;
>   ejct_pkt->wslot.slot = hpdev->desc.win_slot.slot;
> - vmbus_sendpacket(hpdev->hbus->hdev->channel, ejct_pkt,
> + vmbus_sendpacket(hbus->hdev->channel, ejct_pkt,
>   sizeof(*ejct_pkt), (unsigned long)&ctxt.pkt,
>   VM_PKT_DATA_INBAND, 0);
>  
> @@ -1937,7 +1938,9 @@ static void hv_eject_device_work(struct work_struct *work)
>   /* For the two refs got in new_pcichild_device() */
>   put_pcichild(hpdev);
>   put_pcichild(hpdev);
> - put_hvpcibus(hpdev->hbus);
> + /* hpdev has been freed. Do not use it any more. */
> +
> + put_hvpcibus(hbus);
>  }
>  
>  /**
> --
> 2.20.1
>
>
> --
> kernel-team mailing list
> [hidden email]
> https://lists.ubuntu.com/mailman/listinfo/kernel-team

Acked-by: Sultan Alsawaf <[hidden email]>

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

ACK: [d/azure][PATCH 2/2] UBUNTU: SAUCE: PCI: hv: Fix panic by calling hv_pci_remove_slots() earlier

Sultan Alsawaf
In reply to this post by Marcelo Henrique Cerri
On Thu, Aug 08, 2019 at 12:34:53PM -0300, Marcelo Henrique Cerri wrote:

> From: Dexuan Cui <[hidden email]>
>
> BugLink: https://bugs.launchpad.net/bugs/1837661
>
> The slot must be removed before the pci_dev is removed, otherwise a panic
> can happen due to use-after-free.
>
> Fixes: 15becc2b56c6 ("PCI: hv: Add hv_pci_remove_slots() when we unload the driver")
> Signed-off-by: Dexuan Cui <[hidden email]>
> Cc: [hidden email]
> Signed-off-by: Marcelo Henrique Cerri <[hidden email]>
> ---
>  drivers/pci/controller/pci-hyperv.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index ae91e708796f..3bd94149b9d2 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -2714,8 +2714,8 @@ static int hv_pci_remove(struct hv_device *hdev)
>   /* Remove the bus from PCI's point of view. */
>   pci_lock_rescan_remove();
>   pci_stop_root_bus(hbus->pci_bus);
> - pci_remove_root_bus(hbus->pci_bus);
>   hv_pci_remove_slots(hbus);
> + pci_remove_root_bus(hbus->pci_bus);
>   pci_unlock_rescan_remove();
>   hbus->state = hv_pcibus_removed;
>   }
> --
> 2.20.1
>
>
> --
> kernel-team mailing list
> [hidden email]
> https://lists.ubuntu.com/mailman/listinfo/kernel-team

Acked-by: Sultan Alsawaf <[hidden email]>

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

Cmnt: [d/azure][PATCH 0/2] LP: #1837661 - [linux-azure] CRI-RDOS | Live migration only takes 10 seconds, but the VM was unavailable for 2 hours

Marcelo Henrique Cerri
In reply to this post by Marcelo Henrique Cerri

Just a note: the missing commits that are mentioned on the bug were
already included to our 5.0 kernels via stable updates.

On Thu, Aug 08, 2019 at 12:34:51PM -0300, Marcelo Henrique Cerri wrote:

> BugLink: https://bugs.launchpad.net/bugs/1837661
>
> It's still not clear if the fixes are really needed for 4.15. So I'm
> submitting the fixes for 5.0 so it can make into the next SRU
> cycle. The fixes for 4.15 will be handled separately.
>
> Dexuan Cui (2):
>   PCI: hv: Fix a use-after-free bug in hv_eject_device_work()
>   UBUNTU: SAUCE: PCI: hv: Fix panic by calling hv_pci_remove_slots()
>     earlier
>
>  drivers/pci/controller/pci-hyperv.c | 17 ++++++++++-------
>  1 file changed, 10 insertions(+), 7 deletions(-)
>
> --
> 2.20.1
>
--
Regards,
Marcelo


--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

ACK: [d/azure][PATCH 0/2] LP: #1837661 - [linux-azure] CRI-RDOS | Live migration only takes 10 seconds, but the VM was unavailable for 2 hours

Connor Kuehl
In reply to this post by Marcelo Henrique Cerri
On 8/8/19 8:34 AM, Marcelo Henrique Cerri wrote:

> BugLink: https://bugs.launchpad.net/bugs/1837661
>
> It's still not clear if the fixes are really needed for 4.15. So I'm
> submitting the fixes for 5.0 so it can make into the next SRU
> cycle. The fixes for 4.15 will be handled separately.
>
> Dexuan Cui (2):
>   PCI: hv: Fix a use-after-free bug in hv_eject_device_work()
>   UBUNTU: SAUCE: PCI: hv: Fix panic by calling hv_pci_remove_slots()
>     earlier
>
>  drivers/pci/controller/pci-hyperv.c | 17 ++++++++++-------
>  1 file changed, 10 insertions(+), 7 deletions(-)
>

Acked-by: Connor Kuehl <[hidden email]>

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team
Reply | Threaded
Open this post in threaded view
|

APPLIED: [d/azure][PATCH 0/2] LP: #1837661 - [linux-azure] CRI-RDOS | Live migration only takes 10 seconds, but the VM was unavailable for 2 hours

Khaled Elmously
In reply to this post by Marcelo Henrique Cerri
On 2019-08-08 12:34:51 , Marcelo Henrique Cerri wrote:

> BugLink: https://bugs.launchpad.net/bugs/1837661
>
> It's still not clear if the fixes are really needed for 4.15. So I'm
> submitting the fixes for 5.0 so it can make into the next SRU
> cycle. The fixes for 4.15 will be handled separately.
>
> Dexuan Cui (2):
>   PCI: hv: Fix a use-after-free bug in hv_eject_device_work()
>   UBUNTU: SAUCE: PCI: hv: Fix panic by calling hv_pci_remove_slots()
>     earlier
>
>  drivers/pci/controller/pci-hyperv.c | 17 ++++++++++-------
>  1 file changed, 10 insertions(+), 7 deletions(-)
>
> --
> 2.20.1
>
>
> --
> kernel-team mailing list
> [hidden email]
> https://lists.ubuntu.com/mailman/listinfo/kernel-team

--
kernel-team mailing list
[hidden email]
https://lists.ubuntu.com/mailman/listinfo/kernel-team