Skip to content

Commit

Permalink
Parallel.ForEach should not be used for IO bound tasks (#22957)
Browse files Browse the repository at this point in the history
* Parallel.ForEach with CPU intensive operations

* Refactored as per the code review comments
  • Loading branch information
surenderssm authored Mar 3, 2021
1 parent 060a8dc commit 67ff206
Show file tree
Hide file tree
Showing 5 changed files with 151 additions and 78 deletions.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Write a simple parallel program using Parallel.ForEach
description: In this article, learn how to enable data parallelism in .NET. Write a Parallel.ForEach loop over any IEnumerable or IEnumerable<T> data source.
ms.date: 02/14/2019
ms.date: 02/23/2021
dev_langs:
- "csharp"
- "vb"
Expand All @@ -19,7 +19,7 @@ This example shows how to use a <xref:System.Threading.Tasks.Parallel.ForEach%2A
## Example

This example assumes you have several .jpg files in a *C:\Users\Public\Pictures\Sample Pictures* folder and creates a new sub-folder named *Modified*. When you run the example, it rotates each .jpg image in *Sample Pictures* and saves it to *Modified*. You can modify the two paths as necessary.
This example demonstrates <xref:System.Threading.Tasks.Parallel.ForEach%2A?displayProperty=nameWithType> for CPU intensive operations. When you run the example, it randomly generates 2 million numbers and tries to filter to prime numbers. The first case iterates over the collection via a `for` loop. The second case iterates over the collection via <xref:System.Threading.Tasks.Parallel.ForEach%2A?displayProperty=nameWithType>. The resulting time taken by each iteration is displayed when the application is finished.

[!code-csharp[TPL_Parallel#03](../../../samples/snippets/csharp/VS_Snippets_Misc/tpl_parallel/cs/simpleforeach.cs#03)]
[!code-vb[TPL_Parallel#03](../../../samples/snippets/visualbasic/VS_Snippets_Misc/tpl_parallel/vb/simpleforeach.vb#03)]
Expand All @@ -43,14 +43,6 @@ In Visual Studio, there are Visual Basic and C# console application templates fo

From the command line, you can use either the .NET Core CLI commands (for example, `dotnet new console` or `dotnet new console -lang vb`), or you can create the file and use the command-line compiler for a .NET Framework application.

For a .NET Core project, you must reference the **System.Drawing.Common** NuGet package. In Visual Studio, use the NuGet Package Manager to install the package. Alternatively, you can add a reference to the package in your \*.csproj or \*.vbproj file:

```xml
<ItemGroup>
<PackageReference Include="System.Drawing.Common" Version="4.5.1" />
</ItemGroup>
```

To run a .NET Core console application from the command line, use `dotnet run` from the folder that contains your application.

To run your console application from Visual Studio, press **F5**.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,40 +1,84 @@
// <snippet03>
using System;
using System.IO;
using System.Threading;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;
using System.Drawing;

public class Example
namespace ParallelExample
{
public static void Main()
class Program
{
// A simple source for demonstration purposes. Modify this path as necessary.
string[] files = Directory.GetFiles(@"C:\Users\Public\Pictures\Sample Pictures", "*.jpg");
string newDir = @"C:\Users\Public\Pictures\Sample Pictures\Modified";
Directory.CreateDirectory(newDir);

// Method signature: Parallel.ForEach(IEnumerable<TSource> source, Action<TSource> body)
Parallel.ForEach(files, (currentFile) =>
{
// The more computational work you do here, the greater
// the speedup compared to a sequential foreach loop.
string filename = Path.GetFileName(currentFile);
var bitmap = new Bitmap(currentFile);

bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone);
bitmap.Save(Path.Combine(newDir, filename));

// Peek behind the scenes to see how work is parallelized.
// But be aware: Thread contention for the Console slows down parallel loops!!!

Console.WriteLine($"Processing {filename} on thread {Thread.CurrentThread.ManagedThreadId}");
//close lambda expression and method invocation
});

// Keep the console window open in debug mode.
Console.WriteLine("Processing complete. Press any key to exit.");
Console.ReadKey();
static void Main()
{
// 2 million
var limit = 2_000_000;
var numbers = Enumerable.Range(0, limit).ToList();

var watch = Stopwatch.StartNew();
var primeNumbersFromForeach = GetPrimeList(numbers);
watch.Stop();

var watchForParallel = Stopwatch.StartNew();
var primeNumbersFromParallelForeach = GetPrimeListWithParallel(numbers);
watchForParallel.Stop();

Console.WriteLine($"Classical foreach loop | Total prime numbers : {primeNumbersFromForeach.Count} | Time Taken : {watch.ElapsedMilliseconds} ms.");
Console.WriteLine($"Parallel.ForEach loop | Total prime numbers : {primeNumbersFromParallelForeach.Count} | Time Taken : {watchForParallel.ElapsedMilliseconds} ms.");

Console.WriteLine("Press any key to exit.");
Console.ReadLine();
}

/// <summary>
/// GetPrimeList returns Prime numbers by using sequential ForEach
/// </summary>
/// <param name="inputs"></param>
/// <returns></returns>
private static IList<int> GetPrimeList(IList<int> numbers) => numbers.Where(IsPrime).ToList();

/// <summary>
/// GetPrimeListWithParallel returns Prime numbers by using Parallel.ForEach
/// </summary>
/// <param name="numbers"></param>
/// <returns></returns>
private static IList<int> GetPrimeListWithParallel(IList<int> numbers)
{
var primeNumbers = new ConcurrentBag<int>();

Parallel.ForEach(numbers, number =>
{
if (IsPrime(number))
{
primeNumbers.Add(number);
}
});

return primeNumbers.ToList();
}

/// <summary>
/// IsPrime returns true if number is Prime, else false.(https://en.wikipedia.org/wiki/Prime_number)
/// </summary>
/// <param name="number"></param>
/// <returns></returns>
private static bool IsPrime(int number)
{
if (number < 2)
{
return false;
}

for (var divisor = 2; divisor <= Math.Sqrt(number); divisor++)
{
if (number % divisor == 0)
{
return false;
}
}
return true;
}
}
}
// </snippet03>
// </snippet03>
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net5.0</TargetFramework>
</PropertyGroup>

</Project>
Original file line number Diff line number Diff line change
@@ -1,37 +1,60 @@
'<snippet03>
Imports System.IO
Imports System.Threading
Imports System.Threading.Tasks
Imports System.Drawing

Module ForEachDemo

Sub Main()
' A simple source for demonstration purposes. Modify this path as necessary.
Dim files As String() = Directory.GetFiles("C:\Users\Public\Pictures\Sample Pictures", "*.jpg")
Dim newDir As String = "C:\Users\Public\Pictures\Sample Pictures\Modified"
Directory.CreateDirectory(newDir)

Parallel.ForEach(files, Sub(currentFile)
' The more computational work you do here, the greater
' the speedup compared to a sequential foreach loop.
Dim filename As String = Path.GetFileName(currentFile)
Dim bitmap As New Bitmap(currentFile)

bitmap.RotateFlip(System.Drawing.RotateFlipType.Rotate180FlipNone)
bitmap.Save(Path.Combine(newDir, filename))

' Peek behind the scenes to see how work is parallelized.
' But be aware: Thread contention for the Console slows down parallel loops!!!

Console.WriteLine($"Processing {filename} on thread {Thread.CurrentThread.ManagedThreadId}")
'close lambda expression and method invocation
End Sub)


' Keep the console window open in debug mode.
Console.WriteLine("Processing complete. Press any key to exit.")
Console.ReadKey()
End Sub
End Module
'</snippet03>
Imports System.Collections.Concurrent

Namespace ParallelExample
Class Program
Shared Sub Main()
' 2 million
Dim limit = 2_000_000
Dim numbers = Enumerable.Range(0, limit).ToList()

Dim watch = Stopwatch.StartNew()
Dim primeNumbersFromForeach = GetPrimeList(numbers)
watch.Stop()

Dim watchForParallel = Stopwatch.StartNew()
Dim primeNumbersFromParallelForeach = GetPrimeListWithParallel(numbers)
watchForParallel.Stop()

Console.WriteLine($"Classical foreach loop | Total prime numbers : {primeNumbersFromForeach.Count} | Time Taken : {watch.ElapsedMilliseconds} ms.")
Console.WriteLine($"Parallel.ForEach loop | Total prime numbers : {primeNumbersFromParallelForeach.Count} | Time Taken : {watchForParallel.ElapsedMilliseconds} ms.")

Console.WriteLine("Press any key to exit.")
Console.ReadLine()
End Sub

' GetPrimeList returns Prime numbers by using sequential ForEach
Private Shared Function GetPrimeList(numbers As IList(Of Integer)) As IList(Of Integer)
Return numbers.Where(AddressOf IsPrime).ToList()
End Function

' GetPrimeListWithParallel returns Prime numbers by using Parallel.ForEach
Private Shared Function GetPrimeListWithParallel(numbers As IList(Of Integer)) As IList(Of Integer)
Dim primeNumbers = New ConcurrentBag(Of Integer)()
Parallel.ForEach(numbers, Sub(number)

If IsPrime(number) Then
primeNumbers.Add(number)
End If
End Sub)
Return primeNumbers.ToList()
End Function

' IsPrime returns true if number is Prime, else false.(https://en.wikipedia.org/wiki/Prime_number)
Private Shared Function IsPrime(number As Integer) As Boolean
If number < 2 Then
Return False
End If

For divisor = 2 To Math.Sqrt(number)

If number Mod divisor = 0 Then
Return False
End If
Next

Return True
End Function
End Class
End Namespace
'</snippet03>
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>net5.0</TargetFramework>
</PropertyGroup>

</Project>

0 comments on commit 67ff206

Please sign in to comment.