Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallel.ForEach should not be used for IO bound tasks #22957

Merged
merged 2 commits into from
Mar 3, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Write a simple parallel program using Parallel.ForEach
description: In this article, learn how to enable data parallelism in .NET. Write a Parallel.ForEach loop over any IEnumerable or IEnumerable<T> data source.
ms.date: 02/14/2019
ms.date: 02/23/2021
dev_langs:
- "csharp"
- "vb"
Expand All @@ -19,7 +19,7 @@ This example shows how to use a <xref:System.Threading.Tasks.Parallel.ForEach%2A

## Example

This example assumes you have several .jpg files in a *C:\Users\Public\Pictures\Sample Pictures* folder and creates a new sub-folder named *Modified*. When you run the example, it rotates each .jpg image in *Sample Pictures* and saves it to *Modified*. You can modify the two paths as necessary.
This example showcases the power of Parallel.ForEach for CPU intensive operations. Only for demonstration purpose we have picked up operation around Prime numbers. When you run the example, it randomly generates 2 million numbers and tries to filter only Prime Numbers. In first case we iterate over the collection via simple for loop, in second case we iterate over the collection via Parallel.ForEach. In the end we display the time taken in both the cases.
surenderssm marked this conversation as resolved.
Show resolved Hide resolved

[!code-csharp[TPL_Parallel#03](../../../samples/snippets/csharp/VS_Snippets_Misc/tpl_parallel/cs/simpleforeach.cs#03)]
[!code-vb[TPL_Parallel#03](../../../samples/snippets/visualbasic/VS_Snippets_Misc/tpl_parallel/vb/simpleforeach.vb#03)]
Expand All @@ -43,14 +43,6 @@ In Visual Studio, there are Visual Basic and C# console application templates fo

From the command line, you can use either the .NET Core CLI commands (for example, `dotnet new console` or `dotnet new console -lang vb`), or you can create the file and use the command-line compiler for a .NET Framework application.

For a .NET Core project, you must reference the **System.Drawing.Common** NuGet package. In Visual Studio, use the NuGet Package Manager to install the package. Alternatively, you can add a reference to the package in your \*.csproj or \*.vbproj file:

```xml
<ItemGroup>
<PackageReference Include="System.Drawing.Common" Version="4.5.1" />
</ItemGroup>
```

To run a .NET Core console application from the command line, use `dotnet run` from the folder that contains your application.

To run your console application from Visual Studio, press **F5**.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,40 +1,112 @@
// <snippet03>
using System;
using System.IO;
using System.Threading;
using System.Collections.Concurrent;
surenderssm marked this conversation as resolved.
Show resolved Hide resolved
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading.Tasks;
using System.Drawing;

public class Example
namespace ParallelExample
{
public static void Main()
class Program
{
// A simple source for demonstration purposes. Modify this path as necessary.
string[] files = Directory.GetFiles(@"C:\Users\Public\Pictures\Sample Pictures", "*.jpg");
string newDir = @"C:\Users\Public\Pictures\Sample Pictures\Modified";
Directory.CreateDirectory(newDir);

// Method signature: Parallel.ForEach(IEnumerable<TSource> source, Action<TSource> body)
Parallel.ForEach(files, (currentFile) =>
{
// The more computational work you do here, the greater
// the speedup compared to a sequential foreach loop.
string filename = Path.GetFileName(currentFile);
var bitmap = new Bitmap(currentFile);

bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone);
bitmap.Save(Path.Combine(newDir, filename));

// Peek behind the scenes to see how work is parallelized.
// But be aware: Thread contention for the Console slows down parallel loops!!!

Console.WriteLine($"Processing {filename} on thread {Thread.CurrentThread.ManagedThreadId}");
//close lambda expression and method invocation
});

// Keep the console window open in debug mode.
Console.WriteLine("Processing complete. Press any key to exit.");
Console.ReadKey();
static void Main(string[] args)
surenderssm marked this conversation as resolved.
Show resolved Hide resolved
{
// 2 million
int limit = 2 * 1000000;
surenderssm marked this conversation as resolved.
Show resolved Hide resolved
var inputs = new List<int>(limit);
Random radomGenerator = new Random();
for (int index = 0; index < limit; index++)
{
inputs.Add(radomGenerator.Next());
}
surenderssm marked this conversation as resolved.
Show resolved Hide resolved

var watch = new Stopwatch();
watch.Start();
var primeNumbers = GetPrimeList(inputs);
watch.Stop();
surenderssm marked this conversation as resolved.
Show resolved Hide resolved

var watchForParallel = new Stopwatch();
watchForParallel.Start();
var primeNumbersFromParallel = GetPrimeListWithParallel(inputs);
watchForParallel.Stop();
surenderssm marked this conversation as resolved.
Show resolved Hide resolved

Console.WriteLine($"Classical For loop | Total prime numbers : {primeNumbersFromParallel.Count} | Time Taken : {watch.ElapsedMilliseconds} ms.");
surenderssm marked this conversation as resolved.
Show resolved Hide resolved
Console.WriteLine($"Parallel.ForEach loop | Total prime numbers : {primeNumbersFromParallel.Count} | Time Taken : {watchForParallel.ElapsedMilliseconds} ms.");

Console.WriteLine("Press any key to exit.");
Console.ReadLine();
}

/// <summary>
/// GetPrimeList returns Prime numbers by using sequential ForEach
/// </summary>
/// <param name="inputs"></param>
/// <returns></returns>
static IList<int> GetPrimeList(IList<int> inputs)
{
var primeNumbers = new List<int>();

foreach (var item in inputs)
{
if (IsPrime(item))
{
primeNumbers.Add(item);
}
}

return primeNumbers;
}
surenderssm marked this conversation as resolved.
Show resolved Hide resolved

/// <summary>
/// GetPrimeListWithParallel returns Prime numbers by using Parallel.ForEach
/// </summary>
/// <param name="inputs"></param>
/// <returns></returns>
static IList<int> GetPrimeListWithParallel(IList<int> inputs)
{
var primeNumbers = new ConcurrentBag<int>();

Parallel.ForEach(inputs, item =>
{
if (IsPrime(item))
{
primeNumbers.Add(item);
}
});

return primeNumbers.ToList();
}
surenderssm marked this conversation as resolved.
Show resolved Hide resolved

/// <summary>
/// IsPrime returns true if number is Prime, else false.(https://en.wikipedia.org/wiki/Prime_number)
/// </summary>
/// <param name="number"></param>
/// <returns></returns>
static bool IsPrime(int number)
{

if (number <= 1)
{
return false;
}

if (number == 2 || number % 2 == 0)
{
return true;
}

int limit = (int)Math.Floor(Math.Sqrt(number));

for (int index = 3; index <= limit; index += 2)
{
if (number % index == 0)
{
return false;
}
}
return true;
}
surenderssm marked this conversation as resolved.
Show resolved Hide resolved
}
}
// </snippet03>
Original file line number Diff line number Diff line change
@@ -1,37 +1,83 @@
'<snippet03>
Imports System.IO
Imports System.Threading
Imports System.Threading.Tasks
Imports System.Drawing
Imports System.Collections.Concurrent
surenderssm marked this conversation as resolved.
Show resolved Hide resolved

Module ForEachDemo
Namespace ParallelExample
Class Program
Shared Sub Main()

Sub Main()
' A simple source for demonstration purposes. Modify this path as necessary.
Dim files As String() = Directory.GetFiles("C:\Users\Public\Pictures\Sample Pictures", "*.jpg")
Dim newDir As String = "C:\Users\Public\Pictures\Sample Pictures\Modified"
Directory.CreateDirectory(newDir)
'2 million
Dim limit As Integer = 2 * 1000000
Dim inputs = New List(Of Integer)(limit)
Dim radomGenerator As Random = New Random()

Parallel.ForEach(files, Sub(currentFile)
' The more computational work you do here, the greater
' the speedup compared to a sequential foreach loop.
Dim filename As String = Path.GetFileName(currentFile)
Dim bitmap As New Bitmap(currentFile)
For index As Integer = 0 To limit - 1
inputs.Add(radomGenerator.[Next]())
Next

bitmap.RotateFlip(System.Drawing.RotateFlipType.Rotate180FlipNone)
bitmap.Save(Path.Combine(newDir, filename))
Dim watch = New Stopwatch()
watch.Start()
Dim primeNumbers = GetPrimeList(inputs)
watch.Stop()

' Peek behind the scenes to see how work is parallelized.
' But be aware: Thread contention for the Console slows down parallel loops!!!
Dim watchForParallel = New Stopwatch()
watchForParallel.Start()
Dim primeNumbersFromParallel = GetPrimeListWithParallel(inputs)
watchForParallel.Stop()

Console.WriteLine($"Processing {filename} on thread {Thread.CurrentThread.ManagedThreadId}")
'close lambda expression and method invocation
End Sub)
Console.WriteLine($"Classical For loop | Total prime numbers : {primeNumbersFromParallel.Count} | Time Taken : {watch.ElapsedMilliseconds} ms.")
Console.WriteLine($"Parallel.ForEach loop | Total prime numbers : {primeNumbersFromParallel.Count} | Time Taken : {watchForParallel.ElapsedMilliseconds} ms.")

Console.WriteLine("Press any key to exit.")
Console.ReadLine()
End Sub

' Keep the console window open in debug mode.
Console.WriteLine("Processing complete. Press any key to exit.")
Console.ReadKey()
End Sub
End Module
' GetPrimeList returns Prime numbers by using sequential ForEach
Private Shared Function GetPrimeList(inputs As IList(Of Integer)) As IList(Of Integer)
Dim primeNumbers = New List(Of Integer)()

For Each item In inputs

If IsPrime(item) Then
primeNumbers.Add(item)
End If
Next

Return primeNumbers
End Function

' GetPrimeListWithParallel returns Prime numbers by using Parallel.ForEach
Private Shared Function GetPrimeListWithParallel(inputs As IList(Of Integer)) As IList(Of Integer)
Dim primeNumbers = New ConcurrentBag(Of Integer)()
Parallel.ForEach(inputs, Sub(item)

If IsPrime(item) Then
primeNumbers.Add(item)
End If
End Sub)
Return primeNumbers.ToList()
End Function

' IsPrime returns true if number is Prime, else false.(https://en.wikipedia.org/wiki/Prime_number)
Private Shared Function IsPrime(number As Integer) As Boolean
If number <= 1 Then
Return False
End If

If number = 2 OrElse number Mod 2 = 0 Then
Return True
End If

Dim limit As Integer = CInt(Math.Floor(Math.Sqrt(number)))

For index As Integer = 3 To limit Step 2

If number Mod index = 0 Then
Return False
End If
Next

Return True
End Function
End Class
End Namespace
'</snippet03>