Allow toggling filtering of duplicate messages on agent via configuration

[virizar]

Hi @PatrickRitchie

For our applications, we need to be able to allow messages to be sent from the agent even if the content of the message is the same on consecutive observations.

I have noticed that the agent, when checking if the data items of an observation have delta filtering, it checks if the message has not changed and if so prevents the observation to be updated and resend.

MTConnect.NET/libraries/MTConnect.NET-Common/Agents/MTConnectAgent.cs

Lines 1037 to 1065 in cb48e67

	private static bool FilterDelta(IDataItem dataItem, IObservationInput newObservation, IObservationInput existingObservation)
	{
	if (dataItem != null)
	{
	if (!ObjectExtensions.ByteArraysEqual(newObservation.ChangeId, existingObservation.ChangeId))
	{
	if (!dataItem.Filters.IsNullOrEmpty() && dataItem.Representation == DataItemRepresentation.VALUE)
	{
	foreach (var filter in dataItem.Filters)
	{
	if (filter.Type == DataItemFilterType.MINIMUM_DELTA)
	{
	if (filter.Value > 0)
	{
	var x = newObservation.GetValue(ValueKeys.Result).ToDouble();
	var y = existingObservation.GetValue(ValueKeys.Result).ToDouble();
	// If difference between New and Existing exceeds Filter Minimum Delta Value
	return Math.Abs(x - y) > filter.Value;
	}
	}
	}
	}
	return true;
	}
	}
	return false;

MTConnect.NET/libraries/MTConnect.NET-Common/Agents/MTConnectAgent.cs

Lines 801 to 833 in cb48e67

	protected bool CheckCurrentObservation(string deviceUuid, IDataItem dataItem, IObservationInput observation)
	{
	if (_currentObservations != null && observation != null && !string.IsNullOrEmpty(deviceUuid) && dataItem != null)
	{
	var hash = $"{deviceUuid}:{dataItem.Id}";
	_currentObservations.TryGetValue(hash, out var existingObservation);
	if (observation != null && existingObservation != null)
	{
	var timestamp = observation.Timestamp;
	var existingTimestamp = existingObservation.Timestamp.ToUnixTime();
	var existingObservationInput = new ObservationInput(existingObservation);
	// Check Filters
	var update = FilterPeriod(dataItem, timestamp, existingTimestamp);
	if (update) update = FilterDelta(dataItem, observation, existingObservationInput);
	// Update if Filters are passed or if the DataItem is set to Discrete
	if (update || dataItem.Discrete)
	{
	return true;
	}
	}
	else
	{
	return true;
	}
	}
	return false;
	}

I can see that the adapter applications have a FilterDuplicates configuration element to toggle this behavior.

Is this something that could also be relevant for the agent? I can make a PR with what I am current using if this is of value for the agent.

[virizar]

Actually, it seems that each data item can be set to discreet on the device config? that would be the equivalent of this I suppose

[virizar]

Closing this since the discrete attribute of the data items is what should be used for this!