[chore][fileconsumer/archive] - add signature methods for archive feature

[VihasMakwana]

Description:

This PR introduces the archive method to store files older than 3 poll cycles.
The core logic for reading will be introduced in a follow up PR.

Link to tracking Issue: #32727

Testing: To be included in future PRs

Documentation: To be included in future PRs

[VihasMakwana]

@djaglowski This is a draft PR which separates the reader creation and archiving module as per #32727 (comment)

let me know which looks good.

[VihasMakwana]

@djaglowski storing file handle is crucial for us because of the way we handle the archiving:

1. We try to match files against knownFiles array.
2. If a match isn't found, we would accumulate the unmatched files (instead of finding in the archive straightaway). More about it was discussed here.
3. Perform the archiving logic. Described here.

We need the file handle so that we can create a reader from it after the archiving is done.

[djaglowski]

This doesn't make any sense to me. See my comments on the other PR. I think this is a fundamental point that needs to be resolved before bothering with the rest of the PR.

reader.Metadata is sufficient and necessary for creating readers in all other circumstances except where we already have an open file handle, in which case we can do better.

I do not understand why we would or even could have open file handles at this point, since the archive should only be populated by elements which are discarded from knownFiles, which only contains reader.Metadata.

[VihasMakwana]

@djaglowski Thank you for the feedback. I realise now that I might have made things too complex with the initial PR, and we don’t actually need to keep files open while archiving.

[VihasMakwana]

@djaglowski please take a look at this PR. Appreciate your time reviewing this!

---

Addressing your points below:

What does storing the file handle give us?

As I said, I made things too complex with the initial PR, and we don’t actually need to keep files open while archiving.

Are we sure archive should be separate from tracker? Seems very closely related and kind of awkward to manage both.

Agree. Keeping everything in tracker would make it more maintainable.

reader.Metadata is sufficient and necessary for creating readers in all other circumstances except where we already have an open file handle, in which case we can do better.

This makes sense to me.

[VihasMakwana]

@djaglowski Thanks for your time in reviewing this PR!
Once this gets merged, I'll roll out another PR with reading from archive logic.
It'll also contain the test cases for this feature.

We can ofcourse break it down in multiple PRs if necessary.

[djaglowski]

Do we have to worry about these cases? Aren't we checking them and instantiating a NoStateTracker when these would ever be relevant?

[VihasMakwana]

NoStateTracker will be created when WithNoTracking option is selected.
We can create a normal tracker with archiving disabled, by setting pollsToArchive: 0.
A tracker with archiving enabled will have pollsToArchive > 0.

The check t.persister == nil is just a precautionary check.

[djaglowski]

Alright. Maybe in another PR we can clean this up. IMO we have starting to have too many special cases to handle and the easiest solution to that is move to a pattern where the nils pieces are nops instead. Then the implementation doesn't need to worry about details so much.