Consider replacing fcntl with portalocker for Windows support

[jpmckinney]

It's not available on Windows.

[Bjwebb]

It's used for file locking in

lib-cove/libcove/lib/common.py

Lines 1241 to 1270 in 0bc725b

	if os.path.exists(local_org_ids_file):
	with open(lock_file, "w") as lock:
	fcntl.flock(lock, fcntl.LOCK_EX)
	fp = open(local_org_ids_file)
	org_ids = json.load(fp)
	fp.close()
	fcntl.flock(lock, fcntl.LOCK_UN)
	date_str = org_ids.get("downloaded", "2000-1-1")
	date_downloaded = datetime.datetime.strptime(date_str, "%Y-%m-%d").date()
	if date_downloaded != today:
	get_remote_file = True
	else:
	get_remote_file = True
	first_request = True
	if get_remote_file:
	try:
	org_ids = requests.get(orgids_url).json()
	org_ids["downloaded"] = "%s" % today
	with open(lock_file, "w") as lock:
	fcntl.flock(lock, fcntl.LOCK_EX)
	fp = open(local_org_ids_file, "w")
	json.dump(org_ids, fp, indent=2)
	fp.close()
	fcntl.flock(lock, fcntl.LOCK_UN)
	except requests.exceptions.RequestException:
	if first_request:
	raise # First time ever request fails
	pass # Update fails

We could take a different approach to this, but I'm not quite sure how much work it would be.

[jpmckinney]

Why do we need to lock the file, though? In what scenario would there be simultaneous writes?

[Bjwebb]

This gets called from cove-ocds web requests, where we may have multiple processes/threads calling this function at once.

[jpmckinney]

Aha. Renamed issue and found this library that can perhaps be used to support multiple operating systems: https://pypi.org/project/portalocker/

[benvand]

My proposal for safely removing the file locking

The 2 race conditions to account for are:

Process A checks for the existence of the file finds none
Process B checks for the existence of the file finds none
Process A/ B writes new file
Process A/ B writes new file

Process A checks for the existence of the file finds none
Process A opens file for writing
Process B checks for the existence of the file
Process B opens the file
Process B reads nothing
Process A writes to file

In the first case we don't care because the data is never mutated and gathered from the same source.
In the second case the trick is to write the file before you ever open it :) Create a temp file, write to that, then rename it.

This wouldn't work in cases where data is mutated, but it does in ours.

Also added tests

[jpmckinney]

Closed by #74

[jpmckinney]

@Bjwebb You can close this issue to declutter issue tracker.