extract-ancestral-snps-from-1kGP.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4283f15b-42e6-42a6-b3b4-6a0a7544bf9d",
   "metadata": {},
   "source": [
    "# Ancestral SNPs in the 1000 Genomes Project"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adc3193a-208b-487e-9beb-70894b83bd22",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "In this notebook, we will extract the pre-defined fingerprinting (ancestral) SNPs from the vcf files of the 1000 Genomes Project. We will proceed through several steps to ensure we have a clean dataset:\n",
    "\n",
    "1. Extracting relevant SNP positions.\n",
    "2. Combining SNP data from various vcf files.\n",
    "3. Removing structural variants.\n",
    "4. Addressing duplicated chromosomal positions.\n",
    "5. Validating our extraction.\n",
    "6. Compressing and indexing the final vcf for efficient usage."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1947f54-8df8-48a8-a023-7ca769b825fb",
   "metadata": {},
   "source": [
    "## Extracting the fingeprinting SNPs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7d2836a0-f363-4c22-9f9f-459aa0800ed4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chromosome\tposition\tclass\trsid\tgene\talleles\tancestral_allele\tvariation_allele\n",
      "1\t1220751\tsnv\trs2887286\tSDF4\tT,C\tT\tC\n",
      "1\t2352457\tsnv\trs2840528\tMORN1/LOC100129534\tA,G\tA\tG\n",
      "1\t2622185\tsnv\trs3890745\tMMEL1\tT,C\tT\tC\n",
      "1\t3765267\tsnv\trs1181875\tCCDC27\tT,A,C\tT\tA,C\n",
      "1\t3826755\tsnv\trs6663840\tCEP104\tG,A,C\tG\tA,C\n",
      "1\t4304166\tsnv\trs693734\t\tC,A,G,T\tC\tA,G,T\n",
      "1\t4436599\tsnv\trs1674877\tLOC105376674\tC,T\tC\tT\n",
      "1\t4657331\tsnv\trs1483198\tAJAP1\tA,G\tA\tG\n",
      "1\t4927364\tsnv\trs6666453\t\tT,C,G\tT\tC,G\n"
     ]
    }
   ],
   "source": [
    "# Let's begin by exploring and preparing our SNP data.\n",
    "head data/fingerprinting_snps_biallelic.tsv | cut -f 1-8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "85df8d6a-1260-4e8a-b70a-0ecd00a3e0d8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chr1\t1220751\n",
      "chr1\t2352457\n",
      "chr1\t2622185\n",
      "chr1\t3765267\n",
      "chr1\t3826755\n",
      "chr1\t4304166\n",
      "chr1\t4436599\n",
      "chr1\t4657331\n",
      "chr1\t4927364\n",
      "chr1\t5028557\n"
     ]
    }
   ],
   "source": [
    "cat data/fingerprinting_snps_biallelic.tsv | cut -f 1,2 | sed '1d' | sed 's/^/chr/g' > working/fingerprinting_snps_biallelic_chr.txt\n",
    "head working/fingerprinting_snps_biallelic_chr.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0dd9350f-2426-42a4-9ab9-9a1dc31d5cf2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8710 working/fingerprinting_snps_biallelic_chr.txt\n"
     ]
    }
   ],
   "source": [
    "wc -l working/fingerprinting_snps_biallelic_chr.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "05e48f3e-7627-4d7a-b2cc-721c4b5c0302",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "working/1kGP.fingerprinting.staging\n"
     ]
    }
   ],
   "source": [
    "# Create a combined fingerprinting vcf file\n",
    "output=\"working/1kGP.fingerprinting.staging\"\n",
    "echo $output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "5bdeff56-ceba-4d42-9e87-3d012da9f816",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract the header from the first vcf file\n",
    "bcftools view -h `cat data/1kGP.vcfs.txt | head -1` > $output.vcf"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "425aca18-29b4-458e-b8e9-b1bfb812e360",
   "metadata": {},
   "source": [
    "The paths of the vcfs of the chromosomes of the 1000 Genomes Project are in the text file `1kGP.vcfs.txt`. The vcfs were downloaded from the [1000 Genomes 30x on GRCh38](https://www.internationalgenome.org/data-portal/data-collection/30x-grch38)'s [phased SNV/INDEL/SV calls of 3,202 samples](http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000G_2504_high_coverage/working/20220422_3202_phased_SNV_INDEL_SV/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "7b06ddb0-70f5-4de2-a79f-e098214da6cc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr1.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr2.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr3.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr4.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr5.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr6.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr7.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr8.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr9.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr10.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr11.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr12.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr13.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr14.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr15.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr16.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr17.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr18.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr19.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr20.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr21.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr22.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n"
     ]
    }
   ],
   "source": [
    "cat data/1kGP.vcfs.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "08084a2d-e250-4983-9eba-4ce66cff7dba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr1.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m6.856s\n",
      "user\t0m6.818s\n",
      "sys\t0m0.036s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr2.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m7.457s\n",
      "user\t0m7.408s\n",
      "sys\t0m0.048s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr3.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m5.771s\n",
      "user\t0m5.701s\n",
      "sys\t0m0.069s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr4.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m5.327s\n",
      "user\t0m5.254s\n",
      "sys\t0m0.073s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr5.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m5.527s\n",
      "user\t0m5.462s\n",
      "sys\t0m0.065s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr6.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m5.039s\n",
      "user\t0m4.998s\n",
      "sys\t0m0.041s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr7.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m4.574s\n",
      "user\t0m4.544s\n",
      "sys\t0m0.029s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr8.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m5.000s\n",
      "user\t0m4.931s\n",
      "sys\t0m0.069s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr9.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m4.519s\n",
      "user\t0m4.470s\n",
      "sys\t0m0.049s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr10.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m4.502s\n",
      "user\t0m4.437s\n",
      "sys\t0m0.065s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr11.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m4.478s\n",
      "user\t0m4.437s\n",
      "sys\t0m0.040s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr12.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m4.267s\n",
      "user\t0m4.222s\n",
      "sys\t0m0.044s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr13.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m3.394s\n",
      "user\t0m3.361s\n",
      "sys\t0m0.032s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr14.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m2.916s\n",
      "user\t0m2.891s\n",
      "sys\t0m0.024s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr15.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m2.876s\n",
      "user\t0m2.828s\n",
      "sys\t0m0.048s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr16.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m2.890s\n",
      "user\t0m2.845s\n",
      "sys\t0m0.044s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr17.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m2.582s\n",
      "user\t0m2.550s\n",
      "sys\t0m0.032s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr18.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m2.489s\n",
      "user\t0m2.460s\n",
      "sys\t0m0.028s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr19.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m1.895s\n",
      "user\t0m1.859s\n",
      "sys\t0m0.037s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr20.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m1.989s\n",
      "user\t0m1.985s\n",
      "sys\t0m0.005s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr21.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m1.488s\n",
      "user\t0m1.471s\n",
      "sys\t0m0.017s\n",
      "\n",
      "/data/1kGP/vcfs/1kGP_high_coverage_Illumina.chr22.filtered.SNV_INDEL_SV_phased_panel.vcf.gz\n",
      "\n",
      "real\t0m1.447s\n",
      "user\t0m1.421s\n",
      "sys\t0m0.025s\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Loop through each 1kGP chromosome in vcfs.txt and append non-header lines\n",
    "while read -r vcf_file; do\n",
    "    echo $vcf_file\n",
    "    time bcftools view -H --regions-file working/fingerprinting_snps_biallelic_chr.txt $vcf_file >> $output.vcf\n",
    "    echo\n",
    "done < data/1kGP.vcfs.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f59f70d7-b0d0-4af3-9d19-91c81f846a06",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9768\n"
     ]
    }
   ],
   "source": [
    "# Number of variants in the generated vcf\n",
    "bcftools view -H $output.vcf | wc -l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "620201f8-bd61-4d5f-a054-625c362689d7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8710 working/fingerprinting_snps_biallelic_chr.txt\n"
     ]
    }
   ],
   "source": [
    "# Number of variants in the fingerprinting list\n",
    "wc -l working/fingerprinting_snps_biallelic_chr.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "13f78233-8c24-41c3-b09f-79193fb0e04e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chr22\t48977019\t22:48977019:A:G\tA\tG\n",
      "chr22\t49053806\t22:49053806:T:G\tT\tG\n",
      "chr22\t49390502\t22:49390502:G:A\tG\tA\n",
      "chr22\t49421090\tHGSV_248270\tA\t<DEL>\n",
      "chr22\t49457563\t22:49457563:G:A\tG\tA\n",
      "chr22\t49533849\t22:49533849:T:C\tT\tC\n",
      "chr22\t50018612\t22:50018612:C:T\tC\tT\n",
      "chr22\t50118978\t22:50118978:A:G\tA\tG\n",
      "chr22\t50226183\t22:50226183:A:G\tA\tG\n",
      "chr22\t50577409\t22:50577409:T:C\tT\tC\n"
     ]
    }
   ],
   "source": [
    "# Take a glimpse at the generated vcf\n",
    "bcftools view -H $output.vcf | tail | cut -f 1-5"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72af9216-c47f-4819-bf45-dbb888038dc3",
   "metadata": {},
   "source": [
    "## Remove Structural Variants (SV)\n",
    "Keep only the variants with allele sizes = 1bp (e.g., remove chr22:49421090 [HGSV_248270])."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "be5dad9d-56f5-4090-8259-d992263b9c3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "bcftools view -i 'strlen(REF) = 1 && strlen(ALT) = 1' $output.vcf  > $output\"_without_sv\".vcf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3d4a89ed-5472-4383-becb-526d0dc0fa80",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chr22\t48783199\t22:48783199:A:G\tA\tG\n",
      "chr22\t48977019\t22:48977019:A:G\tA\tG\n",
      "chr22\t49053806\t22:49053806:T:G\tT\tG\n",
      "chr22\t49390502\t22:49390502:G:A\tG\tA\n",
      "chr22\t49457563\t22:49457563:G:A\tG\tA\n",
      "chr22\t49533849\t22:49533849:T:C\tT\tC\n",
      "chr22\t50018612\t22:50018612:C:T\tC\tT\n",
      "chr22\t50118978\t22:50118978:A:G\tA\tG\n",
      "chr22\t50226183\t22:50226183:A:G\tA\tG\n",
      "chr22\t50577409\t22:50577409:T:C\tT\tC\n"
     ]
    }
   ],
   "source": [
    "# Display the last few entries of the VCF file after removing structural variants to verify the changes\n",
    "bcftools view -H $output\"_without_sv\".vcf | tail | cut -f 1-5"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3151f69-4a94-4809-9861-1abc99594cce",
   "metadata": {},
   "source": [
    "## Remove Duplicated Chromosomal Positions\n",
    "This is most likely due to multiallelic variants in the 1kGP."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "4a1e94e3-1258-4f91-b799-a2237baf391f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chr10\t125555957\n",
      "chr10\t7384682\n",
      "chr1\t110189511\n",
      "chr11\t18595637\n",
      "chr11\t86633635\n",
      "chr12\t56894884\n",
      "chr13\t110364539\n",
      "chr17\t34702551\n",
      "chr17\t5015163\n",
      "chr1\t77361047\n",
      "chr18\t78847535\n",
      "chr2\t215946984\n",
      "chr2\t233852579\n",
      "chr3\t59184489\n",
      "chr4\t113640577\n",
      "chr5\t13847875\n",
      "chr5\t2697846\n"
     ]
    }
   ],
   "source": [
    "# Identify and save duplicated chromosomal positions to a file.\n",
    "bcftools view -H $output\"_without_sv\".vcf | cut -f1,2 | sort | uniq -d > working/duplicated_positions.txt\n",
    "cat working/duplicated_positions.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "68bb4432-a899-4c3a-b93f-8ba605a47541",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove SNPs with duplicated chromosomal positions from the VCF file.\n",
    "bcftools view -T ^working/duplicated_positions.txt -o $output\"_without_sv_without_duplicates\".vcf $output\"_without_sv\".vcf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "1f633bf2-471c-42bf-b911-ebd99511671a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chr10-100057800\n",
      "chr10-100152307\n",
      "chr10-100349445\n",
      "chr10-100499949\n",
      "chr10-100881934\n",
      "chr10-10116693\n",
      "chr10-101195800\n",
      "chr10-101608897\n",
      "chr10-101692888\n",
      "chr10-101930813\n"
     ]
    }
   ],
   "source": [
    "cat working/fingerprinting_snps_biallelic_chr.txt | sed 's/\\t/-/g' | sort -u > working/in.snps.txt\n",
    "head working/in.snps.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f977092f-1ed9-4328-bc1b-3a88f36f8575",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8710 working/in.snps.txt\n"
     ]
    }
   ],
   "source": [
    "# Count the number of unique SNPs in the transformed data.\n",
    "wc -l working/in.snps.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "b540a075-c9f7-4603-80d7-4729d766d35d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chr10-100057800\n",
      "chr10-100152307\n",
      "chr10-100349445\n",
      "chr10-100499949\n",
      "chr10-100881934\n",
      "chr10-10116693\n",
      "chr10-101195800\n",
      "chr10-101608897\n",
      "chr10-101692888\n",
      "chr10-101930813\n"
     ]
    }
   ],
   "source": [
    "bcftools view -H $output\"_without_sv_without_duplicates\".vcf | cut -f 1,2 | sed 's/\\t/-/g' | sort -u > working/out.snps.txt\n",
    "head working/out.snps.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1cbf9c25-e546-4654-9cdc-dd7240f1b251",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8687 working/out.snps.txt\n"
     ]
    }
   ],
   "source": [
    "# Count the number of unique SNPs extracted from the processed vcf file.\n",
    "wc -l working/out.snps.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "351c3364-f57e-4d8c-89df-0a3a5f2f7c2f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chr10-125555957\n",
      "chr10-7384682\n",
      "chr1-110189511\n",
      "chr11-18595637\n",
      "chr11-86633635\n",
      "chr12-56894884\n",
      "chr13-110364539\n",
      "chr1-33689901\n",
      "chr1-62109755\n",
      "chr17-34702551\n",
      "chr17-5015163\n",
      "chr1-77361047\n",
      "chr18-78847535\n",
      "chr2-162272314\n",
      "chr2-215946984\n",
      "chr2-233852579\n",
      "chr3-59184489\n",
      "chr4-113640577\n",
      "chr5-13847875\n",
      "chr5-2697846\n",
      "chr7-81318201\n",
      "chr8-71685451\n",
      "chr9-18595550\n"
     ]
    }
   ],
   "source": [
    "# Identify SNPs from the original list that didn't make it to the final vcf.\n",
    "grep -v -f working/out.snps.txt working/in.snps.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "8ffc5ee8-c5c3-4873-965b-54f9d299f9ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compress the clean vcf\n",
    "cat $output\"_without_sv_without_duplicates\".vcf | bgzip -c > data/1kGP.fingerprinting.vcf.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "acd40192-52a6-49b9-9916-75d3bf1112ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Index the compressed clean vcf\n",
    "tabix data/1kGP.fingerprinting.vcf.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "6c199b91-8039-4f64-becc-b6e0c0d0c93f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "chr22\t48783199\t22:48783199:A:G\tA\tG\t.\t.\tAC=1954;AF=0.305122;CM=77.3726;AN=6404;AN_EAS=1170;AN_AMR=980;AN_EUR=1266;AN_AFR=1786;AN_SAS=1202;AN_EUR_unrel=1006;AN_EAS_unrel=1008;AN_AMR_unrel=694;AN_SAS_unrel=978;AN_AFR_unrel=1322;AF_EAS=0.157265;AF_AMR=0.276531;AF_EUR=0.414692;AF_AFR=0.407055;AF_SAS=0.205491;AF_EUR_unrel=0.411531;MAF_EUR_unrel=0.411531;AF_EAS_unrel=0.159722;MAF_EAS_unrel=0.159722;AF_AMR_unrel=0.285303;MAF_AMR_unrel=0.285303;AF_SAS_unrel=0.201431;MAF_SAS_unrel=0.201431;AF_AFR_unrel=0.405446;MAF_AFR_unrel=0.405446;AC_EAS=184;AC_AMR=271;AC_EUR=525;AC_AFR=727;AC_SAS=247;AC_EUR_unrel=414;AC_EAS_unrel=161;AC_AMR_unrel=198;AC_SAS_unrel=197;AC_AFR_unrel=536;AC_Het_EAS=164;AC_Het_AMR=179;AC_Het_EUR=309;AC_Het_AFR=405;AC_Het_SAS=193;AC_Het_EUR_unrel=246;AC_Het_EAS_unrel=143;AC_Het_AMR_unrel=122;AC_Het_SAS_unrel=159;AC_Het_AFR_unrel=298;AC_Het=1250;AC_Hom_EAS=20;AC_Hom_AMR=92;AC_Hom_EUR=216;AC_Hom_AFR=322;AC_Hom_SAS=54;AC_Hom_EUR_unrel=168;AC_Hom_EAS_unrel=18;AC_Hom_AMR_unrel=76;AC_Hom_SAS_unrel=38;AC_Hom_AFR_unrel=238;AC_Hom=704;HWE_EAS=0.210882;ExcHet_EAS=0.108514;HWE_AMR=0.0552453;ExcHet_AMR=0.979489;HWE_EUR=0.934795;ExcHet_EUR=0.484954;HWE_AFR=0.0716397;ExcHet_AFR=0.970789;HWE_SAS=0.707617;ExcHet_SAS=0.71265;HWE=8.09645e-06;ExcHet=0.999997\tGT\t1|0\t0|0\t0|1\t0|1\t0|1\t1|0\t0|0\t0|0\t0|1\t0|0\t0|0\n",
      "chr22\t48977019\t22:48977019:A:G\tA\tG\t.\t.\tAC=5245;AF=0.819019;CM=78.5874;AN=6404;AN_EAS=1170;AN_AMR=980;AN_EUR=1266;AN_AFR=1786;AN_SAS=1202;AN_EUR_unrel=1006;AN_EAS_unrel=1008;AN_AMR_unrel=694;AN_SAS_unrel=978;AN_AFR_unrel=1322;AF_EAS=0.841026;AF_AMR=0.768367;AF_EUR=0.751185;AF_AFR=0.848264;AF_SAS=0.866889;AF_EUR_unrel=0.749503;MAF_EUR_unrel=0.250497;AF_EAS_unrel=0.842262;MAF_EAS_unrel=0.157738;AF_AMR_unrel=0.760807;MAF_AMR_unrel=0.239193;AF_SAS_unrel=0.865031;MAF_SAS_unrel=0.134969;AF_AFR_unrel=0.847201;MAF_AFR_unrel=0.152799;AC_EAS=984;AC_AMR=753;AC_EUR=951;AC_AFR=1515;AC_SAS=1042;AC_EUR_unrel=754;AC_EAS_unrel=849;AC_AMR_unrel=528;AC_SAS_unrel=846;AC_AFR_unrel=1120;AC_Het_EAS=152;AC_Het_AMR=169;AC_Het_EUR=241;AC_Het_AFR=227;AC_Het_SAS=128;AC_Het_EUR_unrel=196;AC_Het_EAS_unrel=127;AC_Het_AMR_unrel=124;AC_Het_SAS_unrel=108;AC_Het_AFR_unrel=160;AC_Het=917;AC_Hom_EAS=832;AC_Hom_AMR=584;AC_Hom_EUR=710;AC_Hom_AFR=1288;AC_Hom_SAS=914;AC_Hom_EUR_unrel=558;AC_Hom_EAS_unrel=722;AC_Hom_AMR_unrel=404;AC_Hom_SAS_unrel=738;AC_Hom_AFR_unrel=960;AC_Hom=4328;HWE_EAS=0.535039;ExcHet_EAS=0.807446;HWE_AMR=0.525224;ExcHet_AMR=0.800116;HWE_EUR=0.749569;ExcHet_EUR=0.370201;HWE_AFR=0.696647;ExcHet_AFR=0.704323;HWE_SAS=0.0743029;ExcHet_SAS=0.977787;HWE=0.0563969;ExcHet=0.97553\tGT\t1|0\t1|1\t1|1\t1|1\t0|1\t1|1\t1|1\t1|1\t1|1\t0|1\t1|1\n",
      "chr22\t49053806\t22:49053806:T:G\tT\tG\t.\t.\tAC=4750;AF=0.741724;CM=79.0809;AN=6404;AN_EAS=1170;AN_AMR=980;AN_EUR=1266;AN_AFR=1786;AN_SAS=1202;AN_EUR_unrel=1006;AN_EAS_unrel=1008;AN_AMR_unrel=694;AN_SAS_unrel=978;AN_AFR_unrel=1322;AF_EAS=0.782906;AF_AMR=0.867347;AF_EUR=0.798578;AF_AFR=0.669093;AF_SAS=0.647255;AF_EUR_unrel=0.792247;MAF_EUR_unrel=0.207753;AF_EAS_unrel=0.78373;MAF_EAS_unrel=0.21627;AF_AMR_unrel=0.854467;MAF_AMR_unrel=0.145533;AF_SAS_unrel=0.658487;MAF_SAS_unrel=0.341513;AF_AFR_unrel=0.673222;MAF_AFR_unrel=0.326778;AC_EAS=916;AC_AMR=850;AC_EUR=1011;AC_AFR=1195;AC_SAS=778;AC_EUR_unrel=797;AC_EAS_unrel=790;AC_AMR_unrel=593;AC_SAS_unrel=644;AC_AFR_unrel=890;AC_Het_EAS=208;AC_Het_AMR=114;AC_Het_EUR=193;AC_Het_AFR=413;AC_Het_SAS=274;AC_Het_EUR_unrel=153;AC_Het_EAS_unrel=182;AC_Het_AMR_unrel=91;AC_Het_SAS_unrel=222;AC_Het_AFR_unrel=296;AC_Het=1202;AC_Hom_EAS=708;AC_Hom_AMR=736;AC_Hom_EUR=818;AC_Hom_AFR=782;AC_Hom_SAS=504;AC_Hom_EUR_unrel=644;AC_Hom_EAS_unrel=608;AC_Hom_AMR_unrel=502;AC_Hom_SAS_unrel=422;AC_Hom_AFR_unrel=594;AC_Hom=3548;HWE_EAS=0.329905;ExcHet_EAS=0.165985;HWE_AMR=1;ExcHet_AMR=0.504237;HWE_EUR=0.215512;ExcHet_EUR=0.925801;HWE_AFR=0.19928;ExcHet_AFR=0.107977;HWE_SAS=1;ExcHet_SAS=0.56012;HWE=0.249123;ExcHet=0.88456\tGT\t0|1\t1|0\t1|0\t1|1\t1|0\t1|1\t1|1\t1|1\t1|1\t0|1\t1|1\n",
      "chr22\t49390502\t22:49390502:G:A\tG\tA\t.\t.\tAC=2399;AF=0.37461;CM=81.0644;AN=6404;AN_EAS=1170;AN_AMR=980;AN_EUR=1266;AN_AFR=1786;AN_SAS=1202;AN_EUR_unrel=1006;AN_EAS_unrel=1008;AN_AMR_unrel=694;AN_SAS_unrel=978;AN_AFR_unrel=1322;AF_EAS=0.168376;AF_AMR=0.459184;AF_EUR=0.408373;AF_AFR=0.450168;AF_SAS=0.358569;AF_EUR_unrel=0.406561;MAF_EUR_unrel=0.406561;AF_EAS_unrel=0.164683;MAF_EAS_unrel=0.164683;AF_AMR_unrel=0.455331;MAF_AMR_unrel=0.455331;AF_SAS_unrel=0.350716;MAF_SAS_unrel=0.350716;AF_AFR_unrel=0.450076;MAF_AFR_unrel=0.450076;AC_EAS=197;AC_AMR=450;AC_EUR=517;AC_AFR=804;AC_SAS=431;AC_EUR_unrel=409;AC_EAS_unrel=166;AC_AMR_unrel=316;AC_SAS_unrel=343;AC_AFR_unrel=595;AC_Het_EAS=155;AC_Het_AMR=242;AC_Het_EUR=297;AC_Het_AFR=422;AC_Het_SAS=249;AC_Het_EUR_unrel=235;AC_Het_EAS_unrel=132;AC_Het_AMR_unrel=164;AC_Het_SAS_unrel=197;AC_Het_AFR_unrel=315;AC_Het=1365;AC_Hom_EAS=42;AC_Hom_AMR=208;AC_Hom_EUR=220;AC_Hom_AFR=382;AC_Hom_SAS=182;AC_Hom_EUR_unrel=174;AC_Hom_EAS_unrel=34;AC_Hom_AMR_unrel=152;AC_Hom_SAS_unrel=146;AC_Hom_AFR_unrel=280;AC_Hom=1034;HWE_EAS=0.185447;ExcHet_EAS=0.926928;HWE_AMR=0.927544;ExcHet_AMR=0.594142;HWE_EUR=0.460073;ExcHet_EUR=0.797427;HWE_AFR=0.176682;ExcHet_AFR=0.925214;HWE_SAS=0.0165313;ExcHet_SAS=0.994388;HWE=3.50269e-07;ExcHet=1\tGT\t0|0\t0|0\t0|1\t0|1\t0|0\t0|0\t0|0\t1|0\t1|1\t1|0\t0|1\n",
      "chr22\t49457563\t22:49457563:G:A\tG\tA\t.\t.\tAC=1265;AF=0.197533;CM=81.7617;AN=6404;AN_EAS=1170;AN_AMR=980;AN_EUR=1266;AN_AFR=1786;AN_SAS=1202;AN_EUR_unrel=1006;AN_EAS_unrel=1008;AN_AMR_unrel=694;AN_SAS_unrel=978;AN_AFR_unrel=1322;AF_EAS=0.0777778;AF_AMR=0.104082;AF_EUR=0.130332;AF_AFR=0.43449;AF_SAS=0.108985;AF_EUR_unrel=0.127237;MAF_EUR_unrel=0.127237;AF_EAS_unrel=0.0833333;MAF_EAS_unrel=0.0833333;AF_AMR_unrel=0.112392;MAF_AMR_unrel=0.112392;AF_SAS_unrel=0.115542;MAF_SAS_unrel=0.115542;AF_AFR_unrel=0.433434;MAF_AFR_unrel=0.433434;AC_EAS=91;AC_AMR=102;AC_EUR=165;AC_AFR=776;AC_SAS=131;AC_EUR_unrel=128;AC_EAS_unrel=84;AC_AMR_unrel=78;AC_SAS_unrel=113;AC_AFR_unrel=573;AC_Het_EAS=81;AC_Het_AMR=90;AC_Het_EUR=147;AC_Het_AFR=450;AC_Het_SAS=117;AC_Het_EUR_unrel=118;AC_Het_EAS_unrel=74;AC_Het_AMR_unrel=70;AC_Het_SAS_unrel=99;AC_Het_AFR_unrel=325;AC_Het=885;AC_Hom_EAS=10;AC_Hom_AMR=12;AC_Hom_EUR=18;AC_Hom_AFR=326;AC_Hom_SAS=14;AC_Hom_EUR_unrel=10;AC_Hom_EAS_unrel=10;AC_Hom_AMR_unrel=8;AC_Hom_SAS_unrel=14;AC_Hom_AFR_unrel=248;AC_Hom=380;HWE_EAS=0.381599;ExcHet_EAS=0.873868;HWE_AMR=0.633376;ExcHet_AMR=0.737205;HWE_EUR=0.724833;ExcHet_EUR=0.347322;HWE_AFR=0.495799;ExcHet_AFR=0.249716;HWE_SAS=1;ExcHet_SAS=0.584996;HWE=3.14711e-12;ExcHet=1\tGT\t1|0\t0|1\t0|0\t0|0\t0|1\t1|0\t0|0\t0|0\t0|0\t0|1\t0|0\n",
      "chr22\t49533849\t22:49533849:T:C\tT\tC\t.\t.\tAC=4429;AF=0.691599;CM=82.1428;AN=6404;AN_EAS=1170;AN_AMR=980;AN_EUR=1266;AN_AFR=1786;AN_SAS=1202;AN_EUR_unrel=1006;AN_EAS_unrel=1008;AN_AMR_unrel=694;AN_SAS_unrel=978;AN_AFR_unrel=1322;AF_EAS=0.691453;AF_AMR=0.590816;AF_EUR=0.612164;AF_AFR=0.843225;AF_SAS=0.63228;AF_EUR_unrel=0.61332;MAF_EUR_unrel=0.38668;AF_EAS_unrel=0.696429;MAF_EAS_unrel=0.303571;AF_AMR_unrel=0.597983;MAF_AMR_unrel=0.402017;AF_SAS_unrel=0.626789;MAF_SAS_unrel=0.373211;AF_AFR_unrel=0.846445;MAF_AFR_unrel=0.153555;AC_EAS=809;AC_AMR=579;AC_EUR=775;AC_AFR=1506;AC_SAS=760;AC_EUR_unrel=617;AC_EAS_unrel=702;AC_AMR_unrel=415;AC_SAS_unrel=613;AC_AFR_unrel=1119;AC_Het_EAS=237;AC_Het_AMR=231;AC_Het_EUR=283;AC_Het_AFR=244;AC_Het_SAS=270;AC_Het_EUR_unrel=225;AC_Het_EAS_unrel=204;AC_Het_AMR_unrel=163;AC_Het_SAS_unrel=223;AC_Het_AFR_unrel=179;AC_Het=1265;AC_Hom_EAS=572;AC_Hom_AMR=348;AC_Hom_EUR=492;AC_Hom_AFR=1262;AC_Hom_SAS=490;AC_Hom_EUR_unrel=392;AC_Hom_EAS_unrel=498;AC_Hom_AMR_unrel=252;AC_Hom_SAS_unrel=390;AC_Hom_AFR_unrel=940;AC_Hom=3164;HWE_EAS=0.244511;ExcHet_EAS=0.909474;HWE_AMR=0.576002;ExcHet_AMR=0.748507;HWE_EUR=0.154503;ExcHet_EUR=0.942214;HWE_AFR=0.37578;ExcHet_AFR=0.197113;HWE_SAS=0.429713;ExcHet_SAS=0.826231;HWE=3.37622e-05;ExcHet=0.999987\tGT\t0|1\t1|1\t1|0\t1|1\t1|1\t1|1\t1|1\t1|0\t1|0\t1|0\t1|1\n",
      "chr22\t50018612\t22:50018612:C:T\tC\tT\t.\t.\tAC=3128;AF=0.488445;CM=82.7628;AN=6404;AN_EAS=1170;AN_AMR=980;AN_EUR=1266;AN_AFR=1786;AN_SAS=1202;AN_EUR_unrel=1006;AN_EAS_unrel=1008;AN_AMR_unrel=694;AN_SAS_unrel=978;AN_AFR_unrel=1322;AF_EAS=0.35641;AF_AMR=0.457143;AF_EUR=0.494471;AF_AFR=0.485442;AF_SAS=0.640599;AF_EUR_unrel=0.501988;MAF_EUR_unrel=0.498012;AF_EAS_unrel=0.363095;MAF_EAS_unrel=0.363095;AF_AMR_unrel=0.456772;MAF_AMR_unrel=0.456772;AF_SAS_unrel=0.638037;MAF_SAS_unrel=0.361963;AF_AFR_unrel=0.476551;MAF_AFR_unrel=0.476551;AC_EAS=417;AC_AMR=448;AC_EUR=626;AC_AFR=867;AC_SAS=770;AC_EUR_unrel=505;AC_EAS_unrel=366;AC_AMR_unrel=317;AC_SAS_unrel=624;AC_AFR_unrel=630;AC_Het_EAS=283;AC_Het_AMR=242;AC_Het_EUR=334;AC_Het_AFR=433;AC_Het_SAS=286;AC_Het_EUR_unrel=267;AC_Het_EAS_unrel=246;AC_Het_AMR_unrel=167;AC_Het_SAS_unrel=232;AC_Het_AFR_unrel=326;AC_Het=1578;AC_Hom_EAS=134;AC_Hom_AMR=206;AC_Hom_EUR=292;AC_Hom_AFR=434;AC_Hom_SAS=484;AC_Hom_EUR_unrel=238;AC_Hom_EAS_unrel=120;AC_Hom_AMR_unrel=150;AC_Hom_SAS_unrel=392;AC_Hom_AFR_unrel=304;AC_Hom=1550;HWE_EAS=0.207481;ExcHet_EAS=0.113322;HWE_AMR=0.927541;ExcHet_AMR=0.588289;HWE_EUR=0.176996;ExcHet_EUR=0.097555;HWE_AFR=0.384164;ExcHet_AFR=0.832207;HWE_SAS=0.478317;ExcHet_SAS=0.238674;HWE=0.436649;ExcHet=0.795988\tGT\t1|1\t0|0\t1|1\t0|0\t1|0\t0|0\t1|0\t1|0\t0|0\t0|0\t0|1\n",
      "chr22\t50118978\t22:50118978:A:G\tA\tG\t.\t.\tAC=1873;AF=0.292473;CM=82.9223;AN=6404;AN_EAS=1170;AN_AMR=980;AN_EUR=1266;AN_AFR=1786;AN_SAS=1202;AN_EUR_unrel=1006;AN_EAS_unrel=1008;AN_AMR_unrel=694;AN_SAS_unrel=978;AN_AFR_unrel=1322;AF_EAS=0.165812;AF_AMR=0.397959;AF_EUR=0.421011;AF_AFR=0.208847;AF_SAS=0.318636;AF_EUR_unrel=0.421471;MAF_EUR_unrel=0.421471;AF_EAS_unrel=0.162698;MAF_EAS_unrel=0.162698;AF_AMR_unrel=0.409222;MAF_AMR_unrel=0.409222;AF_SAS_unrel=0.313906;MAF_SAS_unrel=0.313906;AF_AFR_unrel=0.2118;MAF_AFR_unrel=0.2118;AC_EAS=194;AC_AMR=390;AC_EUR=533;AC_AFR=373;AC_SAS=383;AC_EUR_unrel=424;AC_EAS_unrel=164;AC_AMR_unrel=284;AC_SAS_unrel=307;AC_AFR_unrel=280;AC_Het_EAS=156;AC_Het_AMR=228;AC_Het_EUR=323;AC_Het_AFR=289;AC_Het_SAS=255;AC_Het_EUR_unrel=260;AC_Het_EAS_unrel=136;AC_Het_AMR_unrel=160;AC_Het_SAS_unrel=205;AC_Het_AFR_unrel=226;AC_Het=1251;AC_Hom_EAS=38;AC_Hom_AMR=162;AC_Hom_EUR=210;AC_Hom_AFR=84;AC_Hom_SAS=128;AC_Hom_EUR_unrel=164;AC_Hom_EAS_unrel=28;AC_Hom_AMR_unrel=124;AC_Hom_SAS_unrel=102;AC_Hom_AFR_unrel=54;AC_Hom=622;HWE_EAS=0.370899;ExcHet_EAS=0.851466;HWE_AMR=0.510711;ExcHet_AMR=0.775934;HWE_EUR=0.254644;ExcHet_EUR=0.141617;HWE_AFR=0.543465;ExcHet_AFR=0.770816;HWE_SAS=0.57363;ExcHet_SAS=0.750826;HWE=0.00158026;ExcHet=0.999312\tGT\t1|1\t0|1\t0|0\t0|1\t0|0\t1|0\t1|1\t1|1\t0|1\t0|1\t0|0\n",
      "chr22\t50226183\t22:50226183:A:G\tA\tG\t.\t.\tAC=5496;AF=0.858214;CM=82.9843;AN=6404;AN_EAS=1170;AN_AMR=980;AN_EUR=1266;AN_AFR=1786;AN_SAS=1202;AN_EUR_unrel=1006;AN_EAS_unrel=1008;AN_AMR_unrel=694;AN_SAS_unrel=978;AN_AFR_unrel=1322;AF_EAS=0.845299;AF_AMR=0.695918;AF_EUR=0.776461;AF_AFR=0.989362;AF_SAS=0.894343;AF_EUR_unrel=0.776342;MAF_EUR_unrel=0.223658;AF_EAS_unrel=0.849206;MAF_EAS_unrel=0.150794;AF_AMR_unrel=0.690202;MAF_AMR_unrel=0.309798;AF_SAS_unrel=0.902863;MAF_SAS_unrel=0.097137;AF_AFR_unrel=0.985628;MAF_AFR_unrel=0.0143722;AC_EAS=989;AC_AMR=682;AC_EUR=983;AC_AFR=1767;AC_SAS=1075;AC_EUR_unrel=781;AC_EAS_unrel=856;AC_AMR_unrel=479;AC_SAS_unrel=883;AC_AFR_unrel=1303;AC_Het_EAS=153;AC_Het_AMR=192;AC_Het_EUR=215;AC_Het_AFR=17;AC_Het_SAS=109;AC_Het_EUR_unrel=175;AC_Het_EAS_unrel=132;AC_Het_AMR_unrel=131;AC_Het_SAS_unrel=83;AC_Het_AFR_unrel=17;AC_Het=686;AC_Hom_EAS=836;AC_Hom_AMR=490;AC_Hom_EUR=768;AC_Hom_AFR=1750;AC_Hom_SAS=966;AC_Hom_EUR_unrel=606;AC_Hom_EAS_unrel=724;AC_Hom_AMR_unrel=348;AC_Hom_SAS_unrel=800;AC_Hom_AFR_unrel=1286;AC_Hom=4810;HWE_EAS=1;ExcHet_EAS=0.580576;HWE_AMR=0.109063;ExcHet_AMR=0.961052;HWE_EUR=0.568241;ExcHet_EUR=0.752967;HWE_AFR=0.0922158;ExcHet_AFR=0.996488;HWE_SAS=0.285179;ExcHet_SAS=0.887435;HWE=2.38107e-10;ExcHet=1\tGT\t1|1\t1|0\t1|1\t0|1\t0|1\t1|1\t1|0\t1|1\t1|1\t1|1\t1|0\n",
      "chr22\t50577409\t22:50577409:T:C\tT\tC\t.\t.\tAC=1113;AF=0.173798;CM=83.5656;AN=6404;AN_EAS=1170;AN_AMR=980;AN_EUR=1266;AN_AFR=1786;AN_SAS=1202;AN_EUR_unrel=1006;AN_EAS_unrel=1008;AN_AMR_unrel=694;AN_SAS_unrel=978;AN_AFR_unrel=1322;AF_EAS=0.4;AF_AMR=0.159184;AF_EUR=0.042654;AF_AFR=0.115342;AF_SAS=0.190516;AF_EUR_unrel=0.0427435;MAF_EUR_unrel=0.0427435;AF_EAS_unrel=0.394841;MAF_EAS_unrel=0.394841;AF_AMR_unrel=0.15562;MAF_AMR_unrel=0.15562;AF_SAS_unrel=0.190184;MAF_SAS_unrel=0.190184;AF_AFR_unrel=0.121785;MAF_AFR_unrel=0.121785;AC_EAS=468;AC_AMR=156;AC_EUR=54;AC_AFR=206;AC_SAS=229;AC_EUR_unrel=43;AC_EAS_unrel=398;AC_AMR_unrel=108;AC_SAS_unrel=186;AC_AFR_unrel=161;AC_Het_EAS=280;AC_Het_AMR=126;AC_Het_EUR=50;AC_Het_AFR=186;AC_Het_SAS=189;AC_Het_EUR_unrel=39;AC_Het_EAS_unrel=238;AC_Het_AMR_unrel=84;AC_Het_SAS_unrel=160;AC_Het_AFR_unrel=143;AC_Het=831;AC_Hom_EAS=188;AC_Hom_AMR=30;AC_Hom_EUR=4;AC_Hom_AFR=20;AC_Hom_SAS=40;AC_Hom_EUR_unrel=4;AC_Hom_EAS_unrel=160;AC_Hom_AMR_unrel=24;AC_Hom_SAS_unrel=26;AC_Hom_AFR_unrel=18;AC_Hom=282;HWE_EAS=0.931552;ExcHet_EAS=0.570209;HWE_AMR=0.397578;ExcHet_AMR=0.856291;HWE_EUR=0.31406;ExcHet_EUR=0.902095;HWE_AFR=0.625006;ExcHet_AFR=0.341152;HWE_SAS=0.693108;ExcHet_SAS=0.377503;HWE=1.45871e-07;ExcHet=1\tGT\t0|0\t0|0\t0|0\t0|0\t0|0\t0|0\t0|0\t0|0\t0|0\t0|0\t0|0\n"
     ]
    }
   ],
   "source": [
    "# Check the final clean 1000GP vcf with the fingerprinting snps\n",
    "bcftools view -H data/1kGP.fingerprinting.vcf.gz | tail | cut -f 1-20"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}