Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LR-546 L3 level security code fixes #45

Merged
merged 62 commits into from
May 25, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
a6f39ae
LR-546 PII code implementation for data-products
Hari-stackroute May 5, 2023
d5231e7
Merge remote-tracking branch 'upstream/release-5.3.0' into LR-546
Hari-stackroute May 5, 2023
43aa464
LR-546 code changes for PII in reports
Hari-stackroute May 6, 2023
e10bf7f
LR-546 code changes for PII in reports
Hari-stackroute May 8, 2023
59bf476
LR-546 removed pom changes
Hari-stackroute May 8, 2023
0a88c78
LR-546 removed unnecessary test file changes
Hari-stackroute May 8, 2023
d36c914
LR-546 removed duplicate code
Hari-stackroute May 8, 2023
8accd97
LR-546 tested zip functionality
Hari-stackroute May 8, 2023
cafd0b6
LR-546 removed token details from conf
Hari-stackroute May 9, 2023
5f28dfd
LR-546 added test cases
Hari-stackroute May 10, 2023
c0dcb4b
LR-546 removed access-token from request
Hari-stackroute May 11, 2023
60d804d
LR-546 removed unused methods from HttpUtil
Hari-stackroute May 11, 2023
c686cfb
Merge remote-tracking branch 'upstream/release-5.3.0' into LR-546
Hari-stackroute May 12, 2023
212cae2
LR-546 code changes against review points
Hari-stackroute May 13, 2023
48d3faf
LR-546 code changes against review points
Hari-stackroute May 15, 2023
cb93e78
LR-546 code changes against review points
Hari-stackroute May 15, 2023
ee9f682
LR-546 code changes against review points
Hari-stackroute May 15, 2023
3adf311
LR-546 test-case fix
Hari-stackroute May 15, 2023
ebcc36f
LR-546 test-case fix-1
Hari-stackroute May 15, 2023
31b4951
LR-546 test-case fix-2
Hari-stackroute May 15, 2023
92b54eb
LR-546 code fixes for PII changes
Hari-stackroute May 16, 2023
49ce24d
LR-546 code fixes for PII changes-1
Hari-stackroute May 16, 2023
942dce4
LR-546 code fixes for PII changes-2
Hari-stackroute May 16, 2023
aec99b0
LR-546 level codes changed
Hari-stackroute May 16, 2023
aef04d0
LR-546 removed level and orgid details from JobRequest
Hari-stackroute May 16, 2023
9ee8739
LR-546 removed level and orgid details from JobRequest-1
Hari-stackroute May 16, 2023
fdbb791
LR-546 ignoring test-cases
Hari-stackroute May 16, 2023
b789597
LR-546 modified Encrypt util
Hari-stackroute May 17, 2023
0954e49
LR-546 expection with tenant preferance endpoint configuration
Hari-stackroute May 17, 2023
c1c50bf
LR-546 expection with tenant preferance endpoint configuration-1
Hari-stackroute May 17, 2023
e07afeb
LR-546 expection with tenant preferance endpoint configuration-2
Hari-stackroute May 17, 2023
8ee1d96
LR-546 expection with tenant preferance endpoint configuration-3
Hari-stackroute May 17, 2023
fcdbef1
LR-546 dev env configuration changes for spark
Hari-stackroute May 18, 2023
f0bd47a
LR-546 added logs
Hari-stackroute May 18, 2023
b3d08bb
LR-546 added logs-1
Hari-stackroute May 18, 2023
71fd18e
LR-546 added logs-2
Hari-stackroute May 18, 2023
a01d01b
LR-546 added logs-3
Hari-stackroute May 18, 2023
31f15d2
LR-546 added logs-4
Hari-stackroute May 18, 2023
6108409
LR-546 added logs-5
Hari-stackroute May 18, 2023
f4ab3c2
LR-546 added logs-6
Hari-stackroute May 18, 2023
ab105b5
LR-546 added logs-7
Hari-stackroute May 18, 2023
cdd503c
LR-546 added logs-8
Hari-stackroute May 18, 2023
6feefe6
LR-546 L3 level eccryption throwing errors
Hari-stackroute May 19, 2023
8735fec
LR-546 L3 level eccryption throwing errors-1
Hari-stackroute May 19, 2023
ce2357a
LR-546 L3 level eccryption throwing errors-2
Hari-stackroute May 19, 2023
d09d5a1
LR-546 config changes
Hari-stackroute May 19, 2023
2905ddf
LR-546 commented test-cases
Hari-stackroute May 21, 2023
865efe9
Delete BaseCollectionExhaustJob1.scala
Hari-stackroute May 21, 2023
c416c1a
LR-546 ignored test-cases
Hari-stackroute May 21, 2023
73dd8ca
Merge branch 'LR-546' of https://github.com/Hari-stackroute/data-prod…
Hari-stackroute May 21, 2023
bc801ab
LR-546 ignored test-cases-1
Hari-stackroute May 21, 2023
beaca03
LR-546 ignored test-cases-2
Hari-stackroute May 21, 2023
b380282
LR-546 ignored test-cases-2
Hari-stackroute May 21, 2023
0516264
LR-546 ignored test-cases-3
Hari-stackroute May 21, 2023
e87750c
LR-546 ignored test-cases-4
Hari-stackroute May 21, 2023
8fae531
Merge remote-tracking branch 'upstream/release-5.3.0' into LR-546
Hari-stackroute May 22, 2023
f6cbb12
LR-546 added new line between ecrypted text and file content
Hari-stackroute May 22, 2023
cc02b5a
LR-546 L4 level changes
Hari-stackroute May 23, 2023
07ee5e8
LR-546 L4 level changes-1
Hari-stackroute May 23, 2023
f4a61ec
LR-546 L4 level changes-2
Hari-stackroute May 23, 2023
875fd2a
LR-546 L3 level changes
Hari-stackroute May 25, 2023
a930400
LR-546 L3 level changes-1
Hari-stackroute May 25, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
LR-546 level codes changed
  • Loading branch information
Hari-stackroute committed May 16, 2023
commit aec99b0dd19cd7e258bb34d2dc0c88f03ba48e33
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,14 @@ object DataSecurityUtil {

def getSecuredExhaustFile(level: String, orgId: String, channel: String, csvFile: String, encryptedKey: String, storageConfig: StorageConfig): Unit = {
level match {
case "L1" =>
case "PLAIN_DATASET" =>

case "L2" =>
case "PASSWORD_PROTECTED_DATASET" =>

case "L3" =>
case "TEXT_KEY_ENCRYPTED_DATASET" =>
val keyForEncryption = DecryptUtil.decryptData(encryptedKey)
encryptionFile(null, csvFile, keyForEncryption, level)
case "L4" =>
case "PUBLIC_KEY_ENCRYPTED_DATASET" =>
val exhaustEncryptionKey = getExhaustEncryptionKey(orgId, channel)
val downloadPath = Constants.TEMP_DIR + orgId
val publicPemFile = httpUtil.downloadFile(exhaustEncryptionKey, downloadPath)
Expand Down Expand Up @@ -123,7 +123,7 @@ object DataSecurityUtil {
var localPath = ""
var tempDir = ""
var resultFile = ""
if(level == "L2") {
if(level == "PASSWORD_PROTECTED_DATASET") {
tempDir = AppConf.getConfig("spark_output_temp_dir") + request.request_id + "/"
val path = Paths.get(url);
objKey = url.replace(filePrefix, "");
Expand All @@ -146,7 +146,7 @@ object DataSecurityUtil {
// $COVERAGE-ON$
val zipPath = localPath.replace("csv", "zip")
val zipObjectKey = objKey.replace("csv", "zip")
if (level == "L2") {
if (level == "PASSWORD_PROTECTED_DATASET") {
val zipLocalObjKey = url.replace("csv", "zip")

request.encryption_key.map(key => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ class TestEncryptFileUtil extends BaseSpec {
val request = Map("popularity" -> 1);
try {
val file = new File("src/test/resources/reports/public.pem")
EncryptFileUtil.encryptionFile(file ,"src/test/resources/reports/ap.csv","","L1")
EncryptFileUtil.encryptionFile(file ,"src/test/resources/reports/ap.csv","","PLAIN_DATASET")
} catch {
case ex: UnirestException => Console.println(s"Invalid Request for url: ${url}. The job failed with: " + ex.getMessage)
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -327,7 +327,7 @@ class TestUserInfoExhaustJob extends BaseReportSpec with MockFactory with BaseRe
//Unit test case for save and update requests
it should "execute the update and save request method" in {
implicit val fc = new FrameworkContext()
val jobRequest = JobRequest("'do_1131350140968632321230_batch-001:channel-01'", "123", "userinfo-exhaust", "SUBMITTED", """{\"batchId\": \"batch-001\"}""", "user-002", "channel-01", System.currentTimeMillis(), None, None, None, None, Option(""), Option(0), Option("test-123"), None, Option("0123456789"), Option("L1"))
val jobRequest = JobRequest("'do_1131350140968632321230_batch-001:channel-01'", "123", "userinfo-exhaust", "SUBMITTED", """{\"batchId\": \"batch-001\"}""", "user-002", "channel-01", System.currentTimeMillis(), None, None, None, None, Option(""), Option(0), Option("test-123"), None, Option("0123456789"), Option("PASSWORD_PROTECTED_DATASET"))
val req = new JobRequest()
val jobRequestArr = Array(jobRequest)
val storageConfig = StorageConfig("local", "", outputLocation)
Expand Down