Development V1 (#63)

* feat: basic setup

* feat: Setup NestJS API

* fix: preinstall script not working

* feat: Added nx support

* fix: start script not running

* feat: Added API to create project

* fix: swagger response

* feat: Added Update API and made DTO reusable

* feat: Added Delete project API

* fix: Updated module imports

* feat: Added Template repository

* fix: updated response to return _id

* feat: Added setup for templates API with get-templates route

* feat: Added create-template API

* feat: Added Update template API

* feat: Added valid mongodb id validation to params

* feat: Added delete API and added error-handling for not-found documents

* feat: Added authHeaderName to project

* feat: Added shared lib

* feat: Added tranformer to tranform code

* feat: Added column repository

* feat: Added template columns add & fetch routes

* feat: Switched create-template API to use projectId from param

* feat: Added valid callbackUrl validation

* feat: setup localstack and saved sample file to localstack while updating schema

* feat: Added Validation for env variables and Added env variable types

* feat: Added authentication support

* feat: Added file repository

* feat: Upload repository

* feat: mapping repository

* feat: Added file upload API

* feat: Completed CSV File Upload API

* feat: removed json2csv dependency and build csv by logic

* feat: Updated file information on upload

* feat: Updated file names to match others

* fix: removed comment

* fix: Added preinstall scripts to run after instalation to build packages

* feat: Removed columnKeys from column and added key and alternateKeys

* feat: temp commit

* feat: Finalized column mapping

* feat: Added Get Headings API

* feat: Moved template from body to param for upload

* feat: Deleted old validator and updated invalid message for new one

* feat: Moved mapping APIs to mapping controller

* feat: Finalize Mapping API

* feat: Added all-data file to upload and added api to retrive file while review

* fix: Updated column data templateId to _templateId

* fix: Fix columns not deleting issue

* feat: Added Review API

* feat: Removed additional properties from schema while processing

* feat: wrapped errors in list

* fix: Regex pattern validation

* fix: Updated fixes and Added validations

* feat: refactored and separated file mimetypes from supported types

* feat: Added facility to store reviewed data

* feat: saved invalid & valid data in files and proceed accordingly

* feat: Added stages validation to mapping API

* feat: Added field to whether proceed for invalid data or not

* feat: Reused get-upload usecase

* feat: Added not-found upload validation in mapping

* feat: Added confirm review API

* Create CODE_OF_CONDUCT.md

* feat: Added documentation

* Fixed documentation errors

Fixed documentation errors

* feat: Setup RabbitMQ

* feat: Added Storage service to @impler/shared

* feat: Implemented Processing functionality for upload file

* feat: Put types in seperate file for file-processing

* feat: Added webhook-log dal

* feat: Added facility to finalize upload

* Update README.md

"Setup" means system whereas "Set up" is used in the context of setting up a system. Fixed some other documentation errors too. @chavda-bhavik Please review.

* feat: Updated recursion flow for processing file

* feat: Updated end-review flow to start the process after review is finalized

* feat: Added validation error messages

* feat: Implemented review API to return data in pagination

* feat: Added embed App

* feat: Added widget App

* feat: Added react package to help import

* feat: Added packages

* feat: Applied modifications to embed lib

* feat: Renamed import-helper package to react

* fix: updated embed lint scripts

* feat: Organized widget structure

* feat: Added widget types and util functions in impler/shared

* feat: removed commnt and updated packages

* fix: Added missing types for styled-components in @impler/react

* feat: Updated docker script and removed unused backendUrl from embed

* feat: Updated @impler/react to to be used in packages

* feat: Added demo to @impler/react package

* feat: Removed default background color and added filter for projectId not defined

* feat: Added scripts

* feat: Added ReadME for @impler/react demo app

* fix: Removed unused dependencies for @impler/embed

* feat: Reorganized widget components

* feat: Get started with mantine and ModalContainer component

* fix: Removed styled-components dependency and used @emotion/react instead

* feat: Added Modal and Stepper Design System components with Storybook

* feat: Renamed ModalContainer to Modal

* feat: Added Button Component

* feat: Added gitignore

* feat: Added Dropzone component to Design System

* feat: Added File component to design system

* feat: Added Dropzone component

* feat: Added mapping Item component

* feat: Added Table component to design system

* feat: Added Pagination component to design-system

* feat: Added Empty Data Story for Table

* feat: Updated Modal and Stepper styles to meet the design needs

* feat: Added Select component

* feat: Updated select component to meet the design needs

* feat: Updated Dropzone, Select and Stepper component styles to meet the design needs

* feat: Added required prop and Responsivness to MappingItem

* feat: Updated Dropzone and File components to meed design needs:

* feat: Removed total data showing from Pagination

* feat: Added warning to Table

* feat: Added Warning icon and updated config

* feat: Added left-icon to Button and added Download icon

* feat: Implemented Phases static design

* feat: Updated confirmModal flow

* feat: Made Modal overlay transparent

* feat: Updated dropzone styles and showed file size properly

* feat: Updated Phases Modal design and made height to remain same

* feat: Updated scrollbar design

* feat: Added Complete Step

* feat: removed unwanted embed script and streamlined widget open flow

* feat: Made ParentCommand Utils reusable

* feat: Created separate package @impler/client for API Communication

* feat: Added Common API to check if User can access API or not

* feat: Updated valid check API to consider ProjectID and Template

* feat: Reused access-key names and wrapped axios calls for error handling

* feat: Updated layout architecture and Added authentication to Widget

* fix: Widget Build issue

* feat: Refactored code to reuse EventNames and functionality

* feat: Disabled widget logging warning for widget

* feat: made start script to start widget and api together

* feat: Added Prompt modal

* feat: Showed Prompt Modal before Closing and Reseting import

* feat: Added error to Dropzone component

* feat: Made Widget to load only when show is clicked

* feat: Updated select component to be used with ReactHookForm

* feat: Added headers support to HttpClient

* feat: Added Loading Overlay component

* feat: Added Reusable Upload and Template interfaces

* feat: Added APIs for UploadFile and GetTemplates

* feat: Added Loading Indicators to Footer Buttons

* feat: Added Phase1 basic implementation

* feat: Implemented Phase1 functionality

* fix: Fix issue of first value is always selected in select

* fix: Fix isssue of wrong URL creation

* feat: Added download sample functionality to widget

* feat: Added NoMagicNumber linting

* feat: Added commad to build packages before commit

* feat: Added notification functionality for download error

* fix: Fix issue of Template error not reseting after value changes

* feat: Removed building from pre-commit

* feat: Made consistency of using PhasesEnum in layout and heading

* Implementation Phase 2 (#41)

* fix: Issue of Footer actions not comming properly

* feat: Added properties to MappingItem to meet development needs

* feat: Made updates to App Context, fix warning in svg icon and Added Mapping APIs

* feat: Created new component for Phase2 columns heading

* feat: Implemented Mapping

* feat: Added loading to Phase 2 (#43)

* Fix in Upload Again and Reset

* feat: Made ReactQuery cache clear on upload reset

* feat: Made app to reset progress on close and Upload Again

* feat: Updated footer actions

* Implementation of Phase 3 & 4 (#46)

* feat: Added Review Interface and API

* feat: Updated error message formatting

* feat: Updated error variable configuration

* feat: Showed review data

* feat: Implemented Phase3

* feat: Implemented Phase 4

* feat: Added loading indicator in Review

* Export Review data (#50)

* feat: Added path to shared folder and made invalid data file public

* fix: Removed unwanted json2csv types

* feat: Added invalidCSVData file parameters to entity

* feat: Stored invalid data to csv file while reviewing

* feat: Added default page limit and page to review API

* feat: Moved Get-Template-Uploads to template controller

* feat: Added Get Upload API

* feat: Added Export data functionality

* feat: fix typo in confirmation with wrong data

* fix: Fix storybook run error

* fix: Consistency issue among Footer, Title and Stepper

* Fix Impler not defined & No Such Bucket errors (#51)

* feat: Updated embed import way

* fix: Formatted 'impler not defined' error

* feat: Made table headings sticky (#53)

* Readme Updates & Build workflow (#54)

* feat: Added Test Build workflow

* feat: Made nestjs available

* feat: Optimized and renamed workflow file

* feat: Updated version to 0.1.0

* fix: scripts in @impler/client (#55)

* Lerna Setup (#56)

* fix: Removed unwanted variables from queue-manager env files

* feat: Added setup-project-env script

* feat: Added learna

* v0.1.0

* feat: Added nvmrc

* feat: Fix clean & prebuild scripts

* Setup widget demo app (#57)

* feat: Moved widget-demo to app

* feat: Updated env setup script to setup env for widget-demo app

* Fixs & Updates (#58)

* feat: Made widget to not open window and renamed .env.example to .example.env in widget-demo

* refactor: removed unused demo folder from @impler/react

* feat: Made widget error more specified

* refactor: Organized components hierarchy for better management of state

* fix: Removed http-server from embed and added script for starting

* fix: Added guards for valid/invalid data not found

* feat: Added missing environment variables

* feat: Updated number formattion in Confirm and Complete modal

* Review phase Updates (#59)

* feat: Added missing environment variables ad env-validator to queue-manager

* feat: Handled edge case for invalid data not found

* Env Updates (#60)

* fix: Removed unused env variables

* feat: Removed unused start:widget-dev script

* feat: Added setup for widget env file

* Validation Tests (#61)

* test: Added AJVService test cases for isRequired, isEmail and Email types

* feat: Added RegExp test cases

* test: Added Number test cases

* test: Added test cases for Date

* test: Added test cases for Select type

* test: Added test cases for Any type

* fix: Removed unused test scripts and Added test script to test everything

* Packages publish updates (#62)

* feat: Updated Shared, client, react packages package.json to publish

* feat: Updated package.json and added env script for widget

* feat: Removed workspace prefix from packages usage

Co-authored-by: Rutam Prita Mishra <[email protected]>
Co-authored-by: Akash Kundu <[email protected]>

.commitlintrc.json

@@ -0,0 +1,28 @@
+{
+  "extends": ["@commitlint/config-conventional"],
+  "rules": {
+    "subject-case": [
+      2,
+      "always",
+      ["sentence-case", "start-case", "pascal-case", "upper-case", "lower-case"]
+    ],
+    "type-enum": [
+      2,
+      "always",
+      [
+        "build",
+        "chore",
+        "ci",
+        "docs",
+        "feat",
+        "fix",
+        "perf",
+        "refactor",
+        "revert",
+        "style",
+        "test",
+        "sample"
+      ]
+    ]
+  }
+}

.editorconfig

@@ -0,0 +1,11 @@
+# EditorConfig is awesome: https://EditorConfig.org
+
+# top-most EditorConfig file
+root = true
+
+# Unix-style newlines with a newline ending every file
+[*]
+end_of_line = lf
+insert_final_newline = true
+indent_style = space
+indent_size = 2

.eslintignore

@@ -0,0 +1,12 @@
+# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
+.grunt
+
+# node-waf configuration
+.lock-wscript
+
+# Dependency directory
+# https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git
+node_modules
+
+# OSX
+.DS_Store

.eslintrc.js

@@ -0,0 +1,95 @@
+module.exports = {
+  root: true,
+  extends: [
+    'airbnb-typescript',
+    'plugin:import/typescript',
+    'plugin:@typescript-eslint/recommended',
+    'prettier',
+    'plugin:prettier/recommended',
+    'plugin:promise/recommended',
+  ],
+  ignorePatterns: ['.eslintrc.js', '*.json', 'jest.config.js'],
+  plugins: ['import', 'promise', '@typescript-eslint', 'prettier'],
+  parser: '@typescript-eslint/parser',
+  settings: {
+    'import/parsers': {
+      '@typescript-eslint/parser': ['.ts', '.tsx'],
+    },
+  },
+  parserOptions: {
+    project: './tsconfig.json',
+    ecmaVersion: 2020,
+    sourceType: 'module',
+  },
+  rules: {
+    '@typescript-eslint/space-before-blocks': 'off',
+    '@typescript-eslint/lines-between-class-members': 'off',
+    'react/jsx-wrap-multilines': 'off',
+    'react/jsx-filename-extension': 'off',
+    'multiline-comment-style': ['error', 'starred-block'],
+    'promise/catch-or-return': 'off',
+    '@typescript-eslint/explicit-module-boundary-types': 'off',
+    '@typescript-eslint/no-unused-expressions': 'off',
+    '@typescript-eslint/explicit-function-return-type': 'off',
+    'react/jsx-closing-bracket-location': 'off',
+    '@typescript-eslint/no-var-requires': 'off',
+    'no-unused-vars': 'error',
+    '@typescript-eslint/no-unused-vars': ['error'],
+    'mocha/no-mocha-arrows': 'off',
+    '@typescript-eslint/default-param-last': 'off',
+    'no-return-await': 'off',
+    'no-await-in-loop': 'off',
+    'no-continue': 'off',
+    'no-console': 'warn',
+    'no-magic-numbers': 'error',
+    'no-prototype-builtins': 'off',
+    'import/no-cycle': 'off',
+    'class-methods-use-this': 'off',
+    '@typescript-eslint/no-use-before-define': 'off',
+    '@typescript-eslint/no-explicit-any': 1,
+    'no-restricted-syntax': 'off',
+    '@typescript-eslint/interface-name-prefix': 'off',
+    'no-underscore-dangle': 'off',
+    'import/prefer-default-export': 'off',
+    // A temporary hack related to IDE not resolving correct package.json
+    'import/no-extraneous-dependencies': 'off',
+    'react/jsx-one-expression-per-line': 'off',
+    'react/jsx-no-bind': 'off',
+    'lines-between-class-members': 'off',
+    'max-classes-per-file': 'off',
+    'react/react-in-jsx-scope': 'off',
+    'max-len': ['warn', { code: 140 }],
+    '@typescript-eslint/return-await': 'off',
+    'no-restricted-imports': [
+      'error',
+      {
+        patterns: ['@impler/shared/*', '@impler/dal/*', '!import2/good'],
+      },
+    ],
+    'padding-line-between-statements': [
+      'error',
+      { blankLine: 'any', prev: ['const', 'let', 'var'], next: ['if', 'for'] },
+      { blankLine: 'any', prev: ['const', 'let', 'var'], next: ['const', 'let', 'var'] },
+      { blankLine: 'always', prev: '*', next: 'return' },
+    ],
+    'id-length': ['error', { min: 2, exceptions: ['i', 'e', 'a', 'b', '_', 't'], properties: 'never' }],
+    '@typescript-eslint/naming-convention': [
+      'error',
+
+      { selector: 'enumMember', format: ['UPPER_CASE'] },
+      { selector: 'enum', format: ['PascalCase'], suffix: ['Enum'] },
+      { selector: 'class', format: ['PascalCase'] },
+      { selector: 'variableLike', format: ['camelCase', 'UPPER_CASE', 'PascalCase'], leadingUnderscore: 'allow' },
+      {
+        selector: 'interface',
+        format: ['PascalCase'],
+        prefix: ['I'],
+      },
+      {
+        selector: ['function'],
+        format: ['camelCase'],
+        leadingUnderscore: 'allow',
+      },
+    ],
+  },
+};

.github/workflows/test-build.yml

@@ -0,0 +1,50 @@
+# This is a basic workflow to help you get started with Actions
+
+name: Test Build is happening
+
+# Controls when the action will run. Triggers the workflow on push or pull request
+# events but only for the master branch
+on:
+  push:
+    branches: [ "main" ]
+  pull_request:
+    # The branches below must be a subset of the branches above
+    branches: [ "main", "next", "development" ]
+  schedule:
+    - cron: '25 2 * * 4'
+
+# A workflow run is made up of one or more jobs that can run sequentially or in parallel
+jobs:
+  # This workflow contains a single job called "build"
+  build:
+    environment: Development
+    if: "!contains(github.event.head_commit.message, 'build skip')"
+    # The type of runner that the job will run on
+    runs-on: ubuntu-latest
+    timeout-minutes: 80
+
+    # Steps represent a sequence of tasks that will be executed as part of the job
+    steps:
+      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
+      - uses: actions/checkout@v2
+      - name: Setup kernel for react native, increase watchers
+        run: echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
+      - uses: actions/setup-node@v2
+        with:
+          node-version: '16.15.1'
+
+      - name: Cache pnpm modules
+        uses: actions/cache@v2
+        with:
+          path: ~/.pnpm-store
+          key: ${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
+          restore-keys: |
+            ${{ runner.os }}-
+
+      - name: Nestjs
+        run: npm i -g @nestjs/cli
+
+      - uses: pnpm/[email protected]
+        with:
+          version: 7.9.4
+          run_install: true

.gitignore

@@ -0,0 +1,5 @@
+node_modules
+dist
+build
+
+.env

.husky/commit-msg

@@ -0,0 +1,4 @@
+#!/usr/bin/env sh
+. "$(dirname -- "$0")/_/husky.sh"
+
+npx --no -- commitlint --edit "$1"

.husky/pre-commit

@@ -0,0 +1,4 @@
+#!/usr/bin/env sh
+. "$(dirname -- "$0")/_/husky.sh"
+
+pnpm lint-staged

.npmrc

@@ -0,0 +1,2 @@
+auto-install-peers=true
+strict-peer-dependencies=false

.nvmrc

@@ -0,0 +1 @@
+16.15.1

.prettierignore

@@ -0,0 +1,2 @@
+# package.json is formatted by package managers, so we ignore it here
+package.json

.prettierrc

@@ -0,0 +1,11 @@
+{
+  "printWidth": 120,
+  "trailingComma": "es5",
+  "singleQuote": true,
+  "semi": true,
+  "tabWidth": 2,
+  "quoteProps": "as-needed",
+  "jsxSingleQuote": false,
+  "arrowParens": "always",
+  "endOfLine": "lf"
+}

.vscode/settings.json

@@ -0,0 +1,11 @@
+{
+    "typescript.tsdk": "node_modules/typescript/lib",
+    "npm.packageManager": "pnpm",
+    "editor.defaultFormatter": "dbaeumer.vscode-eslint",
+    "editor.formatOnSave": true,
+    "eslint.format.enable": true,
+    "editor.codeActionsOnSave": {
+      "source.fixAll": true
+    },
+}
+

CODE_OF_CONDUCT.md

@@ -0,0 +1,128 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+[email protected].
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
+
+Community Impact Guidelines were inspired by [Mozilla's code of conduct
+enforcement ladder](https://github.com/mozilla/diversity).
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see the FAQ at
+https://www.contributor-covenant.org/faq. Translations are available at
+https://www.contributor-covenant.org/translations.