add ability to import users

[sbwalker]

No description provided.

[thabaum]

Thank you!

[thabaum]

Is there the potential to also have the ability to import user files from a zip that can contains the CSV file for each user and holds a folder format within each created user profile user folder if exists or something of this nature if you can follow me here?

I seen this injects the file service and after reviewing while in this migration of users I thought I would ask.

Also I see this says ability to import users, but looking at logic it seems it would potentially be a CSV file per user? What I am asking is does this import many users from one file or one user imported per CSV file uploaded?

I will test this feature out soon let you know how it goes once I fully understand the template format to be used.

Thanks again!

[sbwalker]

@thabaum a CSV file is intended to contain many users - the purpose of this feature is to bulk load users. A template is provided so the user knows the expected import file format (Oqtane.Server/wwwroot/users.csv) with a button in the UI to download the template

Email - required
Username - defaults to Email if not provided
DisplayName - defaults to Username if not provided
Roles - comma delimited list of role names (note that default role of Registered User does not need to be specified)
Profile Fields - list of Profile fields defined for a site (these are dynamic - the template includes the default Profile fields however these can obviously be customized)

there is no need for a ZIP file

[thabaum]

OK, Thanks for additional clarity.

What I was hoping to ask and sorry for any confusion:

As an enhancement could we potentially down the road add a feature to also upload user files, and this way we can programmatically generate user data in the CSV and per line also have a relating folder named by line associated with the user ie 1,2,3,4,5.....110000... to upload the user files as well such as the ProfileImageId. Potentially two ways to upload new users as it may be overkill for a lot of sites. Some site migrations may have user files and it would be nice to be able to organize them into the users' My Folder while importing. for example: PhotoFileId that can be uploaded at same time and set for imported users would be great enhancement along with ability to add the user files they may need included with their profile My Folder and be able to organize them such as folder paths My Folder\Photos or My Folder\Documents for each users file types they may import along with user data.

Now that I have tested this new enhancement I can give some feedback here as well.

On first attempt I wanted to test some "incorrect" CSV template data and it said the user was imported, I did not see anything in the database added.

Here was the data I presented to it without studying the code further just guessing what could be potentially the proper data to present. Just this single line thinking maybe it had a default set of fields as shown to import.

[email protected],Jim,Jim,Administrator,Jim,Host,,,,,,

This CSV uploaded and message stated importing the user successfully, but no user data imported into the User database table.

Then I did 5 users following using this:

Email,Username,DisplayName,Roles,FirstName,LastName,Street,City,Region,Country,PostalCode,Phone
[email protected],user1,User One,Role1,John,Doe,123 Main St,New York,NY,USA,10001,555-123-4567
[email protected],user2,User Two,Role2,Jane,Smith,456 Elm St,Los Angeles,CA,USA,90001,555-987-6543
[email protected],user3,User Three,Role1,Robert,Johnson,789 Oak St,Chicago,IL,USA,60001,555-789-1234
[email protected],user4,User Four,Role3,Alice,Brown,101 Pine St,San Francisco,CA,USA,94101,555-321-7890
[email protected],user5,User Five,Role2,Michael,Williams,555 Cedar St,Boston,MA,USA,02101,555-555-5555

All users where generated and created the roles not currently present for each user.

I then added the fields at the top of the Jim file and uploaded/imported it successfully into the database using:

Email,Username,DisplayName,Roles,FirstName,LastName,Street,City,Region,Country,PostalCode,Phone
[email protected],Jim,Jim,Administrator,Jim,Host,,,,,,

Oqtane should not have told me it imported a user when it did not (when I didnt add the fields to the first line). A check to see if the user exists after the user has been imported to verify and report any inconsistencies in the template such as fields missing if expected?

Maybe a "generate report" option to show if any users are duplicates so they didnt get imported due to user already existing.
In the event there are duplicates Username setting options maybe can

• generate a unique number at the end as an option to handle them if checked.
• update the user with updated information from file
• ignore and move to next user.
• option to send users an email notification to inform the user to request a password reset to login or send them the temporary password.
• Also upon password reset request if user email has not been verified and they click on a password reset link, this should verify the email account if it does not already or an option to set it to do so in my opinion of coarse.

Trying to add to multiple roles using a space but it goes by names so it just adds a long user name.

I was able to accomplish this by adding a line for each role while also noticing the last entry will update the profile information with the last updated info entry while testing behavior(which would be expected). which would be 555333 Cedar Ln for Street example

Email,Username,DisplayName,Roles,FirstName,LastName,Street,City,Region,Country,PostalCode,Phone
[email protected],user14,User 14,Role1,Dave,Johns,5553 Cedar Ln,Phoenix,AZ,USA,85001,555-999-9999
[email protected],user14,User 14,Role2,Dave,Johns,5553 Cedar Ln,Phoenix,AZ,USA,85001,555-999-9999
[email protected],user14,User 144,Role3,Dave,Johns,555333 Cedar Ln,Phoenix,AZ,USA,85001,555-999-9999

So if you have some users with more than one role a new line created for each role using this template. And you can add "Roles" as an additional field for more than one,but sometimes you may have 5 or more roles and others only have 1 or 2 if any. Not sure if this is suggested way but this approach does add the user to additional roles and overwrites field settings for that user. Which maybe you do not want in the event you are importing from a couple other systems that may have duplicate usernames but are different users.

Maybe a test can be ran to show the end results along with any conflicts and options to handle them to then a confirmation to apply them.
Email,Username,DisplayName,Roles,Roles,FirstName,LastName,Street,City,Region,Country,PostalCode,Phone

For 2 roles adding Roles twice should work as well but you might not know how many roles you have for the users so adding a line for each role to add works.

I was trying "Role1, Role2, Role3" and ,"Role1 Role2 Role3" and ...,"Role1" "Role2" "Role3",... which gave results that shifted things to each property or one long named role as expected but thought maybe it could be a way to handle the case of 'Roles". Just notes for anyone attempting this might help cut the chase a little. It would be a challenge for the script to possibly handle multiple roles in one line but maybe this is desirable enhancement to check if the property is "Roles" and handle that differently than the others to allow adding more than one on one line.

This is what I was able to come up with for different results.

So for a recap these are some concerns:

1. Issue of a successful user import message with no error if user is not imported due to CSV file improperly formatted (missing properties first line for example)

2. Adding users with multiple roles

3. Updating existing users or not (if they already exist maybe the current information is more up-to-date than what is being imported) As in if the user exists maybe you dont want to update the profile information and reset the users password or whatever this feature might do logically for a scenario to consider.

4. Import/Update button for users or something stating it will update existing users and a message stating: "Imported or Updated Successfully" for clarity purposes to let a host know this will also update users that currently exist.

5. Logic to upload files for users that maybe are included in a zip file you can also upload that puts all the files for a user in their profile folder and sets the folder number in relation to the CSV line and drops everything after ./1/{folders}/{files} of the zip file into My Folder/{folders}/{files} for the user and a way to set one of the file names as the PhotoFileId it gets after uploading to the users directory accordingly.

Have any questions for clarity please let me know.

Great enhancement this is huge, thank you!

[sbwalker]

@thabaum thank you for testing a variety of different scenarios and providing feedback. Please note that the goal is to provide a very simple import capability without extensive features. I view this as an opportunity for a developer to create a more powerful user import module and make it available to the community - either open source or commercial. The responsibility of the platform is to enable developers to create more advanced solutions - not to provide all of the advanced solutions itself. Some specific feedback to the items raised:

1. Issue of a successful user import message with no error if user is not imported due to CSV file improperly formatted (missing properties first line for example)

I will take a look. Import issues are supposed to be logged to the Event Log, but the behavior you describe sounds like a problem

2. Adding users with multiple roles

the parsing logic is removing double quotes incorrectly. I will resolve this issue.

3. Updating existing users or not (if they already exist maybe the current information is more up-to-date than what is being imported) As in if the user exists maybe you dont want to update the profile information and reset the users password or whatever this feature might do logically for a scenario to consider.

the import is designed to handle both insert and update by design. This allows for bulk updates in addition to bulk inserts. Passwords are only set on inserts. I do not intend to add any additional "options" at this time - this will be a default behavior of the user import.

4. Import/Update button for users or something stating it will update existing users and a message stating: "Imported or Updated Successfully" for clarity purposes to let a host know this will also update users that currently exist.

I can add clarification to the messaging

5. Logic to upload files for users that maybe are included in a zip file you can also upload that puts all the files for a user in their profile folder and sets the folder number in relation to the CSV line and drops everything after ./1/{folders}/{files} of the zip file into My Folder/{folders}/{files} for the user and a way to set one of the file names as the PhotoFileId it gets after uploading to the users directory accordingly.

this is definitely out of scope and will not be included in the user import. Since there are major enhancements coming related to folder providers it is important to not add more features which depend on the current implementation of file storage. This capability is an example of something which could be included in a more advanced third party user import module.

[sbwalker]

@thabaum #3283 addresses some of your feedback above:

1. Uses tab delimited rather than CSV format for import file
2. Provides better help text
3. Provides more detailed user feedback
4. Makes all columns optional except for email address
5. Updates DisplayName on update scenarios
6. Handles comma delimited roles properly
7. Logs all import events to Event Log