TWE: IS21: RP001: Intern Exit Interview Recordings: De-identify: Intern 003

[pandanista]

Dependencies

- [ ] All the user interview issues have been created IS21 didn't set up those issues, but all interviews were conducted. See TWE: IS21: RP001: Intern Exit Interview Roll Call and Session Table
- [ ] Interview Recording Upload Form needs to be created The interviews were conducted before we set up the upload form, so it doesn't apply in RP001's case

• Interview recordings have been uploaded and moved into Internship - PII's "My Drive"
• Interview transcripts have been created

Overview

We need to clean up and de-identify RP001's Intern Exit interview transcripts so that we can move on to the data analysis and insights generation phase.

During these tasks, you will be asked to add links to the Resources/Instructions section, and that section is at the bottom of this issue (go view it now).

Another tip is to use split browser windows side by side when working on any GitHub issue, so you can edit the issue on one window, and check the resources or click through links on the other window to avoid a lot of back and forth. Make sure to refresh both windows to reflect any changes you made, including updating the issue or checking the boxes on one of them.

Action Items

• UX lead adds the assignee to the Internship - PII's "My Drive" as a Contributor so that they have access to Internship - PII's My Drive where the recordings and transcripts with PII are stored.
  • UX lead accesses the Internship - PII's "My Drive"
  • Choose "Manage members", which is located towards the top right side of the browser
  • Enter the assignee's email address and select the role as "Contributor"
  • Confirm with the assignee that they have access to the Internship - PII's "My Drive"

Customize Resource Links

• Customize Resource for Wiki Page Link

  • Go to the wiki page: Research Output Overview (Resources # 1.01)
  • Find relevant wiki page for RP001
  • Copy the link for the wiki page
  • In Resources # 2.01, place the link you just copied between parentheses at the end of the line with no space in between the right bracket ] and the left parenthesis (, so it turns into a hyperlink
  • Choose "Update comment" in Github and make sure all the checkboxes above have been checked

• Customize Resource for this Research Plan's Google Drive Folder

  • Open the Google Drive's Research by Type Folder (Resources # 1.02)
  • Choose the Intern folder
  • Choose the RP001 folder
  • Copy the link of the RP001 folder
    1. Choose the three vertical dots on the right side of the RP001 folder
    2. Choose "Share"
    3. Choose "Get link"
    4. Choose "Copy Link" and "Done"
  • In Resources # 2.02, place the link you just copied between parentheses at the end of the line with no space in between the right bracket ] and the left parenthesis (, so it turns into a hyperlink
  • Choose "Update comment" in Github and make sure all the checkboxes above have been checked

• Customize Resource for Interview Recordings and Transcriptions Tracking Sheet

  • Open the Research Plan's Google Drive folder in the Internship's shared drive (Resource # 2.02)
  • Locate the TWE: IS21: RP001: Intern Exit Interview Recordings and Transcriptions Tracking Sheet in the folder
  • Copy the link of TWE: IS21: RP001: Intern Exit Interview Recordings and Transcriptions Tracking Sheet
    1. Choose the three vertical dots on the top right side of the file
    2. Choose "Share"
    3. Choose "Get link"
    4. Choose "Copy Link" and "Done"
  • In Resources # 2.03, place the link you just copied between parentheses at the end of the line with no space in between the right bracket ] and the left parenthesis (, so it turns into a hyperlink
  • Choose "Update comment" in Github and make sure all the checkboxes above have been checked

• Customize Resource for Interview Recording Folder stored in the Internship - PII's My Drive

  • Log into your Google account so you will be able to access the Internship - PII's 'My Drive' in the next steps
  • Choose Shared drives > Internship - PII > Internship -PII
  • Choose TWE Interview Recordings by Plan # folder in Internship - PII's My drive (Resources # 1.03)
  • Locate the video recording folder for RP001 inside the folder TWE Interview Recordings by Plan #
  • Copy the link of the video recording folder for RP001
    1. Choose the three vertical dots on the right side of the folder
    2. Choose "Share"
    3. Choose "Get link"
    4. Choose "Copy Link" and "Done"
  • In Resources # 2.04, place the link you just copied between parentheses at the end of the line with no space in between the right bracket ] and the left parenthesis (, so it turns into a hyperlink
  • Choose "Update comment" in Github and make sure all the checkboxes above have been checked

• Customize Resource for the De-identified Transcripts Folder in the Shared Drive 

  • Go to RP001 folder (Resources # 2.02)
  • Locate the RP001 De-identified Transcripts folder
  • Copy the link of the folder
    1. Choose the three vertical dots on the right side of the folder
    2. Choose "Share"
    3. Choose "Get link"
    4. Choose "Copy Link" and "Done"
  • In Resources # 2.05, place the link you just copied between parentheses at the end of the line with no space in between the right bracket ] and the left parenthesis (, so it turns into a hyperlink
  • Choose "Update comment" in GitHub and make sure all the checkboxes above have been checked

• Customize Resources for the Recording and Transcript you are Assigned to

  • Check the title of this issue to identify the participant number, which comes after De-identify:
  • Open the TWE: IS21: RP001: Intern Exit Interview Recordings and Transcriptions Tracking Sheet (Resources # 2.03)
  • Locate the recording video in .mp4 format that matches the participant number in Column B of the tracking sheet
  • Copy the link of the recording that matches the participant number in Column B of the tracking sheet
  • In Resources # 2.06, place the link you just copied between parentheses at the end of the line with no space in between the right bracket ] and the left parenthesis (, so it turns into a hyperlink
  • Locate the transcript in .txt format that matches the participant number in Column C of the tracking sheet
  • Copy the link of the .txt file that matches the participant number in Column C of the tracking sheet
  • In Resources # 2.07, place the link you just copied between parentheses at the end of the line with no space in between the right bracket ] and the left parenthesis (, so it turns into a hyperlink
  • Prepare for cleaning up and de-identification by converting the .txt file into a Google Doc
    1. Click on the link in Column C Transcription Link that matches the participant number (Resources # 2.07)
    2. Choose Open with Google Docs 
    3. A Google Doc is generated in a new window with the same .txt file name
    4. The Google Doc is now saved into the same folder with the corresponding video and .txt file
    5. Open the RP001 Interview Recording folder (Resources # 2.04)
    6. Open the corresponding folder for Intern 003 and check to make sure a Google Doc transcript has been created
  • Make a copy of the Google Doc that was just generated and rename it
    1. Choose the three vertical dots on the top right side of the file
    2. Choose "Make a copy"
    3. A copy of the Google Doc is generated in the same folder
    4. Copy

    To be de-identified-
    

    1. Go back to the folder where a copy of the Google doc was created
    2. Choose the three vertical dots on the top right side of the file
    3. Choose "Rename"
    4. Remove Copy of from the file name when the Rename text box pops up
    5. Paste what you copied into the beginning of the file name text box
    6. Choose "Ok"
  • Copy the link of the "to be de-identified" Google Doc transcript you just created
    1. Choose the three vertical dots on the top right side of the file
    2. Choose "Share"
    3. Choose "Get link"
    4. Choose "Copy Link" and "Done"
  • In Resources # 2.08, place the link you just copied between parentheses at the end of the line with no space in between the right bracket ] and the left parenthesis (, so it turns into a hyperlink
  • Choose "Update comment" in Github and make sure all the checkboxes above have been checked

Clean up and de-identify the transcript

• Open the Google Doc transcript if it is not open yet (Resource # 2.08)
• Listen to the recording to get yourself familiar with the transcript
• Go to Resources # 1.04 to get a refresher on the cleaning up and de-identification process. This is particularly important if it is your first time cleaning up an interview transcript.
  • Check Resources # 1.04.01 to understand when and how to use brackets [ ] and parenthesis ( ) when working on cleaning up the transcript, so you can learn to use them properly.
  • Watch the videos in Resources # 1.04.02 as they walk you through how to clean up and de-identify the transcript
  • Read Resources# 1.04.03 and 1.04.04 for more best practices to clean up and de-identify an interview transcript
  • Follow the following guidelines on the transcript formatting:
    • Use the 'in-line' format as seen here in Intern 001's transcript
    • Use 6 digit format for all the timestamps, for example, 01:01:01 as hour one/minute one/second one
    • Use "Interviewer" and "Intern ###" throughout the transcript. See an example here in Intern 001's transcript.
      • If there were more than two people recorded on the transcript, name the third person based on the interview set-up or context, such as Program Manager, Note Taker, etc. If you are not sure what to name them, ask leads on the project to clarify.
    • If there are other numerical values in the transcript, transcribe them using texts instead of numbers so we can easily scan the transcript using timestamps as they are in numbers. For example, if a time such as '6:30' is mentioned, type out 'six thirty' when cleaning up the transcript. Another example is if the interviewee mentioned he mainly sought help from '3 other peers', type out 'three' instead of using the number '3'.
• Listen to the interview recording again and clean up the inaccuracies in the transcript as you read along
  • Separate the texts based on speakers and timestamps (timestamps should match the video file timestamps in case a researcher needs to go back and double-check the original video)
  • Add any missed words if the transcribing process missed any or to provide more context
  • Write exactly what they say, even if it is grammatically incorrect or doesn’t make much sense. You can use brackets to correct them if needed, but no need to erase them.
  • Edit any typos and misidentified words in the transcript because the transcribing software is not 100% accurate
  • Deleted repeated words
  • Keep interjections like “hmm”, “uh”, “woah”, “yeah”, “okay”, “um”, “ohh”, “mmm” because they often contain emotions, reactions, and meanings (e.g. Mmmm [no], Mhmm [yes])
  • For verbal filler words such as "um" and "uh", keep a balance between authenticity and readability. See Resources # 1.04.04 for more on this topic.
    • When the verbal fillers are distractions and don't serve any purpose, you may remove them.
    • When the verbal fillers can indicate the interviewee's emotions or thoughts. In this case, keep the verbal fillers and provide context to demonstrate the emotions or thoughts. E.g., (The participant hesitated for a while before coming up with the answer).
  • Where needed, add context in brackets (for important information) or parentheses (for supplemental information) so that a reader can understand what was happening without watching the video. E.g., he [the mentor] was helpful; or (steps off camera).
• Read through the transcripts again and make edits to de-identify any personal information
  • Search for the participant's name
  • Replace with their participant number, i.e. Intern 001
  • Search for unknown speakers in the transcript
  • Replace unknown speaker with UXR number or other participant type/number
  • Read through the transcripts to search for any identifiable information in the interview
  • Replace any personally identifiable information with non-identifiable terms. For example, if someone's name is mentioned, we should use their roles inside square brackets so the reader knows this information was redacted (e.g., [team member], [peer], [mentor], [manager]). If their specific school is mentioned, we should redact that info with either [high school] or [college].
  • If they mentioned any specific issues they worked on, make sure to remove the issue numbers and rephrase the issues they worked on
• Read through the edited transcripts again
  • Focus on punctuation, readability, and formatting.
  • One recommendation is to install LanguageTool Chrome Extension (see Resources 1.05) to clean up the punctuation and verbal ticks in the transcript
• When you're satisfied that the transcript is completely deidentified
  • Click on the version history and select the most current version
  • Click the three dots and select "Make a copy", and the "Copy version" window pops up
  • In the "Name" text box, remove the "Copy of" text at the beginning and the timestamp at the end from the file name
  • Copy

  De-identified Interview Transcript-
  

  • Paste the text you just copied in front of the file name in the "Name" text box
    • The new file name should be formatted like "De-identified Interview Transcript-RP001-UX Researcher Abbreviation###-Intern Abbreviation###"
    • An example: "De-identified Interview Transcript-RP006-U007-I001"
  • Save the renamed de-identified Google doc transcript into the RP001 De-identified Transcripts folder. This is to make sure that the version history does not include any identifiable information.
    1. Click the folder icon under "Folder"
    2. Choose "All locations"
    3. Choose Shared drives > Internship > Internships > Research > Research by Type > Intern > folder for RP001 > RP001 De-identified Transcripts
    4. Choose "Select"
    5. Select "Make a copy"
    6. The copy of the file is generated in a new browser
    7. Dismiss the notification that says the file is now in the shared drive
  • Copy the link of the de-identified transcript in the shared drive
    1. Choose "Share"
    2. Choose "Get link"
    3. Choose "Copy Link" and "Done"
  • Paste what you copied into the matching participant's cell in Column D Transcription Link - de-identified in the tracking sheet (Resources # 2.03)
  • In Resources # 2.09, place the link you just copied between parentheses at the end of the line with no space in between the right bracket ] and the left parenthesis (, so it turns into a hyperlink
  • Choose "Update comment" in GitHub and make sure all the checkboxes above have been checked

• Review with UX Lead
• Product sign-off
• UXR Lead or PM removes the assignee from the Internship - PII's My Drive when closing the issue if the assignee no longer needs to access the PII drive.

Resources/Instructions

Resources for creating this issue

1.01 Wiki: Research Output Overview
1.02 Google Drive Folder: Research by Type
1.03* TWE Interview Recordings by Plan #
1.04 Guidelines to Interviews Page 8
1.04.01 Brackets vs. Parentheses
1.04.02 Video folder
1.04.03 Cleaning Up Zoom Transcriptions for Qualitative Research
1.04.04 Determining Best Practice for Filler Words in Captions and Transcripts
1.05 LanguageTool Chrome Extension

Resources gathered during the completion of this issue

2.01 Wiki: Research Plan: RP001
2.02 Google Drive Folder: RP001
2.03 TWE: IS21: RP001: Intern Exit Interview Recordings and Transcriptions Tracking Sheet
2.04* RP001 Interview Recording Folder
2.05 RP001 De-identified Transcripts Folder
2.06 Intern 003 Recording Link
2.07 Intern 003 Transcript .txt File
2.08 Intern 003 To be De-identified Transcript Google Doc
2.09 Intern 003 De-identified Transcript Google Doc

*This folder can only be accessed from the Internship - PII's "My Drive"

[pandanista]

Hi @Dzotsen, I've assigned you to de-identify the transcript for intern 003 from cohort 2021. Please follow the action items and let me you know if you have any questions.

[pandanista]

Hi @Dzotsen, for the de-identification and cleaning-up process, those videos in the video folder might be helpful. Please leave any feedback or thoughts for the videos in this issue so Drew can incorporate them into the videos he is producing.

[pandanista]

@Dzotsen Please provide update:

1. Progress: "What is the current status of your project? What have you completed and what is left to do?" - we know you just came back from break, please just summarize the progress from before then
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures or links* (if necessary): "Add any pictures or links that will help illustrate what you are working on."

• remember to add links to the top of the issue if they are going to be needed again.
  Thank you.

[pandanista]

@Dzotsen Please provide update:

1. Progress: "What is the current status of your project? What have you completed and what is left to do?" - we know you just came back from break, please just summarize the progress from before then
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures or links* (if necessary): "Add any pictures or links that will help illustrate what you are working on."

• remember to add links to the top of the issue if they are going to be needed again.
  Thank you.

[pandanista]

@Dzotsen Please provide update:

1. Progress: "What is the current status of your project? What have you completed and what is left to do?" - we know you just came back from break, please just summarize the progress from before then
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures or links* (if necessary): "Add any pictures or links that will help illustrate what you are working on."

• remember to add links to the top of the issue if they are going to be needed again.

[pandanista]

@Dzotsen Welcome back, and hope your 2024 is off to a good start.

Would you please provide an update?

1. Progress: "What is the current status of your project? What have you completed and what is left to do?" - we know you just came back from break, please just summarize the progress from before then
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures or links* (if necessary): "Add any pictures or links that will help illustrate what you are working on."

• remember to add links to the top of the issue if they are going to be needed again.

[pandanista]

@Dzotsen I just clarified with the leads that Adam was the tech lead for the website team during that time, not a mentor.

[pandanista]

Added a few guidelines on the transcript formatting. You are already following most of them. I am listing them below as a reference.

1. Use the 'in-line' format as seen here in Intern 001's transcript
2. Use 6 digit format for all the timestamps, for example, 01:01:01 as hour one/minute one/second one
3. Use "Interviewer" and "Intern ###" throughout the transcript. See an example here in Intern 001's transcript.
   • If there were more than two people recorded on the transcript, name the third person based on the interview set-up or context, such as Program Manager, Note Taker, etc. If you are not sure what to name them, ask leads on the project to clarify.
4. If there are other numerical values in the transcript, transcribe them using texts. For example, if a time such as 6:30 is mentioned, type out six thirty when cleaning up the transcript. Another example is if the interviewee mentioned he mainly sought help from 3 other peers, type out three instead of using the number 3.

[pandanista]

@Dzotsen Please provide update:

1. Progress: "What is the current status of your project? What have you completed and what is left to do?" - we know you just came back from break, please just summarize the progress from before then
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures or links* (if necessary): "Add any pictures or links that will help illustrate what you are working on."

• remember to add links to the top of the issue if they are going to be needed again.

[pandanista]

@Dzotsen Please provide update:

1. Progress: "What is the current status of your project? What have you completed and what is left to do?" - we know you just came back from break, please just summarize the progress from before then
2. Blockers: "Difficulties or errors encountered."
3. Availability: "How much time will you have this week to work on this issue?"
4. ETA: "When do you expect this issue to be completed?"
5. Pictures or links* (if necessary): "Add any pictures or links that will help illustrate what you are working on."

• remember to add links to the top of the issue if they are going to be needed again.

[Dzotsen]

I could not follow the instruction for this particular sequence. Instead, I downloaded the transcript and uploaded it to the corresponding folder.

Found it. I think the "History" icon would help in the text to guide the person in the right direction.

[pandanista]

At the UXR team meeting on 2/13, we have figured out the reason why the instrcutions above were hard to follow. It was due to the version history was a unfamiliar feature for Google doc users, and the three dots were hard to find. Therefore, the UX lead will make changes to the template in #509.

[pandanista]

I've reviewed the transcript and made some minor edits.

@sunannie27 Would you please review the Intern 003 De-identified Transcript Google Doc?

[ExperimentsInHonesty]

@pandanista This transcript seems substantially different from others we have seen. For instance,

• there are brackets around the word like, in some places, and not in others. Generally brackets are meant to denote words the person left out, but are needed to make the sentence make sense.
• other brackets in the transcript mention background noises and words that are unclear.

When you were reviewing the transcript did you listen to the recording?

[pandanista]

I have reviewed the recording again, and cleared out the unclear wording in the transcript.

Also, the research team discussed the use of brackets at the UXR meeting on 2/27 and we decided to remove the brackets around the verbal fillers, such as like, but keep the verbal fillers so the transcript stays true to the communication style of the intern. We all agreed that verbal fillers wouldn't affect the outcome and validity of the research, so we were okay keeping them.

Next steps: @Dzotsen will remove the brackets around the verbal fillers and do another round of cleaning-up of the transcript.

[pandanista]

Thank you for being so patient with us, @Dzotsen.

Would you please update "interviewer" with "UXR 003" in intern 003's transcript? Much appreciated!

We decided to include the UXR numbers in the transcripts because 1). there might be different probing questions for semi-structured interviews from different interviewers, and 2). it will be easier to reference the UXR when doing analysis or making highlight reels (if needed).

[Dzotsen]

"Interviewer" changed with "UXR 003" in intern 003's transcript.

[ExperimentsInHonesty]

@pandanista please define next steps for this issue.

[pandanista]

• I have reviewed Intern 003's transcript.
• Ready for product to do a final review and sign off on this issue.

[sunannie27]

@pandanista Looks good to me.

[ExperimentsInHonesty]

@Dzotsen Thanks for the great work on this.