Learning To See But Forgetting To Follow: Visual Instruction Tuning Makes LLMs More Prone To Jailbreak Attacks Code & data for our paper accepted to Safety4ConvAI workshop in LREC/COLING 2024