Copyright [2022] [AI Engineer: Ahmed]
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
AI has been the dark horse for lots of applications and case studies in the medical industry, especially in consultation services.
Most of the AI Engineers in the medical industry start doing research related to diagnoses using CV, NLP, and RL Turbulence.
Today, we're going to dive deep to analyze a dataset I have been asked to do some analysis on which is Diagnose me.
Diagnose me is an LFQA dataset of dialogues between patients and doctors based on factual conversations from icliniq.com and healthcaremagic.com that aims to collect more than 257k of different questions and prescriptions for patients.
Here, I decided to analyze these dialogues and clean them for having them ready for Modeling. Unfortunately, we're not going to do any kinds of modelling here, all that we're going to do is analyse the dataset that we have by cleaning, removing the characters that represent a noise to the data, and adjusting the punctuation so it can help for having factual sentences.
I hope you like it!
