had a challenge. Farhana Zulkernine, Associate Professor and the Coordinator of the Cognitive Science program at the School of Computing at ֱ, is helping them solve it.
Since its launch in 2016, the tele-medicine platform that focused primarily on women’s health has enjoyed robust growth, both by subscription in the United States and United Kingdom and as a service covered by OHIP here in Ontario. Users with a health question can access the platform 24-7 and be connected with a doctor. If they have an Apple watch or similar device, they can even use that to give a doctor their heart rate and temperature, too.
Satisfied users meant repeat users, a good thing. And more traffic overall, meant Your Doctors Online needed more doctors. Also a good thing. But this led to the challenge: Between more users and more doctors, there was no guarantee that the doctor you contacted knew about your case. Ninety percent of the people contacting Your Doctors Online did so via text using a chat program, which meant that the company had written material for patients that recorded useful information about what drugs they were using, what previous symptoms they had reported and so on. That’s all potentially valuable, but wading through it is time-consuming, particularly with a user waiting for guidance.
Enter Dr. Zulkernine, who works on projects involving artificial intelligence (AI) and big data, particularly as the latter relates to the health-care sector, through her at Queen’s. Among the organizations she and her BAM lab have worked with are IBM, Thales Group, Public Health Canada, and CIMVHR (the Canadian Institute for Military and Veterans Health Research).
One of the directors of Your Doctors Online, Nauman Jaffar, had a connection with Dr. Zulkernine, and reached out to her. Could the material the company had gathered be summarized in some way that would make it helpful for a doctor dealing with a patient? Dr. Zulkernine thought that it could. And, fortunately, she had a post-doctoral fellow in her lab, Dr. Hasan Zafari, who was an expert in natural language processing. He would take the lead.
For Dr. Zulkernine, a key feature of the many innovative projects her lab undertakes with private sector companies is the opportunity they provide for her students to get experience different from what they would expect in school. To that end, she works very closely with , a national not-for-profit organization. Mitacs seeks to advance industrial and social innovation in Canada by helping businesses benefit from academic research by jointly funding internships with interested companies for graduate and undergraduate students.
“Many of my projects have been funded by Mitacs,” she says. “It gives students a chance to interact with companies directly, learn business and consumer perspectives and develop effective solutions.” For her industry partners, Mitacs helps reduce their costs and exposes them to talented potential future employees. For this project, she obtained financing for two undergraduates to work with Dr. Zafari.
Stated baldly, summarizing online chats sounds simple enough. In practice, however, it soon proved very difficult.
“There have been several challenges,” says Dr. Zafari. For one, when people typed in descriptions of their symptoms or listed the medicines they are on, their spellings could be widely erratic – so much so that the dictionaries that Dr. Zulkernine’s teams worked with to create the summaries might not be able to identify what words they were trying to use and correct them. This is essential for the algorithm to create a good summary. Beyond that, they are trying to create a system that would create a summary from scratch, a difficult thing to do. How do you determine which material is important and which isn’t? Given the same material, “even two doctors would have different perspectives on what is a good summary,” says Dr. Zafari. Tao Ma, one of the undergraduates paid for by Mitacs, worked on trying to create a system for annotating the texts to create summaries using one data pipeline tool while Zafari worked with another, after which they compared results. These chat summaries were then evaluated by medical professionals.
The system that Dr. Zulkernine, Zafari and their undergraduates finally developed for Your Doctors Online takes a novel, colour-coded approach to summarizing the information contained in the chats. Says Dr. Zulkernine, “We highlight sentences in three different colour shades, based on their information content, to identify important sentences while keeping the rest in their original colour.” Doctors working with these summaries can see the key information right away, but no part of any text is left out, so if they feel they need more, it’s available.
The project is “95 percent ready” to be put into action says Raihan Masroor, the CEO of Your Doctors Online. Tao Ma continues to work on the project even now as he pursues his Masters’ degree and is currently co-writing a paper with Dr. Zulkernine and Dr. Zafari detailing their work.
“It was really exciting to get experience on a real project,” he says. Masroor calls Dr. Zulkernine, “a key part of our ongoing R&D, and in fact, she is already looking ahead to build a data-management system that will create summaries from the spoken conversations that the physicians at Your Doctors Online have with patients.”
“The existing voice communication system cannot perceive medical domain specific terms,” she says. “We want to improve domain specific voice to text conversion.”
The Queen’s Partnerships and Innovation (QPI) team helps companies and other external organizations find Queen’s researchers with whom to collaborate. QPI and Research Services colleagues work closely with federal and provincial funding agencies to leverage investments in research, help researchers and partners prepare proposals, and develop associated agreements. QPI also helps researchers and companies manage their relationships.