Questions and Answers from Day 1 and 2

  • Q: How does QXR perform in pediatric TB? Could you please throw some light on the differential performances in any specific cohorts like HIV TB, etc.

    A: QXR can process X-rays that are 06 years and above. In Malawi, the software is used for ACF in a high HIV prevalent population. We can set up a discussion with you to walk you through this in detail.

  • Q: Which patients are selected for QXR? Are these the same patients for which a WRD should be performed?

    A: Hi there, could you kindly elaborate on the patient’s selection? Did you mean for the validation studies? or did you mean who goes through a QXR software? if the latter, all symptomatic, asymptomatic, at-risk, and vulnerable population (as defined by the program) for ACF/ICF as well as Passive case finding can be subjected to CXR for screening and QXR can be used to read them in real time and alerts the health system/program immediately for sputum collection for bacteriological confirmation. I’d be happy to walk you through in detail and answer any more questions you’d have. Please feel free to reach out.

  • Q: Can the AI read CXR from any age (including children)?

    A: Hi, currently QXR can process X-rays of individuals 06 years and above.

  • Q: Does QXR only consider the X-ray or can it consider other symptoms for a more thorough diagnosis?

    A: Interesting question. The TB algorithm of the software is much optimized for final confirmatory results. Currently, it looks at only X-rays for producing the results, however, since the algorithm is fine tuned for final confirmation results with high sensitivity and specificity operating points, the software rarely misses a case. Having said that, considering the symptoms and risk factors is something we will definitely consider.

  • Q: The value addition of radiology has been for asymptomatic patients who show up at facilities and cannot be diagnosed as they do not have symptoms. Do you have any experience with patients with abnormal CXR but have no TB symptoms

    A: Yes, we do. In a few of the TB programs we are working with, they not just screen the symptomatic cases, but also the contacts, by-standers, vulnerable and at-risk population (like 4P’s member, smokers etc). They have seen data that the software was able to pick these asymptomatic cases, who would have been missed otherwise. I’ll be happy to walk you through in detail on these experiences.

  • Q: What are the costs involved in using the QXR long term?

    A: We will be happy to help you with a costing that can work best for your program/ use case. Please feel free to reach out me at reshma.suresh@qure.ai

  • Q: Will a recording of the event be available?

    A: The recording of this event is available on this website.

  • Q: How can Patient adherence to medication be monitored on a routine bases using this technology?

    A: Self verifications are collected, analyzed, and the results provided to a team of supporters, who follow up with those who have not verified. These data are available on a desktop ‘backend’ to healthcare providers as well.

  • Q: What is the minimum system requirement for this technology?

    A: USSD works on virtually all flip phones.

  • Q: Were you able to assess for “accuracy”–meaning were you able to assess if “verification” did in fact mean that the person took their dose?

    A: Yes. Our team actually spent a good chunk of the last three years implementing random urine testing to verify that ‘verification’ corresponds to actually taking medication.

  • Q: DS, DR and TPT/LTBI?

    A: Theoretically: yes. These are all long, difficult treatments, and non-adherence has major implications for the community, not just the individuals. Practically: of course, adapting the platform to a new treatment regiment has to be done with great care.

  • Q: Are Patients able to read?

    A: In Kenya, where we were operating, 90%+

  • Q: What platform are you using to send and receive the SMS/USSD? were there any challenges reaching all subscribers?

    A: I unfortunately don’t know the technical details. I know Keheala built its own, custom platform. I think many people are sufficiently familiar with USSD platforms like ours, so reaching subscribers wasn’t as tough as you might expect. We have quite high engagement. Only a small number (~13%) of patients don’t engage regularly with the platform, and even those patients prefer to remain subscribed so that they can continue to receive reminders.

  • Q: Do you see if any language barrier exists for using this App?

    A: The platform is available in English and Kiswahili. Users choose their language. So far, that’s covered our needs well.

  • Q: Do you need to have separate tools for DRTB and sensitive TB?

    A: It depends. If DR treatments are sufficiently similar to DS treatments, the DR patients can use the same tool, and simply be left on longer. But sometimes the DR treatments differ substantially, and then (at least on our backend) they are enrolled in a separate system.

  • Q: Related to the prior question, what advantages does WeChat offer over other platforms?

    A: This is because WeChat is widely used among Tibetans, so we do not need to introduce a new app.

  • Q: Was there any correlation between frequent co2 alarms and incident TB cases among health workers?

    A: We have not looked at the HCW TB incidence as a result of CO2 intervention in isolation. IPC is implemented as a package which includes, admin controls, PPE, environmental controls.

  • Q: Could you send access to the application? Thank you.

    A: Access to the link is provided in the presentation.

  • Q: I was wondering if the presentations will be shared for our appreciation?

    A: Thank you and yes, recordings will be shared on the forum website after the conclusion of the event, accessible at https://tbhealthtech.org

  • Q: What is the level of training required to implement AI solution for parsing x-ray images?

    A: The lab technician needs to be trained to input information, file naming convention and pre-defined machine location. Qure.ai software selects the images with the set machine location and file names for AI screening. So, minimal training is required for AI implementation.

  • Q: In the absence of a recommendation on the default cutoff score, how did you select the threshold cutoff scores in your program and what are the challenges, if there are any?

    A: QXR processes the images and produces results on signs of TB from CXRs on definite “Yes” or “No” based upon predefine thresholds and configuration set for the algorithm. So, for the intervention this predefined threshold cut off was already available Cut-off score was altered during the intervention based upon the data analysis after 5 months of intervention.

  • Q: So, the fieldworker decides on sputum collection without a physician’s interpretation?

    A: Before the intervention, informal providers were explained the diagnostic algorithm and all the patients having abnormal chest X-Ray findings were eligible for sputum test. Field worker were responsible to collect the samples of presumptive TB patients with abnormal chest X-Ray findings. Abnormal CXR findings through AI were communicated to the field worker through text messages.

  • Q: What are the false positive and negative rates for QXR compared to radiologist diagnoses?

    A: There were 10.8 percent false positives and 6.8 percent false negatives reported compared to radiologist diagnoses.

  • Q: What level of endorsement that was received from the Govt in terms of scaling-up of the technology?

    A: AI based screening is now included in National Strategic Plan of India for TB.

  • Q: What are the challenges?

    A: Informal providers: Uptake of CXR vouchers provided by informal providers in the first few months of the project.
    AI: 1. Identifying the labs with required infrastructure – internet availability and speed, local area network if the computer used for CXR screening did not have internet. 2. Radiologist association support for AI installation was delayed posing a challenge on the number of labs that could be networked for the project.

  • Q: Is there any plan to scale up in other states of India for in view of COVID related reduced case findings?

    A: There are plans for introducing qXR screening for TB in Uttar Pradesh through PATH’s Indian affiliate CHRI. There are various other ongoing interventions in Assam, Bihar, Jharkhand, Nagaland by Qure.ai.

  • Q: Engaging the private sector would usually require some form of incentive. What was your experience in motivating the informal care providers to key into this?

    A: There were no incentives involved in the intervention. In the initial phase, uptake of CXR vouchers by the informal providers was limited which gradually improved with persistent communication with the providers to screen presumptive TB patients.

  • Q: What would be the cost per Xray screened?

    A: INR 240 was the cost per chest X-Ray screened with AI. CXR cost varied between INR 250 to INR 400 in Nagpur.

  • Q: The frontline workers capture the photos. What kinds of photos they capture and how to capture or a machine to capture?

    A: Frontline workers were not involved in the workflow to capture pictures. Digital CXRs were taken in private sector labs by the lab technicians.

  • Q: What are the challenges you are facing?

    A: In general, a key challenge is getting providers to buy into the model and see the value for their customers while trusting in your services. At the moment, the biggest challenge is COVID, which means that PPs aren’t seeing a lot of patients for fear of being closed down by the government. Hence, the majority of our work has focused on private and non-NTP hospitals.

  • Q: It is an interesting presentation. What is the role of PPM in TB: Just detect suspected cases and then refer? treat?

    A: I believe PPM is to expand access to quality-assured care and prevention. This means that PPM is a vehicle to make it easier for patients to access their care of preference, and to make sure that care is of the highest standard.

  • Q: As there are many tools used by PPs, how are these tools being harmonized (data streamlining purposes) at the national level (i.e. at the NTP level)?

    A: I hope that my live response addressed your question, but feel free to message me for a more detailed response.

  • Q: Was privacy not an issue in data sharing between the government and the private sector?

    A: It was a big issue with respect to two key things, provider identity and provider pricing, model. As long as we didn’t share the provider identify with the NTP and didn’t share their pricing with other providers, those collaborating with us were OK with reporting TB patient data to the NTP.

  • Q: What was the role of Pharmaceutical companies in Public awareness. which company?

    A: We did not work with a Pharmaceutical company on this project, but we believe that all life sciences stakeholders, including Pharma companies, can assist in ending TB, especially through the support of awareness building.

  • Q: The private sector is for profit, how did you address the fee demand from the private sector

    A: We did not interfere with the PP’s business and revenue model. Most patients select PP’s for the convenience they offer and are happy to pay for it. Nevertheless, we supported providers and patients by offering discount vouchers for accessing CXR and free Xpert testing. As Xpert testing was provided for free by us, we did not allow PPs to charge for those. That’s the main profit limitation we implemented.

  • Q: How are the results of awareness?

    A: The main awareness-building effort on our first phase was by working with each district’s PP licensing authority to present the project during their semi-annual PP meeting. Once the PPs were sensitized, we worked with local TB and public health officers to identify the high volume providers and engaged them individually through our staff.

  • Q: How did you monitor the treatment outcome of the patients to ensure that they complete the treatment and eventually cured?

    A: Great question. We reached out to the 14 notifying PPs on a monthly basis to obtain a status update on their patient adherence, but in the framework of this project we were unable to ensure treatment completion per se. This is a key area of opportunity for improvement of treatment quality going forward.

  • Q: Is/are any benefit to PPM from their engagement?

    A: I’m not sure I entirely follow this question and what you mean by “their engagement”, so please message me to clarify and I’m happy to respond here.

  • Q: Luan Vo, in your opinion, what is the best software to work with the private sector, and how did you motivate the private sector? Thank you.

    A: I don’t know if there is a “best software” and what that would be, so I’m sorry that I can’t answer this question. I believe that digital tools will be critical for scale-up, whichever may be appropriate for the local context, but also will require a significant amount of training and continuous customer support for the software to be most effective.

  • Q: Are patients typically counseled by pharmacists on the duration of/ and adherence to treatment?

    A: Hi Jamie, thanks for the question. If the patient presents with a prescription for TB and is filling it at the pharmacy (i.e. they are confirmed already) then the pharmacist may provide counsel. However, we believe there is a lot of opportunities to improve pharmacy professional education on adherence/counseling for positive patients.

  • Q: With such a high proportion of confirmed TB among clients referred by pharmacists, are there plans to increase the sensitivity of the screening beyond chronic cough and fever, given that these are likely to pick up persons who are more obviously sick?

    A: Hi Robert, yes this is a very good point and we are thinking about how to widen the screening criteria as the positive rate is so high. Happy to chat further on this and get your insights on the topic – meralli@mclinica.com, for your reference.

  • Q: Is there a plan to expand this App across the country?

    A: To date 5 countries who evaluated the intervention are expanding or taking it to scale.

  • Q: What are the pharmacy features that make this successful and how applicable is this to other settings?

    A: Thanks Elizabeth for the question. The app is built to be pharmacist-centric rather than disease-specific. Taking this approach ensures that pharmacy professionals have access to everything they need to better serve patients and manage their pharmacies – accredited online education, drug information, ADR reporting, discussion forums, etc. This means that SwipeRx becomes used daily. Then when we want to do TB specific programming or another public health programming we can put features within the app and ensure usage. Happy to chat further my email is meralli@mclinica.com for your reference.

  • Q: Are there any incentives for the pharmacists from the insurance company?

    A: Interestingly, there were no direct incentives to pharmacists to refer patients. The campaign tapped the professional and moral obligation of pharmacy professionals to improve their community health. A ‘heroes’ campaign was created to identify and highlight the top referrers. Ultimately, no direct incentives were given however pharmacy professionals continued to refer.

  • Q: Are you thinking of a study which could look at delays happening in TB diagnosis and how much is your app reducing this delay

    A: Counties are monitoring in real-time the availability, accessibility, acceptability, and quality of diagnostic services.

  • Q: In patients who use self medication.what is MDR %?

    A: Thanks for the question. We did not see any MDR+ patients in the pilot so do not have the data for that.

  • Q: What was the total presumptive TB cases referred by the pharmacies? What number the 54% represents? How did you ensure that these referred clients really reached the health facilities in order to receive adequate services (TB diagnosis and treatment if needed)?

    A: Hi Alex, yes we have a great breakdown of this, happy to provide, my email is meralli@mclinica.com

  • Q: An interesting presentation that open eyes for other countries to explore this technology, was staff turn over in pharmacies, not a problem?

    A: Good question. We train the individual rather than the individuals at the outlet level so if they move to other pharmacies they can continue the work there. We did not however see any appreciable turnover of staff during the pilot period.

  • Q: Question for Coaimhe – very exciting work! How do you envision community accountability in contexts where smartphone access is difficult or not feasible? Thanks

    A: Hello David, thank you. To answer your question 1) Parts of the APP are available offline 2) Reporting barriers can also be managed through a USSD channel, so a smartphone is not necessary and 3) Some countries are relying on a model where CHW leaders are reporting on behalf of people with TB, in other words not every person with TB needs a smartphone.

  • Q: The tool has different stages of maturity scale in different countries – how is your learning help you to introduce the tool in any country?

    A: In every country, we learn something new. Because we’re using an implementation science-based approach we are capturing all the learnings as we go and constantly sharing. We have an OneImpact community and at this stage at the end of projects, we are updating the platform, supporting TA and documents based on shared learnings from all countries. In this way, they have all contributed to its development.

  • Q: Is GF CRG considering a One Impact type platform for the HIV grants?

    A: We use TB as the entry point, and then countries tend to add HIV and now COVID, which fits into the AAAQ framework, i.e. they are monitoring the availability, accessibility, acceptability and quality of TB, HIV, COVID services

  • Q: How did you go about the confidentiality law of HIV regarding data sharing with the TB program?

    A: This is a good question. I don’t think it was for me, but I’ll comment anyway.

    Importantly, it is not just HIV confidentiality laws that apply to the collection, storage and use of digital data from people with TB. Several other kinds of laws apply. These include laws that regulate all kinds of digital data, regardless of what it is about; laws that regulate all kinds of health data; and even laws that regulated consumer data, as they may apply to apps used for TB.

    The point is, we should not mistakenly think that the only concerns about TB digital data are concerns about HIV laws.

  • Q: Is there any empirical evidence available for any country to what extent GxAlert was directly associated with DRTB?

    A: I assume you’re asking if GxAlert (or connectivity) has been associated with improved treatment/outcomes for DRTB.

    I know of a couple of countries that have presented studies at The Union Conference showing the benefit of connectivity on the treatment of DR-TB specifically. I’ll add a couple of links to this answer.

  • Q: Internet of things should make capturing data automatically.

    A: Most of the diagnostic instruments are now capable of reporting over the internet in an Internet of Things kind of way. Some of the efforts still required is a place (server or data system) to have those instruments send their testing results. And as I mentioned, there is typically a lack of unique patient ID so that when results are reported (over the IoT) it’s not possible to automatically link them to the right patient or case. Though these challenges can usually be overcome with a little forethought and development on system integration.

  • Q: Yes but we do not even know if Xpert is being used optimally never mind the treatment follow-up. When and how often are machines down? Etc

    A: Progress is being made in the evolving field of “connected diagnostics”. More robust “Operations Dashboards” are being developed to help governments, programs, partners, donors, and manufacturers share a real-time view of the instrument network. As this becomes developed and in place, it will allow the diagnostic lab networks to become reliable and optimized.

  • Q: Can this data system for Xpert be linked with other health data systems being followed in a country so that all diagnosis data could be in one place?

    A: I know our solutions (GxAlert/Aspect) are designed for any kind of diagnostic result or instrument. We already work with countries for HIV, Ebola, COVID, and HCV in the central lab and point of care settings. I know most of the connectivity solutions and providers can support multiple health data systems.