Trust but verify is the wrong paradigm, it should be a paradigm of never trust, & Continuous Validation.
Consider the gravity of an AI imposed mistake in the world of healthcare, specifically Oncology.
“AI-based technologies have the potential to accelerate innovation, increase competition, help to ameliorate health inequities, reduce clinician burnout, and improve care and the care experience for patients,” Tripathi said in his statement. However, Tripathi also said that “we know that there are potential downsides,” so HHS leaders believe their posture on AI must be to trust but verify.” – quote from article linked below…
We would advise them to be careful on the “Trust but Verify” approach.
Instead, implement an aggressive testing mechanism of “CV” Continual Verification/Validation. It would be ideal to operationalize this, but in lieu of that, use human validation to what degree possible and collect metrics on how accurate these AI outputs are.
Once those metrics can be used to draw statistical meaning, increase the amount of automated validation, but never truly give up on human validation unless and until trust in the AI model always remains statistically high and automated testing coverage and accuracy is always 100% thorough and accurate.
What could possibly go wrong? A patient getting chemotherapy when they didn’t have cancer. Patient deaths due to misdiagnosis, large malpractice lawsuits, or all of the above. Related article linked below:
