AI Solution for Clinical Trial Planning starts with answering key questions about how to design experiments that determine whether a drug is effective and safe. Let’s say a company wants to introduce a new drug to the market. How should they plan the experiments to properly assess its effectiveness and safety? How large should the sample be? What should they look for? How should they define the inclusion and exclusion criteria? These are not easy questions, but they must be answered if the company wants to achieve the best possible outcome.
Challenges in Clinical Trial Planning with AI
The client came to us with a very ambitious goal in mind. The goal was to create a recommendation system for planning scenarios of clinical trials. They had a vision of using data mining and clinical trial planning with AI to make the whole process faster, more reliable, and more affordable. This type of work must also remain consistent with established standards for clinical trial quality and conduct, such as ICH E6(R3) Good Clinical Practice (GCP).
The client’s team of programmers and designers had already created a big part of the system before they reached out to Stermedia. What they lacked was a machine learning experience and familiarity with natural language processing. A human can not easily make use of hundreds of thousands of existing protocols documenting past clinical trials; this amount of knowledge first needs to be structured and systematized before it can be useful. The Stermedia team was responsible for testing, recommending, and implementing state-of-the-art solutions that helped with this enormous task.
AI-Powered Clinical Trial Planning System Using NLP and BERT
Our team worked on this project for almost 9 months. Building an AI solution for clinical trial planning required structuring a large database of medical documents, which is a complex task. During this time, we focused on several smaller goals that brought the project closer to realization.
Named Entity Recognition
We started with information extracting – this means searching the document for the information that is needed. For example, we might be interested in finding a specific concept in the text, like a drug name. This problem is known as named entity recognition (NER). The solutions our team chose are based on the modern “transformer” architecture called BERT. What BERT gives us, is the ability to represent words as numbers in a smart way. Words with a similar meaning are assigned similar vectors. The transformer also looks at the surrounding context, which prevents it, for example, from confusing different meanings of one word.
We used a large database of clinical trial protocols to tune BERT to our use case. Then, we trained several neural networks for recognizing concepts in raw text. But training the models manually is very time consuming, so Stermedia also prepared a mechanism to make this process easier for the client. Moreover, we created a method that helps in the automation of creating datasets necessary for teaching the model to recognize new concepts.
The quality of Stermedia’s work is not their only asset: they also have excellent communication standards and are always on time. It never seemed like there were 8 time zones between us! They provided a tremendous amount of value and were real team members of our company.
Tom Walpole, CTO, Trials.ai
Hand-made rules
Sometimes the information extraction process doesn’t need to involve heavy mathematical models. In some cases, we can achieve a lot by automating our own thought process. For example, some professional medical words almost always appear in certain types of texts.
Another reason is that many types of phrases have a rigid structure, thus making them easy to describe and find using regular expressions combined with part of speech detection. For instance, we can imagine many variants of the phrase ‘number of people with’. Different words can be used in place of ‘number’ and ‘people’, but if we think about it beforehand, we can create a more or less complete list of those that actually appear in paragraphs we have to handle.
Once we detect this generalized phrase, we can reasonably expect the noun phrase following it to be the thing these ‘people’ are dealing with in our context. This approach required us to spend some time reading and understanding the text we were processing. The knowledge we gathered this way gave us a lot of ideas for hand-made rules describing what we had learned. Our team not only created and tested a lot of these rules but also prepared a flexible Python framework that allows the client to easily add as many as they like, using their own specialistic insight.
Developing tools and preparing models for the project was a challenge our team really enjoyed. Ultimately, not only did we help the client accomplish their goal but also learned many valuable lessons about the state of art in NLP.
Michał Krasoń, Data Scientist, Stermedia
Text classification
Another capability that we included in our system, is assigning a class to a passage of text. This can be used in multiple scenarios, for example, deciding which study objective -among a known list of choices- is relevant for a protocol.
Both the choice of words and the entire context of the paragraph are important to properly determine which class it should be assigned to. Therefore, in a similar fashion to the NER solution, we used a BERT-based model with context-sensitive representations to perform classification as part of clinical trial protocol analysis with AI.
We used a variety of hand-made rules based on regular expressions and other parsers to create an initial, labeled dataset that would serve as a basis for further improvements by domain experts. Our model, derived from the pre-trained neural network, was fine-tuned on this data. As before, the whole process was incorporated into our framework – simplifying the process of preparing and training this architecture for other classification tasks.
Results of the AI Solution for Clinical Trial Planning
The project was oriented towards delivering tools that would make the future analytical process easier for the client. Overall, it showed how machine learning for clinical trial planning can support practical analytical work in a demanding domain. To summarize the previous sections, our main contributions were:
- testing and recommending solutions to the client’s NLP tasks;
- preparing several microservices for easy training and making predictions with named entity recognition and text classification models;
- creating a mechanism for transforming raw text into a training data set for the named entity recognition task;
- developing an easily expandable framework for using hand-made rules to extract information from text.
In addition, we experimented with many other techniques, like text-generating models, coreference resolution, and relationship extraction. In the process of creating the aforementioned tools and verifying our approaches, we also trained a number of models on experimental data sets. These can be further improved upon by obtaining data of higher quality, but the client can start using them right off the bat as well.
Technologies Behind Clinical Trial Planning with AI
Python, jupyter, pytorch, transformers, flair, spacy, nltk, celery + redis, docker, docker-compose, luigi
Reference letter
Our solution is dedicated to the biggest pharma and biotech companies around the world. During a period of cooperation lasting almost a year, data scientists from Stermedia prepared an easy-to-use framework that simplifies the deployment of new analytical tasks and participated in the design and implementation of analytical solutions. They prepared a set of heuristics for identifying concepts in medical text. Their NLP (natural language processing) expertise lets them recommend, test, and successfully implement several state-of-the-art models for solving NLP problems. Moreover, the data science team suggested suitable NLP architectures taking into account feasibility, ease of use, performance, and applicability to the tasks to be solved.
Our colleagues from Stermedia showed versatility with Python programming, machine learning, and infrastructure development skills used throughout the project – also by creating several microservices for training and serving NLP models. It wasn’t only data science that was a great help to us through – the Stermedia scrum team supported us in software development of web application: frontend in Angular, backend in Python – rewriting a part of the code as well as building new functionalities and microservices deployed on docker. The quality of Stermedia’s work is not their only asset: they also have excellent communication standards and are always on time. It never seemed like there were 8 time zones between us! They provided a tremendous amount of value and were real team members of our company.
Trials.ai
Next steps & further reading
If you want to explore how an AI Solution for Clinical Trial Planning can improve protocol analysis and scenario design, the best starting point is a small, controlled pilot based on a limited set of historical clinical trial documents.
At Stermedia, we help CEOs, startups, and established organizations deliver full AI integration — from needs analysis and proof of concept to production deployment of custom AI solutions.
Continue reading:
- AI in Healthcare for Medical Question Answering – how a domain-specific QA engine helps teams search complex medical documentation and return accurate, explainable answers with cited sources.
- AI Solutions for Healthcare: Screening Revolutionized – a healthcare case study showing how generative AI supports depression screening and improves patient-focused workflows.
- Revolutionizing Public Health with AI and ML – a case study on automating manual notification handling for a public health institution to improve efficiency, safety, and compliance.
Are you inspired?
Let’s talk about your idea.
Let’s talk
