BACKGROUND
At Syneos Health® we believe that our people are our greatest asset. One of our approaches to nurturing a growing leadership community within our organization is to commit to the professional development and growth of our talented internal stakeholders. As part of our ongoing journey to build a culture of excellence within our Medical Affairs organization, our leadership team piloted a field medical affairs global leadership development program (GLDP) for a team of emerging medical science liaison (MSL) leaders.
In the Spring 2023 issue of The MSL Journal, Miller and colleagues report “the best leadership teams protect, encourage, and develop people to achieve their overall success and they do so together.” The GLDP was created on this core philosophy of transformative leadership. We strongly believe that our organization will flourish when we learn and grow together.
Our publication reports on the outcomes of our GLDP pilot program and shows how our leadership team leveraged our internal Syneos Health Analytics Platform (SNAP) to customize each of our MSL’s leadership learning journey. We also show how a supervised machine learning strategy can be applied to predict the likelihood of future MSLs participating in the program and developing as our internal future leaders.
PROGRAM OVERVIEW
The GLDP consisted of 7 high-performing field MSLs who shared a common career goal of advancing within our organization. The average pharmaceutical industry experience of the participants was 10.6 years (range 2-33 years). 60% of the participants reported not having people management experience prior to entering the program.
This group committed to participating in our pilot program for six months (June 2023-November 2023). 14 of our internal medical affairs leaders served as program moderators. The MSLs were exposed to 5 distinct leadership tracks that would help develop and prepare them as future leaders within our organization. The content of these tracks were based on a proprietary leadership curriculum which focused on the “First Time Manager Journey” and included:
- Becoming a Front-Line Leader
- Developing People
- Accountability
- Driving Execution
- Emotional Intelligence
Each MSL prepared a 30-minute presentation to prime the discussion aligned with these five core themes. Each session was 90 minutes in length. Our leaders served as moderators and also shared their leadership experiences related to the specific track among the broader participant group.
We used our Microsoft® Azure OpenAI technology stack to help assess specific learning needs of our program participants. This was accomplished using serial surveys we collected prior to the start, during the mid-point, and at the conclusion of the program. Our AI output gave us insight on how to customize our curriculum to optimize learning for all program participants. Lastly, we used our internal integrated development environment (IDE) to process our machine learning (ML) algorithm.
PRE-PROGRAM SURVEY RESULTS
MSLs were asked to complete a brief survey prior to starting the program. The survey results were entered into Azure OpenAI to identify patterns and insights of the MSL expectations, baseline understanding of leadership, and appropriateness of the leadership content. The initial output is shown below.
MSLs expectations for the GLDP-helped our leaders focus on customizing the curriculum
Trends from surveys provided insight into the MSL baseline understanding of leadership
Objectives were aligned with MSL needs based on historical comparisons of survey output
This approach helped our leaders receive relevant feedback from the MSLs. As a result, the feedback and curriculum content was received as being directionally correct.
MID-PROGRAM SURVEY RESULTS
The mid-point survey results revealed specific leadership topics that our participants wanted to spend more time learning. Our generative AI produced the following feedback and analyses:
- Valuable sessions
a. MSLs found specific sessions more valuable, such as the track on developing people. - Interest in Real-World Applications
a. MSLs expressed a desire for the leaders to share more real-world cases and active problem-solving. - Positive group dynamic
a. MSLs described the group learning setting as respectful and very interactive.
b. This suggested a positive learning environment was fostered. - Commitment to continuous learning
a. MSLs expressed gratitude for having access to additional leadership learning resources.
b. This suggested the MSLs experienced a dynamic and growth-oriented program.
As a result of these survey responses, the remaining half of the leadership curriculum was modified. Our leadership team invested more time and energy in providing real-world examples to the group as well as providing more learning resources and even implementing a post-curriculum mentoring program.
FINAL SURVEY RESULTS
We collected a final survey that gleaned the participants feedback on their entire leadership program experience. 5 of the 7 MSLs completed the program and participated in the final survey. The other 2 MSLs transitioned to external roles.
Despite the low number of participants in this pilot, we decided to integrate a basic machine learning algorithm that could help us predict whether future MSLs might be interested in participating in this leadership program. It is important to emphasize the small number of participants is not a representative sample, however, the machine learning principle we used can be scaled appropriately for larger teams and data sets.
We hope to show that a data-driven approach to traditional surveys using machine learning principles can yield insights into the value an internal leadership development program can have on future MSLs.
WHAT IS MACHINE LEARNING?
Logistic regression is a type of supervised machine learning algorithm that accomplishes binary classification tasks by predicting the probability of an outcome, event, or observation. The model delivers a binary outcome limited to two possible outcomes, such as yes/no or true/false (Kanade, 2022).
We used logistic regression to create a basic machine learning algorithm that could help our leadership team predict which future MSL would likely participate and benefit from our leadership program. This binary decision was based exclusively on the number of years they worked in the pharmaceutical industry.
We took this approach in order to simplify the model and to show the logic behind how machine learning could be used in conjunction with our generative AI results. It is more realistic to perform a multiclass classification approach that takes into account additional variables, but again, we are opting for simplicity in this example to highlight a potential use case of machine learning in medical affairs.
OUR APPROACH TO MACHINE LEARNING
On the final survey results, the MSLs provided a final rating of whether they would recommend this program to their peers. The survey ratings were designated using the following values:
1 = Yes, I would recommend
0 = No, I would not recommend.
5 MSLs provided a rating of “1” and the 2 MSLs who did not complete the program were automatically entered as providing an entry of “0”.
We took these results and used our internal IDE to perform a logistical regression. Below is a snippet of our code and a visualization of our data. It is not important in this publication to understand the coding. However, the main takeaways are as follows:
- There are several numerical and scientific databases that are part of the machine learning toolkit that help analyze data (i.e., pandas, numpy).
- Entering structured data, despite being small in size and dimension, will help in generating a supervised learning model that could be used to augment current decision-making processes.
- Machine learning models need to be trained and tested on a dataset to ensure the model is doing an adequate job of generalizing the input data.
- Output from machine learning still requires human interpretation and models need to go through additional validation and modification before it can serve a practical purpose.
- The more data you have, the better model you can create. ML allows you to visualize the data and provides different ways to give structure to a vast data set.
The ‘gldp’ column represents whether each MSL would (value=1) or would not (value=0) recommend the leadership program to their peers.
A logistical regression can be plotted using the data from our survey results, thus serving as a helpful visual aid of the data.
It is important to note the small sample size we are working with will limit the predictive capacity of the machine learning algorithm we are configuring.
However, the same logic can be applied to a larger data set whereby we use a larger training set to ‘train’ the machine to make a prediction based on the variable we enter. In our case, ‘years’ worked.
This means an MSL with 5 years experience working in the pharmaceutical industry would recommend participating in our leadership development program. This result can also be checked on the graph above.
CONCLUSION
Our pilot Global Leadership Development Program was a success. The MSLs were exposed to a wide array of leadership topics that were customized to their unique learning preferences with the help of generative AI. Analysis of our survey results from generative AI revealed what leadership topics the MSLs were exposed to historically and thus, not interested in learning again. In addition, it provided insight into leadership topics of interest that were not initially part of the curriculum. This custom-tailored approach helped each MSL derive the most coveted leadership knowledge, hence improving their professional growth and development at our company.
Any MSL Leader that is looking to create an enhanced 1:1 coaching experience with their MSL team should explore the opportunities current AI technologies offer. Our approach helped in creating a more robust Global Leadership Development Program. The success of our pilot led us forward with another GLDP program offering in 2024, with an even larger MSL group. We anticipate this personalized MSL learning approach will not only improve MSL professional development, but perhaps even enhance long-term retention statistics across our medical affairs function.
Our approach to leveraging machine learning to predict the future success of our GLDP program will provide greater insight and utility as the number of MSL participants increase. We hope to use the model output as a quantitative measure to complement the plethora of qualitative data emerging from our survey. This data-driven approach has the potential to digitally transform our traditional methodologies of assessing value or ROI from different learning strategies.
Syneos Health® is a leading fully integrated biopharmaceutical solutions organization built to accelerate customer success. We translate unique clinical, medical affairs and commercial insights into outcomes to address modern market realities. Together we share insights, use the latest technologies and apply advanced business practices to speed our customers’ delivery of important therapies to patients. We support a diverse, equitable and inclusive culture.
References
- Kanade, Vijay. 2022 August 30. What is Machine Learning? Definition, Types, Applications, and Trends for 2022. Retrieved January 10, 2024, from What is machine learning? Understanding types & applications (spiceworks.com)
About the authors:
Michael Shim, PharmD, Certified AI Scientist (CAISTM)
Michael Shim serves as a National Field Medical Director for the division of Deployment Solutions at Syneos Health. He is an innovative leader with specialized therapeutic experience in solid tumor oncology and is passionate about artificial intelligence and machine learning implementation in medical affairs.
Larry Dollar, PharmD, MS, MSL-BC
Larry Dollar is a Vice President of Medical Affairs at Syneos Health. He has also been a Senior Director and Field Medical Head at Takeda Oncology since 2014. Larry has also served in various MSL and MSL Director roles with Sanofi/Aventis, TAP, and Millennium Pharmaceuticals since 2000. He received his BS in Pharmacy from the University of Kansas, his MS in Pharmaceutical Sciences from the University of Missouri-Kanas City, and his PharmD from Idaho State University. His pharmaceutical career also includes experience in clinical research, medical research, and regulatory affairs. Larry currently lives in Prairie Village, Kansas, and enjoys spending time with his wife and four children.
Drew Beck
Drew has spent nearly two decades serving across the Life Sciences industry – from roles in direct patient care to leading teams across marketing, sales, and broader commercialization at Lilly, GSK, and Gelesis. Recently, he has also led innovation at Syneos Health, and in his current role, Drew is charged with building and evolving new ways for educating patients, providers, and his peers across the industry.
Cherie Hyder, PharmD
Cherie Hyder is a Senior Director Global Medical Affairs Excellence & Operations at Syneos Health. In her previous job at Biohaven Pharmaceuticals, she supported a virtual launch of Nurtec ODT for acute migraine. She has been involved in drug development for more than 30 years, working at FDA in CDER and pharmaceutical companies including Pfizer, Lilly, Novartis, Solvay, and Avanir, among others. At the University of Missouri, she received a Doctor of Pharmacy degree with the intention to devote her career to pharmaceutical research. She has multiple adjunct faculty appointments and enjoys teaching opportunities.
Jessie Scott, PharmD
Jessie Scott is currently the Syndicated National Director of Field Medical Excellence and Operations at Syneos Health and brings prior pharma experience in leading successful MSL teams. Jessie has experience in diabetes and cardiovascular disease. Her pharma experience includes launching several diabetes products, leading field medical product workstreams, working closely with US & Global on product strategy, executing new initiatives, and developing MSL medical plans plus putting insight into action. Prior to pharma, Jessie was an Assistant Professor at the University of Utah College of Pharmacy and an Internal Medicine Clinical Pharmacist VA Hospital Salt Lake City. She completed an Internal Medicine Residency in Austin following the U of Texas PharmD program and a BS in Pharmacy from Purdue University.
Have an opinion on this article? Send a message to the editor.