RSS

Category Archives: Industry Tips

ESTIMATE AT COMPLETION (EAC)

Project management is a continuous loop of planning what to do, checking on progress, comparing progress to plan, taking corrective action if needed, and re-planning. The fundamental items to plan, monitor, and control are timecost, and performance so that the project stays on schedule, does not exceed its budget, and meets its specifications.  Of course all of these activities are based on having an agreed upon Work Breakdown Structure (tasks/activities) on which to base the schedule and cost estimates.  During the planning phase of a project, the project manager with the assistance of the project team needs to define the process and procedures that will be used during the implementation phase to monitor and control the project’s performance.

Productivity in the pharmaceutical/biotech/medical device industry is going down. Some compounds have reached the billions expenditures cost without any guarantee that it will ever be approved or reach the market.  So how can we evaluate the performance of some of these clinical trials?

I will not go into details in the degree of project management activities managed and performed by a data manager since this can vary widely per company.  A good clinical data manager or manager of data management should be able to implement basic PM principles that will improve quality and timeliness of a clinical trial, regardless if the trial is fully outsourced (e.g. CRO performed most of the work).

You can find my article about the Role of Project Management in Clinical Data Management (2012) here for further reading.

So what is Estimate at Completion or EAC? or What is the project likely to cost?

There are several methods we could use to calculate EAC.

Let’s look at one formula. EAC =  AC (Actual Cost) + ETC (Estimate to Complete)  so what happens when you don’t know the ETC?

We could use the following formula to derive that value: ETC = (BAC – EV) / CPI =>>>>??? So what? More formulas? How do I get BAC or EV or CPI?

Let’s look at those in more details.

 BAC =>>>Budget at Completion (how much did you
budget for the total project?)
CPI =>>> Cost Performance Index (CPI): BCWP/ACWP

EV = Earned Value

Earned Value Analysis example for a phase 1 trial (*figures in the thousands / millions = fictitious  numbers)

The final clinical trial results includes 100 subjects. The estimated cost is $20 per subject.  That results in an estimated budget of $2000 (100 x 20). During the planning, the CRO indicated that would be able to enroll 5 subjects per week.  Therefore the estimated duration of the trial is 20 weeks (100 / 5)

EV blocks: From the project plan

Estimated Budget: $2000

Estimated Schedule: 20 weeks

Planned Value (PV): at the end of the trial is $2000

Variance between planned and actual at the end of the first week:

Based on the estimated scheduled, I should have 25 subjects enrolled. At $20 per subject, the planned value at the end of the week is $500 (25 x 20)

PV = $500

At the end of the first week, the CRO reports that he has enrolled 20 subjects  and the actual cost of that study is $450. With this information we can look at schedule and cost variance.

SV = EV – PV

SV = $400 – $500 = – 100 ($100 work of subject recruitment is behind schedule).

CV = EV – AC

CV = $400 – $450 = -50 ($50 work of the project is over budget)

*negative figures means bad.

Using early results to predict later results:

Schedule Performance Index (SPI)

SPI = EV/PV

SPI = 400/500 = .80

Cost Performance Index (CPI)

CPI = EV/AC

CPI = 400/450 = .89 –> over budget or expending more

These rations can be used to estimate performance of the project to completion based on the early actual experience.

Estimate to Completion (ETC)
ETC= (PV at completion) – EV)/CPI

ETC= (2000 – 400)/CPI

ETC = (1600/.89) =$ 1798 from end of week one (after 5 days) and it will take additional $1798 to complete the study

Estimate at Completion (EAC)

EAC = AC + ETC

EAC = 450 + 1798 = $2248

If nothing changes, based on the actual results at the end of the first week, the study is estimated  to cost $2248 (rather than the planned cost of $2000) and will take 20 percent longer.

The formulas assumes that the accumulative performance reflected in the CPI is likely to continue for the duration of the project.

You do not need to memorize all of these formulas. There are plenty of tools in the industry that does the computation for you. But if you do not have it available, you can use Excel, set-up your template and plug in the numbers.

Earned Value

 

 

 

 

 

 

 

As per PMI – PMBOK definition, Cost management “…includes the processes involved in estimating, budgeting, and controlling costs so that the project can be completed within the approved budget.”   A Guide to the Project Management Body of Knowledge (PMBOK® Guide).

We have shown you, that PM tools such as Earned Value  Analysis, can be applied to clinical trials or specific work break down (WBS) activities within the data management team.

Based on the above outcome of the project performance related to the schedule, the data manager should be able to determine if she should modify the current plan or revise the original plan.

It is a perfect tool for data managers and managers of data managers and could be part of your risk based processes.

If bringing efficiency, improving data quality and significantly reducing programming time after implementing CDISC standards is on your radar screen, I’d love to chat when it’s convenient. All the best.

Anayansi Van Der Berg has an extensive background in clinical data management as well as experience with different EDC systems including Oracle InForm, InForm Architect, Central Designer, CIS, Clintrial, Medidata Rave, Central Coding, OpenClinica Open Source and Oracle Clinical. SAS, CDASH/SDTM (CDISC standards implementation and mapping), SAS QC checks and clinical data reporting.

Source:

A Guide to the Project Management Body of Knowledge (PMBOK® Guide).

Notes from my PM class at Keller 2007-2009

Images – Google images

FAIR USE-
“Copyright Disclaimer Under Section 107 of the Copyright Act 1976, allowance is made for “fair use” for purposes such as criticism, comment, news reporting, teaching, scholarship, and research. Fair use is a use permitted by copyright statute that might otherwise be infringing. Non-profit, educational or personal use tips the balance in favor of fair use.”

Advertisements
 

Tags: , , , , , ,

Solving Data Collection Challenges

Solving Data Collection Challenges

Cross-partnership between sponsors and CROs for the collection and analysis of clinical trial data are complex. As a result there are a number of issues encountered during the running of  trial.

As with many projects, standardization projects like CDISC is a huge undertake. It requires resources, technology and knowledge-transfer. The industry (FDA for example) has been working on standardization for years but on September 2013, it became official, in which the FDA released a ‘Position Statement‘.

 Data Collection

According to the WHO, data collection is defined as the ongoing systematic collection, analysis, and interpretation of health data necessary for designing, implementing, and evaluating public health prevention programs.

Sources of data: primarily case report books or (e)CRF forms, laboratory data and patient report data or diaries.

 Challenges of data collection

It is important for the CROs / service providers to be aware of the potential challenges they may face when using different data collection methods for partnership clinical studies. Having several clients does not mean having several standards or naming conventions. This is the main reason why CDISC is here. So why are many CROs or service providers not using CDISC standards?

Another challenge is time limitations. Some clinical trials run for just a few weeks / months.

It may be found difficult to understand the partnership in the amount of time they have. Hence, most CROs and service providers prefer to perform manual mapping at the end of the trial, hence, re-work and manual work.

Funding also plays a key challenge for CDISC-compliance data collection study. Small researchers or biotechnology companies that do not have the resources in-house, out-sourced this task to CROs or service providers and are not interested whether it is compliance as long as it is save them money. But would it save money now instead of later in the close-out phase?

If there is a shortage of funding this may not allow the CRO or service provider all the opportunities that would assist them in capturing the information they need as per CDISC standards.

We really don’t have the level of expertise or the person dedicated to this that would bring, you know, the whole thing to fruition on the scale in which it’s envisioned – Researcher

Role of the Library

There is a clear need for libraries (GL) to move beyond passively providing technology to embrace the changes within the industry. The librarian functions as one of the most important of medical educators. This role is frequently unrecognized, and for that reason, too little attention is given to this role. There has been too little attention paid to the research role that should be played by the librarian. With the development of new methods of information storage and dissemination, it is imperative that the persons primarily responsible for this function should be actively engaged in research. We have little information at the present time as to the relative effectiveness of these various media. We need research in this area. Librarians should assume an active role in incorporating into their area of responsibility the various types of storage media. [http://www.ncbi.nlm.nih.gov/pmc/articles/PMC232677/]

Review and Revise

At the review and revise stage it might be useful for the CRO or service provider to consider what the main issues are when collecting and organizing the data on the study. Some of these issues include: ensuring sponsors, partners and key stakeholders were engaged in the scoping phase and defining its purpose; the objectives have been considered; the appropriate data collection methods have been used; the data has been verified through the use of multiple sources and that sponsors have approved the data that is used in the final clinical data report.

Current data management systems must be fundamentally improved so that they can meet the capacity demand for secure storage and transmission of research data. And while there can be no definitive tools and guideline, it is certain that we must start using CDISC-standards from the data collection step to avoid re-inventing the wheel each time a new sponsor or clinical researcher ask you to run their clinical trial.

RA eClinica is a established consultancy company for all essential aspects of statistics, clinical data management and EDC solutions. Our services are targeted to clients in the pharmaceutical and biotech sector, health insurers and medical devices.

The company is headquarter in Panama City and representation offices with business partners in the United States, India and the European Union.  For discussion about our services and how you can benefit from our SMEs and cost-effective implementation CDISC SDTM clinical data click here.

 

 

Tags: , , , , , , , , , ,

Marketing without Research

The topic of marketing research is an excellent place to start learning about marketing. A marketing campaign is only as good as the research that backs it up.
Consider the case of Coca Cola introducing “New Coke”, replacing their traditional formula. The product caused such a sensation that television programs were interrupted to tell consumers.

My question to you all is this? Why did New Coke fail so miserably? What was wrong with the research that Coke did before they introduced the product ?

Focus groups are still very popular today when doing initial marketing research. I have run focus groups and participated in many.

I remember when my marketing professor in my MBA program did some consulting for the name for a new drink. The two names that he did focus group research on were “Rattlesnake Cooler” and “Cactus Cooler.” I am sure you all know Cactus Cooler now, and it is a very successful product.

anayansi gamboa Cactus Cooler

I think one of the things researchers did was a focus group. The results of the focus group were much more negative than expected so the researchers did a different survey. The results of that survey were negative, but not as much as the focus group. They already knew what they were going to end up doing and they didn’t listen to what the surveys said. With such a common product as Coke, you have to be careful with changes. They didn’t listen to what their consumers were telling them.

Think that when a mega-company like Coca Coke disregards focus group results, it shows that they really care about the profits they’re after and not keeping loyal customers. Part of the marketing department’s job is not to create the thirst for Coke, but to create the desire for Coke over another soft drink. The whole point of using a focus group should be to get a handle on what the public wants, not what the company wants the public to drink.

According to the text, “companies can conduct their own marketing research or hire other companies to do it for them. Good marketing research is characterized by the scientific method, creativity, multiple research methods, accurate model building, cost-benefit analysis, healthy skepticism, and an ethical focus.” A perfect example of this is to remember the part in the movie “Daddy Day Care” where Eddie Murphy’s character is responsible for marketing cereal to kids. What he does is holds a focus group for the kids to see if they like the cereal. Focus groups are great if researchers are willing to listen to the people they are tying to sell to.

That is an excellent point. If the people who hired the focus group professionals, they need to be prepared to listen to the results. Too often, company executives or others in the company have “pet projects” that they want to see succeed no matter what the cost. There are many cases where marketers or other high level people did not listen to what data the focus group produced.

This is a fatal mistake.

This is one reason why companies are outsourcing a lot of marketing functions because everything is so specialized. Rather than having a large internal marketing force, companies will often outsource focus groups or other research to ensure that the results are unbiased. Of course, if the company execs don’t listen, the research is wasted.

What did the focus group concentrate on?

Source: {EDC Developer}

 

 
 

Tags: , , , , , , , ,

From Non-SAS Programmer to SAS Programmer

SAS Programmers come from many different educational backgrounds. Many has started their careers as a Data Manager in a CRO environment and grew to become a SAS programmer. Others have gone to college and pursued degrees in math, statistics or computer science degree.

Do you have SAS Skills? First, you need to find out more about statistical programming desire skills and start to slowly learn what SAS programmers and statisticians do in the pharmaceutical industry. It is also important to understand the Drug Development and Regulatory process so that you have a better understanding of the industry as a whole as well as the drug approval process.

In addition, I have personally attended several workshop on Statistics for Non-statistician provided by several of my past employers/clients (GSK, Sanofi-Aventis, etc) so I could have a greater understanding of statistics role. I am personally more inclined to the EDC development than becoming a biostatistician but these are just some of the few steps you could take to grow your career as a SAS programmer.

Practice, Practice, Practice!

To begin learning how to actually program in SAS, it would be a good idea to enroll to a SAS course provided by the SAS Institute near you or via eLearning. I have taken the course SAS Programming 1: Essentials, and I would recommended. You could also join SUGI conferences and other user groups near your city/country. Seek every opportunity to help you gain further understanding on how to efficiently program in the pharmaceutical industry. It could well land you a Junior SAS programming position.

Transitioning to a SAS Programming role: Now that you have gotten your first SAS programming job, you will need to continue your professional development and attend additional training, workshops, seminars and study workgroup meetings. The SAS Institute provide a second level, more advance course Programming II: Manipulating Data with the Data Step, SAS Macro Language and SAS macro Programming Advanced topics. There are also SAS certifications courses available to help you prepare to become a SAS certified programmer.

There is a light at the end of the tunnel: Advance!

Your ongoing development will be very exciting and challenging. Continued attending SAS classes as needed and attending industry related conferences such as PharmaSUG to gain additional knowledge and insight on how to perform your job more effectively and efficiently.

As you can see, it is possible to ‘grow’ a SAS programmer from a non-programming background to an experience programmer. All of the classes, training, and projects you will work on are crucial in expanding your SAS knowledge and will allow you to have a very exciting career opportunity ahead of you.

Anayansi Gamboa has an extensive background in clinical data management as well as experience with different EDC systems including Oracle InForm, InForm Architect, Central Designer, CIS, Clintrial, Medidata Rave, Central Coding, OpenClinica Open Source and Oracle Clinical.

 

Tags: , , , , , , , , , , ,

How to write query texts – 6 template sentences

How to write queries unambiguously expressing what is asked for? Using short, polite sentences? Objectively explaining the underlying inconsistency?

First of all my general guidelines.

My preference is to use no more capitals then needed. Capitals in the middle of a query text, e.g. for CRF fields or for tick box options, could distract from getting the actual question asked. E.g. compare the same query texts, with and without extra capitals. Please verify stop date. (Ensure that stop date is after or at start date and that stop date is not a future date.) Please verify Stop date. (Ensure that Stop date is after or at Start date AND that Stop date is not a future date.)
Referring to CRF fields as they are shown on the CRF. To easily find the involved field(s).
I prefer to leave any ‘the’ before a CRF field referral out of the query text. For more to-the-point query texts. E.g. compare the same query texts, with and without ‘the’ before data fields. Please verify stop date. (Ensure that stop date is after or at start date and that stop date is not a future date.) Please verify the stop date. (Ensure that the stop date is after or at the start date and that the stop date is not a future date.)
Consistency in phrasing a query text can help to quickly write query texts or pre-program query texts in a structured, familiar way. That’s the thought behind the following 6 template sentences for query texts. Which you can use to help you write or program your queries.

The six ‘template’ sentences for query texts:

Please provide…

For asking the study site people to provide required data from patient care recordings. Examples: Please provide date of visit. Please provide date of blood specimen collection. Please provide platelet count. Please provide % plasma cells bone marrow aspirate. Please provide calcium result.

Please complete… For asking the study site people to complete required data as required by the study CRF design. (Not necessarily required for patient care). Examples: Please complete centre number. Please complete subject number. Other frequency is specified, please complete frequency drop-down list accordingly.
Please verify…

For asking the study site people to check date and time fields fulfilling expected timelines. Or for asking the study site people to check field formats. Examples: Please verify start date. (Ensure that start date is before date of visit.) Please verify stop date. (Ensure that stop date is after or at start date and that stop date is not a future date.) Please verify date of blood specimen collection. (Ensure that date of blood specimen collection is before or equal to date of visit and after date of previous visit.) Please verify date last pregnancy test performed. Please verify date of informed consent. (Ensure date of informed consent is equal to date of screening or prior to date of screening.) Please verify date as DDMMYYY.

…., please correct.

For asking the study site people to correct a data recording inconsistent with another data recording. Example: Visit number should be greater than 2, please correct.

…., please tick…

For asking the study site people to complete required tick boxes. Examples: Gender, please tick male or female. Pregnancy test result, please tick negative or positive. Any new adverse events or changes in adverse events since the previous visit, please tick yes or no. Laboratory assessment performed since the previous visit, please tick yes or no. LDH, please tick normal, abnormal or not done.

Please specify…

For asking the study site people to specify the previous data recording. Examples: Please specify other dose. Please specify other frequency. Please specify other method used. Please specify other indication for treatment.

Finally, for query texts popping up during CRF data recording, it could be helpful to put location information in it. Like: Page 12: Please verify start date. (Ensure that start date is after or at start date on page 11.)

Good luck finding your way to structure query texts…

Source:

This article is written by Maritza Witteveen of ProCDM. For clinical data management. You can subscribe to her blog posts at www.procdm.nl.”

 

Tags: , , , , , , , , ,

Data verification puzzles

 

Important part of the data management job is to verify received data. Checking for inconsistencies and unexpected patterns. Verifying that the data is complete, legible, logical and plausible.

However, how to perform data verification?

You could regard the data verification job as completing a couple of puzzles. Each puzzle is one subject participating in the clinical trial or clinical study at stake. As such, the puzzles resemble each other a great deal. But they are not exact copies. Each subject, each puzzle, is (slightly) different, unique.

Pleasant and thoughtful team action:

Do you have a puzzle somewhere in a cupboard? More than one from the same series? At least 2 puzzles with > 100 pieces each? Open the boxes, drop their content in one pile on the table and start completing the puzzles/subjects. The more pieces in place of a puzzle, the more evident which pieces to expect.

1. Get the parts received, divide them per subject/puzzle and start making all the puzzles. The clinical information up on each subject is coming in pieces, per completed visit data, per available adverse event information. In the beginning you’ll thus work with lots of incomplete puzzles.

2. Any holes in any puzzle/subject, any missing parts, you need to look for/query. Note that holes are allowed if your puzzle/story is as such! However, leave no unexpected holes. Meaning that if an assessment took place, you want to have the corresponding result(s) completed.

3. Any duplicate pieces, get rid of them. Please query.

4. Any pieces not fitting your puzzle/subject story, you need to check up on. Maybe they belong to another puzzle/subject. Or they are incomplete and can therefore not fit (yet). They could even be wrong delivered and not belong to the study at all.

5. Any pieces fitting but rotated 90 or 180 degrees, please turn/query. Get the puzzle showing a logical story.

6. Any pieces damaged, please try to fix the damaged parts. E.g. spilled coffee over a paper CRF. Illegible text parts. Or unclear texts that can be interpreted differently.

7. Any pieces added at the wrong place, query and bring to their right position. E.g. an error in an assessment date.

In trial/study language, the more data for a subject received and in the database, the easier to get the subject’s story complete. However, the more care needed to get the true story. The logical, plausible subject story. Attention to medication given for an adverse event but missing in the concomitant medication list. Or laboratory shifts to worse results but missing corresponding adverse events listed.

Completing the holes in a puzzle is easy, for data management the edit checks help you tremendously with that. Getting a logical, plausible story for each patient, reflecting the truth, is the real data management challenge. Which takes more than just structuring pieces. It asks you to look and understand the pictures up on the pieces received.

Good luck with your data management puzzles,

Good regards,

Source:

“This is an article of ProCDM. Clinical data management training. Receive tips and the free e-book ‘Five strategies to get reliable, quality clinical data’ by subscribing via http://www.procdm.nl/pages/knowledgebase.asp.”

 

Fair Use Notice: This article/video contains some copyrighted material whose use has not been authorized by the copyright owners. We believe that this not-for-profit, educational, and/or criticism or commentary use on the Web constitutes a fair use of the copyrighted material (as provided for in section 107 of the US Copyright Law. If you wish to use this copyrighted material for purposes that go beyond fair use, you must obtain permission from the copyright owner. Fair Use notwithstanding we will immediately comply with any copyright owner who wants their material removed or modified, wants us to link to their website or wants us to add their photo.

 

Tags: , , , , , , ,

Aside
ABSTRACT
Much effort goes into the specification, development, testing and verification of programmatic edit checks to ensure that the error rate in clinical trial data is sufficiently low as to have no statistically significant effect on the overall trial results. An analysis of several thousand clinical trials, containing over 1.1 billion data values and 1.1 million edit checks, shows that the majority of edit checks (60%) have no impact on data quality; none of these 678,000 edit checks have generated a single data query or discrepancy. What can be learnt from this analysis; can we reduce the overall number of edit checks without compromising data quality; can we identify the ‘high-performing’ edit checks and improve CRF design to avoid data entry errors; are there novel methods that might achieve similar standards of data quality with less effort?

 

Edit checks are necessary to ensure data quality reaches acceptably high levels.

Since programming edit checks takes time and resources, it’s important to ensure that the effort invested maximizes the benefit and re-usability of each edit check.

 

See attached document for full article information published by:
Optimizing Data Validation by Andrew Newbigging, Medidata Solutions Worldwide, London, United Kingdom

 
 

Fair Use Notice: This article/video contains some copyrighted material whose use has not been authorized by the copyright owners. We believe that this not-for-profit, educational, and/or criticism or commentary use on the Web constitutes a fair use of the copyrighted material (as provided for in section 107 of the US Copyright Law. If you wish to use this copyrighted material for purposes that go beyond fair use, you must obtain permission from the copyright owner. Fair Use notwithstanding we will immediately comply with any copyright owner who wants their material removed or modified, wants us to link to their website or wants us to add their photo.

Complexity and effectiveness of edit checks

 

Tags: , , , , , , , , , , , , , , , , , , ,

 
%d bloggers like this: