Astrom Training Unveils IDE – How to use Big Data Analysis to Improve CBT Exams

Astrom Training Solutions has released its new Intelligent Data Engine (IDE), which uses advanced Big Data analysis to evaluate exam results. The main purpose of the IDE application is to help airlines and training centers resolve the following common challenges:

• How to relieve instructors from spending a large amount of time assigning difficulty levels to each question. Do you find this a challenging and highly subjective task?
• How to calculate a pilot’s true theory ability based on their clustered scores. How can Training Managers extract and evaluate the actual knowledge level of pilots?
• How to evaluate the overall quality of an exam. Is it an effective exam, and does it reflect the real ability of pilots?
• How to automatically monitor the quality of the question bank. Which questions are effective, and which questions should be reviewed, dropped, or changed?

Most exam systems use Classical Test Theory (CTT) to analyze exam results. CTT is an approach that is based on simple mathematics--primarily averages, proportions, and correlations.  It is more than 100 years old, but is still used quite often for exam evaluation. CTT is very simple and easy to understand, which makes it convenient for working directly with content authors to evaluate and diagnose tests, however, CTT does not provide a deeper understand of exam structure and is not optimum for improving examinations.

In Contrast, Item Response Theory (IRT) is a highly advanced approach for analyzing tests. Moreover, IRT is used not just to analyze exams; it is a complete psychometric framework that changes how question banks are developed, exam formats are designed, exams are delivered, and scores are produced. There are many benefits to the IRT approach that justify its complexity, and this is why most leading examination authors in the world now utilize IRT.

Our IDE Big Data solution was developed using IRT theory. IRT theory is predominantly based on the Rasch Model. However, our Big Data team has collaborated with a distinguished statistics professor to publish new research in this area based on Bayesian theory. Our IDE algorithms have been developed combining both the Rasch model and more recent advanced Bayesian algorithms to reach the optimal examination analysi.  Astrom’s IDE application now brings this advanced analytical capability to aviation Training Managers.

Press Release sourced from Astrom Training.

You may also check our Terms and Conditions for our Content Policy.