Training & Testing Software

The LANWrights Testing/Survey Engine

LANWrights Inc. has completed a second generation of JavaScript and Java tools designed to deliver tests and questionnaires or surveys across the Web, or from a diskette or CD-ROM. The first generation of these tools has already been used in over a million books and online instances, primarily to support CDs in the four-packs for Certification Insider's Exam Cram and Exam Prep series since 1997. As such, we have provided a testing environment for Microsoft, Novell, Cisco, Sun, CompTIA, and Oracle practice tests, among other certifications and platforms.

The LANWrights test engine supports a simple, powerful XML-based scripting language to help automate the creation of tests and surveys, coupled with easy, straightforward delivery of solicitations of user input across the Web. We also offer a variety of methods to collect results from interactive users. Organizations that wish to deliver tests to their employees or customers, or to solicit input or deliver surveys to such audiences, should find our technology easy to use and deploy, with equally easy access to results and statistics from the targeted populations.

Here's a printer-friendly version of this document, should you wish to print instead of view its contents. (Note that for best results, open this page on a machine running Internet Explorer 5.x or higher, with Word 2000 or higher installed.)

Feature Summary

Here is a summary of our test/survey engine's salient features:

Uses XML-based scripting to create test questions and survey items.
Supports the following question types:
- multiple-choice/single-answer
- multiple-choice/multiple-answer
- select values on scale (1 to 7)
- drag and place
- item ordering
- hot-spot selection (simulates "interface driving")
- multi-tabbed scenarios
Supports static question orders or adaptive testing, with up to five levels of difficulty.
Generates tests directly from question banks or from static scripts.
Includes support for statistical analysis to rank questions by difficulty or to accept/reject items by answer statistics.
Provides automatic test scoring, and includes item reviews to explain correct answers and distractors (after scoring).
Applet- or servlet-based implementation works on CD or Web server.
Includes POST and e-mail data delivery methods.
Works with any Web browser with Java Virtual Machine (JVM 1.1) or newer, on PC, Macintosh, and Linux/Unix computers.
Includes diagnostic, editing, and delivery tools to ease integration with existing Web or CD mastering operations.

Background

LANWrights' chief programmer, Dr. Bill Brogden, has consulted for the San Antonio-based Pyschological Corporation since 1993, performing upgrades and analyses of that company's academic testing programs. In addition, Dr. Brogden's PhD in Chemical Oceanography left him with a vast knowledge of statistical and numerical analysis, which he has put to work in the design of our test engine (primarily for support of question banks for adaptive tests).

Ed Tittel, LANWrights Principal, and other LANWrights employees taught HTML, XHTML, and XML classes at Austin Community College for over four years, and built a set of Web-based training materials for numerous online classes that incorporated the test engine for automatic grading of class exercises and exams.

Outside academia, LANWrights has developed online training and testing materials for Key3Media, Progressive Solutions, Sybex Books, the Coriolis Group, and other organizations. Its employees have written more than 150 computer trade books, including numerous titles on HTML, XHTML, XML, Web design, development, and programming techniques. In our work for Coriolis, we have developed over 100 different question banks, which have been reproduced collectively over a million times in books and via Web site downloads. Although some aspects of the test engine are new, our basic environment has been in production use since early 1998.

Engine Description

LANWrights has learned from its experience, and has developed a secure, easy-to-use testing engine that supports an interactive editor, but that is driven by XML-based scripts from a formal DTD. This permits test developers to work in any visual XML editor, if that's how they're comfortable. However, we find that once someone masters the XML scripting language, they can create new tests or surveys, or convert existing tests to XML, quite quickly using any plain-text editor and a bit of elbow grease. To facilitate quality control, we have built a lint utility for our scripts to check validity, syntax, and structure.

The LANWrights engine supports a wide variety of question types and formats, and user input options, including:

Multiple choice, single answer
Multiple choice, multiple answer
Arbitrary rating scales (usually 1-5 or 1-7)
Exhibit-driven questions, where graphics appear for analysis or evaluation
Simulations, where interaction with hot spots on a screen image is required
Adaptive questions, where correct or incorrect answers lead to new topics (correct) or more detailed questions on the current topic (incorrect)
Association of questions with categories and topics, so that content may be related back to specific sections of text in a book or course materials (or vice-versa)
Multi-tabbed scenario questions, where a text scenario may be broken across multiple tabbed panes, and questions, graphics, or explanatory text may appear on any such panes
Drag and place questions, where graphics may be dragged and placed on a display pane to order related items, elements in a series, or steps in a process
Free-form text input, which may limit the amount of data supplied (and perform matching checks) or permit strings of arbitrary length

LANWrights has built its test engine to support three kinds of tests:

Straight-line tests: A fixed sequence of questions (Microsoft calls this type a "fixed-length" test; Novell calls it a "form test") presented over a fixed period of time. For such tests, the test engine reproduces questions in the same order that they appear in the test script, unless instructed to pick a smaller number of questions than the supplied test script includes. The engine can match specific distributions of questions by category, if instructed to do so, and is supplied with the right mix of questions in its input script.
Adaptive tests: Allow an author to specify minimum and maximum numbers of questions to ask, and use a student's previous answer in a question category to set the level of difficulty for the next question from the same category. Thus, the sequence of questions varies from test to test, and usually requires a pool of questions at least three times greater than the maximum number of questions to be asked. Novell, Microsoft, and other vendors use adaptive tests on a variety of topics. The test engine includes question tracking and randomizing so that no two sequential tests in an ongoing series are likely to contain the same questions.
Short-form tests: A special 30-question, 1-hour version of a straight-line test, created to support Microsoft's 1999 introduction of certification tests of this type. The engine will select 30 questions from a question bank of any size to match specific distributions of questions by category.

Together, these features support the vast majority of questions or input types that tests, surveys, and questionnaires require. Even so, LANWrights stands ready to add support for whatever features a particular licensee may require.

The engine also supports conditional branching, so certain answers can provoke related lines of questioning, whereas others can provoke other lines of questioning or simply lead to the next question in a pre-defined sequence. This makes the tool as well-suited for delivery of questionnaires and surveys as it is for testing. It also makes it equally capable of handling standard sequential tests, as well as more modern adaptive tests.

The current production version of our code is written to the Java 2.0 Java Developer's Kit (JDK), and works with 1.2 and 1.1 browsers as well. The underlying applet that supports the engine is approximately 200 KB in size, but the download can be reduced if all Java object classes the applet can handle are not included. LANWrights will custom-compile only those object classes that your test requires to limit code size.

Because some organizations have issues with permitting Java applets to transit their firewalls, LANWrights has also built a servlet-based implementation of the test engine. This version delivers only plain-vanilla HTML to online users, thereby sidestepping Java security issues altogether. (Note: Because the servlet version only works across the Web from a Web server, for CD use, the applet version is required.) For applications where output from the applet or servlet is required, LANWrights can use signed versions of its code to output data or send e-mail from the test/survey engine itself.

Benefits of XML-based Scripting

Because our test engine uses XML-based scripting to define questions, answer options, and discussion elements for each answer option, our tests also make great teaching tools. That's because they deliver post-scoring information that explains why correct answers are correct and why incorrect answers are incorrect. Interviews with users have confirmed that this helps them better understand the questions and master related technical concepts and information.

But that's not the end of the benefits that our XML-based scripting can deliver. This basis also explains how our applet and servlet implementation can run from the same input as well. Even better, because the structure of XML documents is completely described in the DTDs that govern them, we can use this information to deliver test script information in a variety of different forms outside our test engine. These include the following:

Automatic conversion to Palm Pilot Flashcard format:
By running our test scripts through a conversion program, we can deliver question content in the form of digital flashcard files for download to Palm Pilots, allowing users to review our Q&A in a convenient, portable format.
HTML documents:
Because it's relatively easy to translate our XML markup into equivalent HTML, we can automatically deliver our exam scripts on the Web, or in a collection of static HTML documents.
Printed ASCII:
Because it's also easy to translate our XML markup into nicely-formatted pure ASCII files, we can also create printer-friendly versions of our script files so users can print hard copies of our materials and use them for review.

Best of all, because XML is easy to transform into other formats, we can add support for PDAs or handheld devices as our clients may request. We can also transform our data into other machine-readable formats for easy import into databases, text management systems, and the like.

Delivering Results

The test/suvey engine can deliver its results in any of a variety of forms, including:

Plain text e-mail (not recommended where security is a consideration)
Encrypted e-mail
Unencrypted CGI POST data (not recommended where security is a consideration)
Encrypted CGI POST data

The engine also supports delivery in formats that are easy to import into a database (as tab or comma-delimited text). We have also built tools to permit question banks to be imported into our XML scripting format. Here again, LANWrights can provide custom-programming services to connect our test engine to your database tools or data-handling environments, if these capabilities fail to dovetail with existing data management environments. We plan to add integration to SQL databases for question banks to our interfaces.

Demonstrations Are Available

You can visit samples of our test/survey engine at work through the following URLs on this Web site:

A simple Microsoft Certification test:
http://www.lanw.com/java/examcram
A sample SunSoft Java Certification test:
http://www.lanw.com/java/javacert
A sample questionnaire that shows branching, question styles, e-mail of results, and so forth:
http://www.lanw.com/java/quest/default.htm

For More Information

LANWrights is happy to discuss licensing of our engine or its underlying source code, with interested parties. Please e-mail etittel@lanw.com, or phone 512-252-7497, if LANWrights can be of any service to your organization. A variety of pricing options is available, including some that minimize up-front costs in favor of royalties or usage payments when products that incorporate our technology are sold to customers. LANWrights can also provide custom installation, content development, training, and software development services to help you integrate our engine smoothly into an existing or emerging Web environment.