
                                                                         Page 1
                           True Score (v5.00)


                               HELP PARTS
                   (Print Courier New, Font Size 10)

This version of True Score introduces automatic operation.  With scanner and
True Test Writer files on a floppy disk, click TRUESC.EXE to score the test.  
Print selected reports.  Save all materials related to the test including the 
floppy in your office file cabinet.  The floppy contains a complete record of 
the printed test, multiple versions, answer files, and all of the scoring and 
analysis reports.  It can be rescored for new grades and advanced analyses.

There is a tradeoff between automatic and manual operation.  Automatic (two 
click) operation requires True Text Writer to create tests so multiple-form 
tests can be scored and analyzed as if only one form had been administered.  
Manual operation requires the user to create and select the answer files for 
processing.  Follow the green screen prompts for specific help.

Read pages 2-5 to set up the program for your style and equipment.  Read the 
reference manuals and email support@nine-patch.com for information on more 
advanced use of the software.  Delete the TRUESC folder to remove the program 
from your computer.  

     HELP PARTS  . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1 
     Main Features . . . . . . . . . . . . . . . . . . . . . . . . . . .  2
     A.  Secretary Setup . . . . . . . . . . . . . . . . . . . . . . . .  3
     B.  Teacher Setup . . . . . . . . . . . . . . . . . . . . . . . . .  4
     C.  Your Favorite Word Processor Setup Switch . . . . . . . . . . .  5
     D.  Custom Answer Check Setup Switches  . . . . . . . . . . . . . .  6
     E.  Custom True Score Setup Switches  . . . . . . . . . . . . . . .  7
     F.  Score and Rescore (Get it Right). . . . . . . . . . . . . . . .  8
     G.  Automatic Protocol  . . . . . . . . . . . . . . . . . . . . . .  9
     H.  Semi-Automatic Protocol . . . . . . . . . . . . . . . . . . . .  9
     I.  Manual Protocol (True Test Writer Files not Required) . . . . . 10
     J.  Folders for Manual (Advanced) Operation . . . . . . . . . . . . 11

                             REFERENCE MANUALS

     K.  Answer File Preparation and Check . . . . . . . . . . . . . .  A-1
     L.  Score and Analyze . . . . . . . . . . . . . . . . . . . . . .  S-1

         License and Warranty  . . . . . . . . . . . . . . . . . . . . S-28 
         Order and License Registration Form . . . . . . . . . . . . . S-29
         
     ----------------------------------------------------------------------

ACKNOWLEDGEMENTS:  I am grateful to the faculty and staff of the Department of 
Nursing, Southeast Missouri State University, for their sincere interest in an 
instructional management system.  It was the perfect environment in which to 
transform student improvement research into practical applications that provide 
better, faster, easier, and less expensive feedback.  The software does the 
clerical work needed to optimize instruction, learning and testing for 
teachers and students.


                                                                         Page 2
                           Nine-Patch Software
                   Write, Take, Score and Analyze Tests

                           True Score (v5.00)

A.  Inputs:

    1.  Student marks (scanner or manual entry).
    2.  Corrections for bad questions, typo's and other errors.
    3.  For multiple-form tests, the test file and item analysis index 
        files created by True Test Writer.
    4.  Traditional testing (answer all, knowledge) or students select
        questions to report what they actually know or can do (knowledge 
        and judgment).

B:  Outputs:

    1.  Standard score and item analysis for grading.
        
        Includes score and mark summary, ranked raw scores, item analysis
        (mark distribution, difficulty, discrimination, Kuder-Richardson 20).
        Creates one analysis from multiple-form tests.

    2.  Advanced analysis for test improvement and student advisement.

        Includes questions grouped by topic and level of thinking,  
        student performance by topic and individual question worksheets 
        (who marked and the average score for each question option).

C.  Installing:

    1.  Run directly from the floppy disk, or
    2.  Drag (copy) the TrueSc folder to your hard drive.
    3.  Delete the TrueSc folder to uninstall.

D.  Automatic operation from floppy disk or hard drive:

    1.  Enter marks or copy scanner output to a floppy disk.
    2.  Copy test files and item analysis index files to the floppy.
    3.  Click TRUESC.EXE to run program.
    4.  Print results.
    5.  Save floppy with your tests in your office file cabinet.

E.  Manual operation:

    1.  Reset switches for your choice of features.  Press F5 for changes that 
        are frequently made or edit the switch files (pages 6 and 7).
    2.  Select and click on-screen (green) instructions.


http://www.nine-patch.com  FAX 573-445-4776  VOICE -7275  support@nine-patch.com





                                                                         Page 3
A.  Secretary Setup


This setup is based on the assumption you:

1.  Create and duplicate tests.
2.  Send answer sheets and a new floppy disk to a scanner/scoring service.
3.  Score and analyze student responses.
4.  Print and return results to teachers.
5.  Include all data with True Score on a floppy.


Desktop Shortcut Icons:

1.  Run True Score on Drive C

    Copy the TRUESC folder to the hard disk or unzip the program in a TRUESC 
    folder.  Running the program from a hard disk is much faster than running it
    from a floppy.  Right click on TRUESC.EXE to create a shortcut.  Drag the 
    shortcut icon to your desktop.

2.  Copy and Install True Score from Drive C to Drive A

    Right click on TscC2A.BAT to create a shortcut.  Drag the shortcut icon to  
    your desktop.  This command will copy ONLY the True Score program files on
    drive C into a TRUESC folder on a floppy disk in drive A.


Protocol:

1.  Copy the file(s) used to print the test and the item analysis index file(s) 
    to the floppy disk returned from the scanner service.
2.  Click True Score to score and analyze the test on the disk in the A drive.
3.  Print the results.
4.  Click Copy True Score (TscC2A.BAT) to copy and install ONLY the TRUESC 
    program files into a TRUESC folder on the floppy disk.
5.  Deliver the disk and the scoring service and True Score results to the 
    teacher.


















                                                                         Page 4
B.  Teacher Setup
   

This setup is based on the assumption you:

1.  Create your own tests or have help.


2.  Use a scanner/scoring service or enter student responses by hand.
3.  Do the first scoring yourself or have it done for you.
4.  Do one or more rescorings to "get it right" (correct all manner of errors).
5.  Keep each test on one floppy with all scoring and analyses.


Desktop Shortcut Icons:

(No shortcuts are needed if you click on TRUESC.EXE to run the program.)

1.  Run True Score on Floppy A

    Locate the "Shortcut to TRUESConA" in the TRUESC folder.  Drag the shortcut
    to your desk top.

2.  Run True Score on Your Hard Disk.

    Drag (copy) the TRUESC folder to the hard disk.  Running the program from  
    a hard disk is much faster than running it from a floppy.  Right click on   
    TRUESC.EXE to create a short cut.  Drag the shortcut icon to your desk top.


Protocol:

(If you do your own tests, please see the Secretary Setup Protocol on page 3.)

1.  Review the printouts and decide if any error needs to be corrected.  If so, 
    put the floppy into the A drive and start True Score again.  On rescoring, 
    it will stop at the point you can either select scanner files to reprocess 
    or edit the old answer (ANS) files.  Changes made directly to the answer 
    (ANS) files are permanent, until the scanner files are reprocessed.

2.  See Score and Rescore (Get it Right) for instructions to drop a question,
    drop a question but give each student one point, change the answer key, 
    and other common errors (page 6).


Comments:

1.  The F5 key permits changing between standard and advanced analysis.  You  
    can select a score export file that matches your grade book program (page   
    S-25).

2.  Follow the instructions on the Advanced Question and Student Analysis page 
    to enter keys to group questions or students in any way that you need (page   
    S-26).



                                                                         Page 5
C.  Your Favorite Word Processor Setup Switch


Six word processor switches are included in CustumA.INI (Answer Checking and 
Input) on page 6 and CustomS.INI (Scoring) on page 7.  Each has an advantage.  
For compatibility, file names are limited to eight characters, the (.), and the 
three character extension.

    1. :WordPro=NOTEPAD.EXE                              :Windows 3.x, 95, 98 

       NotePad creates TEXT files.  All Nine-Patch Software input files are 
       TEXT files.  All word processor formatting is discarded in a clean TEXT 
       file.  Very large files cannot be loaded into NotePad.

    2. :WordPro=WRITE.EXE                                :Windows 3.x

       Good for printing score and analysis results.

    3. WordPro=C:\Progra~1\Access~1\WORDPAD.EXE          :95, 98

       Good for viewing files.  This switch is on when the software is shipped 
       as WORDPAD works on large files on Windows 95 and Windows 98.  

    4. :WordPro=C:\Progra~1\Micros~1\Office\WINWORD.EXE  :95

       Good for printing score and analysis results.

    5. :WordPro=C:\Progra~1\Micros~5\WINWORD.EXE         :98
       :WordPro=C:\Progra~1\Micros~3\WINWORD.EXE         :98

       One of these should work on your computer.  If not, use Windows Explorer 
       to display the C:\Program Files folder.  Then open the Program Files 
       folder and display the Microsoft folders.  In the above examples, 
       WINWORD.EXE (WORD), was held in the fifth or the third Microsoft folder      
       from the top.

To find the full path to WINWORD.EXE, click START, then FIND, then FILES AND 
FOLDERS.  Enter WINWORD.EXE in the Name window.  Check INCLUDE SUBFOLDERS.  
Click FIND NOW.  Place your cursor on the divider between IN FOLDER and SIZE 
and drag the divider to the right far enough to see the enter path.

If you have a different word processor than any of these, use FIND to get the 
complete path.  Use Windows Explorer to locate the correct folder and the 
folder count (~3, for example).












                                                                         Page 6
D.  Custom Answer Check Setup Switches  


:Custom Answer Check (CustomA.INI)
:Expectations are that these switches will be set once and not changed.
:
:To change the switches, edit the (:) at the start of the line.
:Remove the (:) to activate a line.
:Insert the (:) to deactivate a line.
:
:
:ButtonSkip=0             :no buttons skipped (manual)
ButtonSkip=1             :skip check and save buttons (automatic)
:
:ADrive=0                 :look on hard disk (any disk) 
ADrive=1                 :Look for scanner output only on disk A:\
:
:APattern=*.*             :Look for all files (default)
APattern=*.txt;*.dat     :Look for nurse.txt (or edit, such as, exam1.dat)
:
:ANS2A=0                  :save ANS to parent drive (default)
:ANS2A=1                  :save ANS to A drive
ANS2A=2                  :save ANS to A and parent drives
:
:WordPro=NOTEPAD.EXE      :Windows 3.1, 95 and 98 (default)
:WordPro=WRITE.EXE        :Windows 3.1      loads short named files
WordPro=C:\Progra~1\Access~1\WORDPAD.EXE         :95 & 98
:WordPro=C:\Progra~1\Micros~1\Office\WINWORD.EXE  :95
:WordPro=C:\Progra~1\Micros~5\WINWORD.EXE         :98
:WordPro=C:\Progra~1\Micros~3\WINWORD.EXE         :98
:
:StudentID=0              :nine character ID field (SS#) (default)
StudentID=1              :14 character name field (7 last name and 2 initials)
:StudentID=2              :Scanner sequence number
:
Message1=0               :operating messages off

:Message1=1               :operating messages on 
:
Scanner1=0               :National Computer Systems  (Set 31Oct99)
:Scanner1=1               :Chatsworth                (Manual entry)
:Scanner1=2               :Scantron                  (Separate program)
:Scanner1=3               :Scanning Systems          (To be developed)
:Scanner1=4               :Apperson                  (Separate program)













                                                                         Page 7
E.  Custom True Score Setup Switches


:Custom Score (CustomS.INI)
:Expectations are that these switches will be set once and not changed.
:
:To change the switches, edit the (:) at the start of the line.
:Remove the (:) to activate a line.
:Insert the (:) to deactivate a line.
:
:DiscrimLevel=0           :ABC&D  D indicated but not summed (default)
DiscrimLevel=1           :ABCD&E E indicated but not summed
:                         :A=1%, B=5%, C=10%, D=20%, E=30% LOS
:
:WordPro=NOTEPAD.EXE      :Windows 3.1, 95, 98 (default)
:WordPro=WRITE.EXE        :Windows 3.1      loads short name files
WordPro=C:\Progra~1\Access~1\WORDPAD.EXE        :(95, 98)
:WordPro=C:\Progra~1\Micros~1\Office\WINWORD.EXE  :(95)
:WordPro=C:\Progra~1\Micros~5\WINWORD.EXE      :98
:WordPro=C:\Progra~1\Micros~3\WINWORD.EXE      :98
:
:RESEXT=short             :results extension short name (95)
:RESEXT=.txt              :results extension, long name
RESEXT=.doc              :results extension, long name
:
Message1=0               :operating messages off
:Message1=1               :operating messages on 
:




























                                                                         Page 8
F.  Score and Rescore (Get it Right)


The second step in quality management related to scoring is rescoring.  Getting 
the scores right.  Rescore to adjust for bad questions for a variety of 
reasons.  


1. Operation:

   a. Automatic:

      The first scoring requires running TRUESC.EXE and printing out the 
      results.  On following runs, the program will stop to give you the choice 
      of reprocessing the scanner files or edit the old answer (ANS) files.  
      Select an answer file.  Click on Edit to make needed changes.  These 
      changes are permanent unless you reprocess the scanner files.

   b. Manual: 

      The program now runs in two parts: True Answer Check (TRUEAC.EXE) and 
      True Score (TRUESC.EXE).  You can use Edit in True Answer Check to make 
      corrections as discussed above or double-click on the answer file in True 
      Score and use your word processor.  These make permanent changes in the 
      answer (ANS) files as long as you do not reprocess scanner files.


2. Corrections:

   a. Totally bad item (drop it):

      Replace (A) with a blank ( ) space on the weight line coded 999000222.        
      It is the second line in the first answer file of a multiple-form test.

   b. Totally bad item (give credit):

      Replace the weight (A) with (G) for one point or (H) for two points.

   c. Two right answers:

      Answer (D) is right.  In class, a student points out that (B) is also  
      right, or on looking at the first scoring printout you observe this 
      condition.  Edit all the (B) responses to (D) on the first alternate form 
      of a multiple-form test.  Use Edit in True Answer Check or your own word 
      processor and always save to a TEXT file.  Then correct the next 
      alternate form for the appropriate letters for the key and new right 
      answer. 

   d. The key is wrong:

      Make the correction in the key (coded 999000111) of each alternate form.  
      Use Edit in True Answer Check or your own word processor and always save  
      to a TEXT file.



                                                                         Page 9
G.  Automatic Protocol

    1.  Create and print the paper test using True Test Writer.

    2.  Copy test scanner output onto a floppy disk (or enter student
        responses manually).

    3.  Copy test files (.FIL) and item analysis index files (.IAI) created by 
        True Test Writer to the floppy disk.

        [For practice, copy the 206 scanner, test, and IAI files to the floppy.]

    4.  Click TRUESC.EXE to score and analyze the student responses.

    5.  Print selected reports.

    6.  Click TscC2A.BAT to install True Score on the floppy disk.

    7.  Deliver all test materials to the teacher.  If True Score has       
        been installed on the floppy, the teacher can rescore the
        test as needed for bad questions, typo's, and other errors 
        found in the first scoring.

        (When the test is rescored, the program automatically changes 
        to semi-automatic operation.  See below).

Restrictions:

1.  All file names are limited to 8 characters, (.), and a 3 letter 
    extension (Windows 3.x, Windows 95, and Windows 98 compatible).
2.  All test files must end with .FIL.
3.  All item analysis index files must end with .IAI.
4.  Scanner files can end in .DAT, .TXT, or set the Apattern switch in 
    CustomA.INI to any extension other than .FIL and .IAI.
5.  The Answer Check routine must be compatible with your scanner 
    output.  Email a sample of your scanner output to support@nine-patch.com 
    to verify compatibility.

6.  Do NOT change any internal file names or date/time lines created by True 
    Test Writer.  Your file names and the internal file names need not be the 
    same.  However all multiple-form files must have sequential names 
    (testa.fil, testb.fil, etc. or test1.fil, test2.fil, etc.)




H. Semi-Automatic Protocol

   1.  When a test is rescored the program automatically changes to semi-
       automatic operation.  It pauses for you to make corrections.

   2.  Set the first switch in CustomA.INI to ButtonSkip=0 to enhance this 
       feature.  Now you have access to all file windows and can monitor the 
       processing of scanner files into answer files ready for scoring.


                                                                        Page 10
I.  Manual Protocol (True Test Writer Files not Required)

    1.  Create and print the paper test by any means (if you plan to 
        obtain one combined item analysis from a multiple-form test, 
        the test must be created with True Test Writer).

    2.  Copy test scanner output onto a floppy disk (or enter student
        responses manually).

    3.  If a multiple-form test created by True Test Writer, copy the 
        item analysis index files (.IAI) to the floppy disk.

    4.  Press F5 to select manual operation.  Enhance by setting the first 
        switch in CustomA.INI to ButtonSkip=0.
        
    5.  Click TRUESC.EXE to score and analyze the student responses.  
        You now have access to the file windows for selecting files, 
        editing answer sheet errors, and for all scoring options.

    6.  Print selected reports.

    7.  Click TscC2A.BAT to install True Score on the floppy disk.

    8.  Deliver all test materials to the teacher.  If True Score has       
        been installed on the floppy, the teacher can rescore the
        test as needed for bad questions, typo's, and other errors 
        found in the first scoring.

Restrictions:

1.  All file names are limited to 8 characters, (.), and a 3 letter 
    extension (Windows 3.1, Windows 95, and Windows 98 compatible).
2.  Answer files (.ANS) and item analysis index files (.IAI) must have    
    matching names when processing multiple-form tests.
3.  Scanner or answer files and item analysis index files must be 
    copied to their respective SCANNER, ANSWERS, and IAINDEX folders if 
    not on a floppy.
4.  The Answer Check routine must be compatible with your scanner 
    output.  Email a sample of your scanner output to support@nine-patch.com 
    to verify compatibility.
5.  Manually create an answer file as follows for small tests or when a 
    scanner is not available:

    Column:      1       9 11 15   20   25   30   35
    Answer Key:  999000111 ABCDEABCDEABCDEABCDEABCDE
    Student ID:  Jones  AB ABCEABBCDEABCDEABCDEABCDE
                 435367890 ABCDEAABCDEAAABDDDEAFDDEE

    You can use the Answer Check routine which monitors your input to 
    insure the above format is used when entering student responses 
    manually.  The Chatsworth scanner uses this same format.





                                                                        Page 11
J.  Folders for Manual (Advanced) Operation

Automatic operation picks up files directly from a floppy in the A drive.  A 
sample set of scanner, test, and item analysis index files is included.  For 
manual operation, you must copy these files into the processing folders.

Directory of A:\

206A    TXT       2,962  Sample scanner file for 10 students taking version A.
206B    TXT       3,231  Sample scanner file for 11 students taking version B.

206A    FIL       1,403  Sample version A of the printed test. 
206B    FIL       1,403  Sample version B of the printed test.

206A    IAI       1,837  Sample item analysis index for version A. 
206B    IAI       1,837  Sample item analysis index for version B.

ANSWERS        <FOLDER>  Storage folder for processed answer files. 
IAINDEX        <FOLDER>  Storage folder for item analysis index files.

TRUESC         <FOLDER>  The home folder for True Score.

Directory of A:\TrueSc

README   TXT        815  A few startup instructions, also see HELP.TXT.
HELP     TXT    103,361  View with F1 Key, word processor or print.

TRUEAC   EXE     68,123  Answer Check prepares student marks for scoring.
TRUESC   EXE    171,830  True Score, the main score and analyze program.
CUSTOMA  INI      1,832  Manual switches for Answer Check.
CUSTOMS  INI      1,182  Manual switches for True Score.
TRUESCOR INI      2,189  F5 Key switches in True Score.

SCANNER        <FOLDER>  Processing folder for scanner output.
IAINDEX        <FOLDER>  Processing folder for item analysis index files.
ANSWERS        <FOLDER>  Processing folder for answer files.
PRINTOUT       <FOLDER>  Processing folder for True Test Writer files.

RESULTS        <FOLDER>  Processing folder for True Score results.
TSCTEMP        <FOLDER>  Temporary folder for old stuff left in above files.
KEEP           <FOLDER>  Permanent folder for all the new stuff.

DEMO1    ANS      1,308  Demonstration files to be used to practice manual 
DEMO2    ANS      1,000  operation.  Copy the files to the processing 
DEMO2    IAI        942  folders.  Set the CustomS.INI for manual operation:
DEMO3    ANS      1,111  ButtonSkip=0 and Adrive=0.  Observe the effects
DEMO3    IAI        942  of combining multiple-version test results and
DEMO4    ANS      1,476  of scoring for both knowledge and judgment.

PLOTFIL2 TXT     11,412  Graphs needed to plot quality and quantity when
PLOTFILE TXT      6,431  scoring for both knowledge and judgment.

VBRUN300 DLL    398,416  The Runtime file for WIN3.x, WIN95, and WIN98.
TscC2A   BAT        340  Copies True Score from drive C to drive A.


                                                                       Page A-1
                      REFERENCE MANUAL


K.  Answer File Preparation and Check 



                  True Answer Check Help
         (Specific help is green on each screen.)


     Answer File Preparation and Check Window . . . A-2
       (1) Load
       (2) Process

           Edit
           Manual
       (3) End 

     Process Verify Window  . . . . . . . . . . . . A-3

     Manual Data Entry Window . . . . . . . . . . . A-4
       Edit
       Process
       Verify

     File Structure . . . . . . . . . . . . . . . . A-5
       Scanner (*.DAT, *.TXT)
       Answer  (*.ANS)

     Scanner Output Compatibility . . . . . . . . . A-6
       National Computer Systems
       Chatsworth 
       Scantron
       Scanning Systems
       Apperson


----------------------------------------------------------



















                                                                       Page A-2

                  Answer File Preparation and Check Window

There are three operations.  Normally only the first operation is used.  The 
other two provide for complete quality management of the answers.

A. Operations:

   1. Automatically prepare scanner files for scoring.
   2. Edit answer files manually.
   3. Create and verify answer files manually.

B. Control Buttons:

   1. Load

      This button loads files into the left file box.  The source address or 
      folder is listed below the file box.  Select a file to see the file size, 
      date and time.  Double click a file to load it into the word processor 
      you switch on in the CustomA.INI file.

      Two sample scanner files, TESTA.TXT and TESTB.TXT can be loaded for      
      demonstration purposes.  Select one or both for the following operation.

   2. Process

      Select a scanner file and then click PROCESS to create and verify an 
      answer file for scoring by True Score.  The scanner file name is retained 
      with the extension changed to ANS.  Then run True Score to score and 
      analyze the test.

   3. Edit

      An answer file may need editing for a number of reasons: add more test 
      results, right answer in error, two right answers, the students just 
      could not comprehend a question, need to adjust item weight, etc.  Edit 
      can manually verify any answer file.

   4. Manual

      This is a manual backup for creating and verifying an answer file.  You 
      can score and analyze a test even if the scanner becomes unavailable.

   5. End (Next)

      This first part of True Score does not continue into the scoring routine 
      when operated manually.  You must click True Score, under manual      
      operation, to score the prepared answer file.  This gives you total 
      control of all operations.  After the first run, you may have no more 
      need of the file preparation and check routines.  Rescoring can all be 
      done with True Score, under manual operation, alone.






                                                                       Page A-3
                            Process Verify Window            

A.  Automatic reformatting:

    The scanner answer file is reformatted for easy viewing.  A test of 200  
    questions by 200 students can be input.  A scale is inserted after every 
    five answer lines.  The yellow text window indicates restricted input.  You 
    cannot change the text.  

B.  Correct scanner errors:

    Click Check to verify the scanner answers.  The answers are scanned for 
    blanks and input errors.  If found, they are displayed with a request for 
    the correct mark or space. 

C.  Multiple form results in one file:

    The response file from the scanner may contain results from more than one 
    form of the test.  If so, the answer sheets of each form are saved in 
    separate files.  Scanner file Nurse.NCS becomes Nurse1.ANS, Nurse2.ANS, 
    etc. that match the Nurse1.IAI, Nurse1.IAI, etc. files used by True Score 
    to produce one item analysis for the test.

D.  Final Inspection of Prepared Answer File:

    The finished text is shown for inspection.  An answer key, ID coded 
    999000111 is required.  A weight line, ID coded 999000222 is optional.  
    They can be scanned from sheets or entered by hand.  Click Save.  TestA.TXT 
    becomes a new answer file, TestA.ANS, in the right file box.  Run True 
    Score to score and analyze the test.

E.  Attach notes:

    F4=Notes.  Press the F4 key to attach notes to an answer file.






















                                                                       Page A-4
                            Manual Data Entry Window

A. Operations:

Use this to enter test results from small tests or when a scanner is not 
available.  It often takes less time to enter test results than to prepare and 
send the answer forms to and receive the results from a distant scoring service. 

B. Control Buttons:

   1. Edit

      Any character can be changed with two limits: The tenth character is a 
      space.  The answer characters always appear in upper case.

      Enter new answer lines at the END of the file.  The ID numbers can be in   
      any order.

      Edit can also verify any file manually.  This is often done when an 
      answer file is created manually.

      F4=Notes.  Press F4 to read or attach notes to an answer file.

   2. Manual

      Any number or letter (upper or lower case) can be used in the nine-
      character ID field.  The tenth character is a space.  The answer 
      characters are printed in upper case.  The scale lines appear and 
      disappear on their own.

      Enter the test key 999000111 ABCD etc.  Enter a weight line 999000222 
      AAAAAAA etc. (A = 1, B = 2, C = 3, D = 4, E = 5, and blank = 0).  Enter 
      the student IDs and answers.

      The Save button, stores and returns.  The Exit button, saves and returns 
      to the main menu.  Use Edit to continue after exiting Manual.

   3. Verify

      Click Verify to manually verify a file.  The old text appears in the 
      lower green window.  The entry window turns yellow to indicate restricted 
      input.  Enter the key and student answers just as they were entered 
      before from their answer sheets or tests.  If an error occurs, it is 
      displayed with a request to keep the Old or the New.  If you select Old, 
      you still must enter it correctly. 

      The Save button, stores and returns to Verify.  The Exit button, stores 

      and returns to the main menu.  Verify only starts at the beginning of a 
      file to insure that the process is valid, so exit only when finished.







                                                                       Page A-5
                                File Structure

A test of 200 questions by 200 students can be input from a scanner file or 
manually.  Each manual file must have records from just one test form.

A. Answer Files (*.ANS)

All files ready for scoring must carry the (.ANS) extension.  These answer 
files are ASCII text files.  The answer key and weights can be scanned from 
sheets or entered by hand.  The key ID code is 999000111.  The weight ID code 
is 999000222 and is optional.  The following example is the first five students 
and the first 60 questions in the sample answer file 206AA.ANS.  This is also 
the output of a Chatsworth card reader.

     999000111 DBBCCDCDCCCCCADDCCCBCCACCABBCAABBCCDAAAABCAACDCCBBDBCBBBBBCB
     999000222 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
     ABBEY  EA DBBCCBCDCCCACADDCCCBCCADCABBCAABDCCDAAACBCAACDCCCADBCBBCBBCB
     BABBITTI  DBBCCDCDCCCCCAADCCCBCCACCABBCBABBCCDAAAABCAACDCCBCDBCBBACBCB
     CABALIEFL DBBCCACDCCCCCADDCCCBCCADCABBCBABBCCDAAAABCACCDBCBBDBCBBBBBCB
     DAGUERRLJ DBBCCBCDCCCCCADDCCCBCCACCABBCAADBCDDAAAADCAACDCCCDCBDCABBBCA

Test Form: A.  (This note is separated from data by one blank line.  Press F4 to 
add notes to an answer file.)

B. Scanner files (*.DAT, *.TXT)

Scanner files are also ASCII files.  They must be converted to an ANS file like 
that above before scoring.  There are five scanner companies.  Each scanner can 
create several different output files.  This example uses the National Computer 
Services scanner output.  Each record consists of a number of fields including 
sequence number and answer responses.  Answer responses include blanks (0) and 
read errors (X).  This example is the first five students and the first 10 
questions in the sample scanner file 206AA.FIL.  
----------------------------------------------------------------------------
    Seq #                                 ID#        Name           Answers
   ......                               .........--------------    .........
1...5...10....5...20....5...30....5...40....5...50....5...60....5...70....5.
510000001001092199001   6325 #0001      260      KEY         NS10214223343433
510000002001092199001   6325 #0001      1053     ABBEY       EA    4223323433
510000003001092199001   6325 #0001      1055     BABBITT     I     4223343433
510000004001092199001   6325 #0001      1058     CABALIERO   FL    4223313433
510000005001092199001   6325 #0001      1061     DAGUERRE    LJ    4223323433
510000006001092199001   6325 #0001      1063     EADS        JB    4223343433
-----------------------------------------------------------------------------
Scores can be sorted by scanner sequence number, student ID#, or by student 
name.  Set the switch in the CustumA.INI file for your choice.

The scanner output for all forms of one test can be placed in one scanner 
output file.  Scan a key at the start of each form in the file.








                                                                       Page A-6
                        Scanner Output Compatibility

The following formats can be used by True Score:

A. National Computer Systems

The current automatic operation is set for National Computer Systems output.
----------------------------------------------------------------------------
    Seq #                                 ID#        Name           Answers
   ......                               .........--------------    .........
1...5...10....5...20....5...30....5...40....5...50....5...60....5...70....5.
510000001001092199001   6325 #0001      260      KEY         NS10214223343433
510000002001092199001   6325 #0001      1053     ABBEY       EA    4223323433
510000003001092199001   6325 #0001      1055     BABBITT     I     4223343433
510000004001092199001   6325 #0001      1058     CABALIERO   FL    4223313433
-----------------------------------------------------------------------------

B. Chatsworth

Chatsworth output requires no converstion.  It can be loaded directly into the 
ANSWERS folder.  Key card = 999000111.  Weight card = 999000222.
        ------------------------------------------------------------
        StudentID 1...5...10....5...20....5...30....5...40....5...50
        999000111 DBBCCDCDCCCCCADDCCCBCCACCABBCAABBCCDAAAABCAACDCDBC
        999000222 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
        ABBEY  EA DBBCCBCDCCCACADDCCCBCCADCABBCAABDCCDAAACBCAACDCCCA
        BABBITTI  DBBCCDCDCCCCCAADCCCBCCACCABBCBABBCCDAAAABCAACDCCBC
        CABALIEFL DBBCCACDCCCCCADDCCCBCCADCABBCBABBCCDAAAABCACCDBCBB
------------------------------------------------------------

C. Scantron

These scanner files are comma-delimited ASCII files.  Each record consists of 
four fields: student ID, test form, unused, and answer responses.  Answer 
responses include blanks ( ) and read errors (CE) with a suggested mark (C).

100932091 ,A, 45,C,C,D,A,A,B,C,B,D,E
270525052 ,A, 45,C,C,D,A,A, ,CE,B,D,D
300841083 ,A, 50,C,C,D,A,A,B,B,B,D,C


D. Scanning Systems

E. Apperson

These scanner files are comma delimited ASCII files with the first line the 
key. The following lines are student answers.  Each part consists of a 
character, "A"; a null, "" or an error message, "input error".  The input error 
is reduced to one character, "X" when loaded by True Answer Check.

"A","B","C","D","E","A","B","C","D","E"
"A","B","C","D","E","","INPUT ERROR","C","D","E"
"A","B","C","D","E","A","B","C","D","E"
"A","B","C","D","E","A","B","C","D","E"


                                                                       Page S-1

                          REFERENCE MANUAL


L.  Score and Analyze 


                    True Score Help Parts

        (View with Courier or Other Fixed-Pitch Font)


     Part 1a: The Three List Boxes  . . . . . . . . . . . . .  S-2
     Part 1b: The Control Buttons . . . . . . . . . . . . . .  S-3
     Part 2: Copying and Moving Files . . . . . . . . . . . .  S-4
     Part 3: One Test Item Analysis (DEMO1.ANS) . . . . . . .  S-5
     Part 4: Sample Results . . . . . . . . . . . . . . . . .  S-6
     Part 5: Multiple Form Test Item Analysis . . . . . . . . S-11
     Part 6: Scoring for Knowledge and Judgment . . . . . . . S-16
     Part 7: F5 Customize Reports and Export Files (1998) . . S-26
     Part 8: Advanced Question and Student Analysis (1999)  . S-27
     Part 9: License and Warranty . . . . . . . . . . . . . . S-28
     Part 10: Order and License Registration Form . . . . . . S-29

------------------------------------------------------------------































                                                                       Page S-2

Part 1a: The Three List Boxes              
=============================

Box 1: Answer File (ANS)
------------------------
Copy DEMO1.ANS into the ANWERS folder.  Run True Score.
Select DEMO1.ANS and then click Process.  The answer file 
will be scored and an item analysis prepared.  The results 
will appear in the RESULTS window for you to view (double 
click) or to print out the parts you want.  (Copies of the 
results are also listed in Part4: Sample Results, page S-6.)  

Answer files loaded into the ANSWERS folder appear in the 
Answer File window. (See Part2: Copying and Moving Files,
page S-4, to set your computer for easy file management.)

Box 2: Item Analysis Index (IAI)
--------------------------------
Combined analyses can be prepared to match either the
unformatted source file from which several forms of a test 
were drawn or to match one of the test forms.  True 
Test Writer creates the item analysis index files as it
creates multiple forms of a test.  The index files 
copied into the IAINDEX folder appear here.  

Box 3: Results List Box File Name Extensions
--------------------------------------------
OIA  One Item Analysis, one test was scored and analyzed.
TRA  Test Referenced Analysis, a test was selected as the                               
     reference file for a multiple form set of tests.
URA  Unformatted Referenced Analysis, the unformatted source      
     file was the reference file for a set of tests.
KNS  KNowledge Score (traditional) export file.
KJS  Knowledge and Judgment Score export file.

The analysis of the DEMO1.ANS answer file appears as 
DEMO1.OIA in RESULTS.

DEMO2.ANS and DEMO3.ANS are each half of DEMO1.ANS with the 
question and answer sequences changed in DEMO3.ANS.  Select 
DEMO2.ANS and DEMO3.ANS answer files and DEMO2.IAI (item 
analysis index) as the test reference file.  Click Process 
to combine the two forms of the test into a single item 
analysis (DEMO2.TRA) in the RESULTS window.

DEMO2.ANS and DEMO3.ANS can also be analyzed individually.  
Their results would then be DEMO2.OIA and DEMO3.OIA.  They 
would not be as good as the combined analysis.

Move your own answer file into the ANSWERS folder and 
click Analyze.  




                                                                       Page S-3
Part 1b: The Control Buttons
============================

The manual operating sequence is (1) select answer files, (2) select a 
reference file, and (3) process.  In automatic operation, this is done for you 
on the first scoring of a scanner file.  On the second scoring, the program 
pauses for you to edit the answer files or reprocess the scanner file.  

Use the LOAD buttons below to copy (drag) answer and item analysis index files 
you want into the ANSWERS and IAINDEX processing folders.  You are responsible 
for matching files having identical names ending in ANS and IAI respectively.  
Then run True Score.  If not satisfied, make changes, and run again.

Load Ans
--------
This button loads answer files into the ANSWERS processing folder in two ways.  
It requests the full path and folder for answer files ending in ANS.  Actual 
file names are not needed.  It also calls the edit and manual entry routines.  
You can select to edit the old answer files, or reprocess the scanner files.
You can make corrections and entering analysis keys (page S-28).

When not using a scanner, press F5 and select Operation: Manual.  Then select 
Question Source: Any Disk if you are working without a disk in floppy drive A.
Select Drive A to save the manually entered marks saved on drive A.

Load Ref
--------
This button requests the location of reference files (Item Analysis Index 
files) that match the file names in the Answer Files box.  If Any Drive is set 
as a source (F5), the button suggests the KEEP folder created by True Test 
Writer. This is handy when you create and score your own tests.  If the A Drive 
is set as a source (F5), it only looks on the A drive for item analysis index 
files.

(3) Process
-----------
If Any Drive is set as a source (F5), you have complete control to select 
answer files and reference files.  If the A Drive is set for floppy operation, 
clicking Process automatically selects the first item analysis index file in 
the Reference File Window that matches the first form of a multiple-form test 
and scores all forms of the test showing in the Answers File Window.

Keep
----
Keep saves the results files and all files used in their production.  Each 
rescoring can be saved to a different Keep folder.  If the same folder is used, 
new files overwrite old files of the same name.

Delete
------
This clears the Results List Box.  

Exit
----
You will be reminded to save all source and results files to a Keep folder.

                                                                       Page S-4
Part 2: Copying and Moving Files
================================

Windows 98 and Windows 95
-------------------------
Open Windows Explorer twice.  Right click a blank area on
the taskbar.  Click Cascade or Tile to arrange the two 
windows.  You can now easily copy a file from one to the 
other.  Right click a blank area on the task bar and then
click Undo Cascade or Undo Tile to restore the windows.

Windows 3.1
-----------
A. If the File Manager icon is showing below this   
   window, give it a click to open, or go to B.

   1. If it contains only one directory window, click the 
      Maximize Button to get some space, then click the 
      Window menu and New Window.

   2. Click TILE so both directory windows are in view. 

   3. Click the Minimize Button to reduce the File 
      Manager again to an icon and save it on the Task List. 

   Now files can be copied or moved before, during and after    
   running TRUE SCORE.

B. If the File Manager icon is not showing below this 
   window, it must be found.  Press Ctrl + Esc key to 
   present the Task List.  If the File Manager is here, 
   double-click it and proceed as at A or go to C.

C. If the File Manager is not on the Task List, double click 
   the Program Manager.  If the File Manager is here, click 
   it and proceed as at A.  If it is not, look for the MAIN 
   application group or icon.  Click to open it.  The File 
   Manager is normally here.

Windows 98, Windows 95 and Windows 3.x
--------------------------------------
Click the Help menu on the Explorer or the File Manager 
title bar for instructions to copy or move files.  The main 
idea is to select (click) a file and then drag it (hold the 
mouse button down) to the new folder.  If done with the
Control key down, it copies.  If done with the control 
key up, it moves the file.

TRUE SCORE picks up files from the ANSWERS and the IAINDEX 
folders and puts the results into the RESULTS folder.  Use 
the File Manager to load, unload, view, and edit files in 
these folders.  Click File and then Associate to relate word 
processors to a selected file with a non-standard file name 
extension (also see page 5 above).


                                                                       Page S-5

Part 3: One Test Item Analysis (DEMO1.ANS)
==========================================

This only requires an answer file.  Copy DEMO1.ANS into
the ANSWERS folder.  Double click DEMO1.ANS (in List 
Box 1) to view it.  Each line in the DEMO1.ANS holds a 
Student ID and the answer marks.  Comments can be added 
below a blank line.

479021001 CBDBCBCCBAADCDDACCAA
480900002 ADCBDADBBAADCDDACCAB
482923003 CBACBBDEBABCCDACBCBB
483065004 CDBABBCEBAADCDBACCDA
483849005 DDBCCCDBBAADCDCACCAA
485080006 CDCCBCDEBABDCDDACBDC
485112007 CDCCCDDDAAADCDDCCCAA
485987008 CDBABBDBAAADCDCACBAA
486765009 CDCABADBBAADCDDACCAA
486926010 CCDABACBBAAACBADAADD
486943011 CDBABADBBAADCDDACCAA
487963012 CDDABADBBAADCDDACCAA
489846013 CDCABBDCBAADCBADCCAA
489909014 CACABDDBBAAACDDACDCA
489948015 AAAACDDBBAABCDDACCAA
491907016 CDBABDDBBAADCDDACCAA
494862017 CACABBCDBAADCDBCCACA
497687018 CDCACBCDAADDCADACACD
498741019 CDBDCBDBBAADCCDACCAA
498749020 CDCACCDEBAADCDDACCAA
499843021 CCBCCDDCBAADCDDACCAA
508236022 CDCACBDBBAADCDDACCAA
515943023 BACABDDDBADDCDDDCCAA
999000111 CDCABCDBBAADCDDACCAA
999000222 AAAAAAAAAAAAAAAAAAAA

(Leave a space below data lines before commenting.)

A 20 question test taken by 23 students plus an answer
key with all questions with a weight of one (A).
------------------------------------------------------------
1-9       Student ID (Letters or Numbers).
11-111    100 Answers (ABCDE).
999000111 Answer Key ID.
999000222 Weight ID.
          Weights: (Blank = 0, A=1, B=2, C=3, D=4, F=5).
------------------------------------------------------------
An answer file must include a key line and one line for each 
student.  A weight line is optional.  If the weight line is 
omitted, all questions are scored with the same weight.  

Copy your own answer file to the ANSWERS folder of TRUE SCORE.
Run TRUE SCORE.  View or print the results.



                                                                       Page S-6

Part 4: Sample Results
======================


Each part of the results contains its own description.  A 
few comments of how these have been used are presented here.

Scoring Summary (Page S-7)
--------------------------
The Score Summary has been used to audit the answer sheet 
scanner, to compare two students suspected of cheating, and 
as a report to students.  When used as a report, more space 
was added between individual results and the score summary 
was cut into parts with a table knife.  Or the score summary 
was fastened to the wall by the left edge and a razor blade 
used to slit the individual results apart.

Ranked Raw Scores (Page S-8)
----------------------------
The left side of the student ID column was cut off before 
posting.  The percent score was posted to a grade book after 
adjusting for bad questions.  This was done by hand from the 
report and by computer from the export score file.  

Item Analysis (Page S-9)
------------------------
How the parts of each question performed and how well the 
question performed in relation to the rest of the test is 
presented here.  These details are useful in editing 
questions.  Of greater concern is setting the current 
grades.  If the average Chi Square is 3.84 or higher, the 
raw scores can be assigned grades with confidence.

The individual Chi Square values are conservative.  The 
individual Point Biserial R values are liberal.  The truth 
is somewhere in between when using small numbers of scores.

Bad Questions are marked (**).  One score-to-grade 
conversion dropped bad questions and distributed their 
weight over the rest of the test.  It allowed those who 
answered correctly to retain their raw scores plus the 
adjustment.  This produced a bonus for answering a question 
that the rest of the class could not answer.  At the other 
extreme, is a conversion that gives the question a zero 
weight and rescores the test.  The idea is that a bad 
question is not valid for scoring.

Short Form Item Analysis (Page S-10)
------------------------------------
This summary was developed for practical reasons.  This slip 
of paper and a copy of the test are all that is needed to
guide class discussion.  Any question that is marked on the 
difficulty line needs to be discussed with the class.  The 
reasons are given below the analysis.  The mark option 
distribution of the full analysis can provide more details.

                                                                       Page S-7

SCORING SUMMARY
---------------
              DEMO1.ANS  13:41:23  02-18-1998
  (View and Print with Courier or Other Fixed-Pitch Font)
------------------------------------------------------------
 STUDENT  WG  BK  RT  SCORE AN:CBDBCBCCBA ADCDDACCAA 
479021001  7   0  13  013.0 EV:2000000022 2222222222 

 STUDENT  WG  BK  RT  SCORE AN:ADCBDADBBA ADCDDACCAB 
480900002  5   0  15  015.0 EV:0220002222 2222222220 

 STUDENT  WG  BK  RT  SCORE AN:CBACBBDEBA BCCDACBCBB 
482923003 12   0   8  008.0 EV:2000202022 0022000200 

 STUDENT  WG  BK  RT  SCORE AN:CDBABBCEBA ADCDBACCDA 
483065004  6   0  14  014.0 EV:2202200022 2222022202 

 STUDENT  WG  BK  RT  SCORE AN:DDBCCCDBBA ADCDCACCAA 
483849005  5   0  15  015.0 EV:0200022222 2222022222 

 STUDENT  WG  BK  RT  SCORE AN:CDCCBCDEBA BDCDDACBDC 
485080006  6   0  14  014.0 EV:2220222022 0222222000 

 STUDENT  WG  BK  RT  SCORE AN:CDCCCDDDAA ADCDDCCCAA 
485112007  6   0  14  014.0 EV:2220002002 2222202222 

 STUDENT  WG  BK  RT  SCORE AN:CDBABBDBAA ADCDCACBAA 
485987008  5   0  15  015.0 EV:2202202202 2222022022 

 STUDENT  WG  BK  RT  SCORE AN:CDCABADBBA ADCDDACCAA 
486765009  1   0  19  019.0 EV:2222202222 2222222222 

 STUDENT  WG  BK  RT  SCORE AN:CCDABACBBA AACBADAADD 
486926010 12   0   8  008.0 EV:2002200222 2020000000 

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

 STUDENT  WG  BK  RT  SCORE AN:CCBCCDDCBA ADCDDACCAA 
499843021  6   0  14  014.0 EV:2000002022 2222222222 

 STUDENT  WG  BK  RT  SCORE AN:CDCACBDBBA ADCDDACCAA 
508236022  2   0  18  018.0 EV:2222002222 2222222222 

 STUDENT  WG  BK  RT  SCORE AN:BACABDDDBA DDCDDDCCAA 
515943023  6   0  14  014.0 EV:0022202022 0222202222 

------------------------------------------------------------
STUDENT:  Examinee's ID number or name (9 characters).
WG:       The number of wrong answers.
BK:       Blank = not marked (counted as wrong).
RT:       The number of right answers.
SCORE:    Raw score (right x weight).
AN:       The answers marked by the examinee.
EV:       0 = wrong, 1 = not marked, 2 = right.

                                                                       Page S-8

RANKED RAW SCORES
-----------------
         DEMO1.ANS  13:41:23  02-18-1998
--------------------------------------------------
RANK  STUDENT    WG   RT  % SCORE  RAW SCORE
--------------------------------------------------
  1  486765009    1   19   095.0     019.0
  2  508236022    2   18   090.0     018.0
  3  486943011    2   18   090.0     018.0
  4  498749020    2   18   090.0     018.0
  5  491907016    2   18   090.0     018.0
  6  487963012    2   18   090.0     018.0
  7  483849005    5   15   075.0     015.0
  8  498741019    5   15   075.0     015.0
  9  489846013    5   15   075.0     015.0
 10  489909014    5   15   075.0     015.0

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

 18  515943023    6   14   070.0     014.0
 19  479021001    7   13   065.0     013.0
 20  494862017    8   12   060.0     012.0
 21  497687018   10   10   050.0     010.0
 22  482923003   12    8   040.0     008.0
 23  486926010   12    8   040.0     008.0
--------------------------------------------------
AVERAGE:                   072.6     014.5
--------------------------------------------------
WG:         The number of wrong answers including blanks.
RT:         The number of right answers.
RAW SCORE:  A quantitative estimate (right x weight).
% SCORE:    The raw score expressed as a percentile.


                         HISTOGRAM
              DEMO1.ANS  13:41:23  02-18-1998
------------------------------------------------------------
PCT   SCORE  FREQ
------------------------------------------------------------
 95     19     1 :X
 90     18     5 :XXXXX
 85     17     0 :
 80     16     0 :
 75     15     6 :XXXXXX
 70     14     6 :XXXXXX
 65     13     1 :X
 60     12     1 :X
 55     11     0 :
 50     10     1 :X
 45      9     0 :
 40      8     2 :XX
------------------------------------------------------------
 73     15    23
------------------------------------------------------------

                                                                       Page S-9

ITEM ANALYSIS
-------------
                       DEMO1.ANS  13:41:23  02-18-1998
    MARK DISTRIBUTION, ONE ESTIMATE OF DIFFICULTY AND TWO OF DISCRIMINATION
           (View and Print with Courier or Other Fixed-Pitch Font)
------------------------------------------------------------------------------
TEST           MARK OPTION DISTRIBUTION            DIFFICULTY (%)     DISCRIM-
ITEM   ---------------------------------------    -----------------   INATION
       BLANK       KNOW, BELIEVE, OR GUESS                            --------
               A=1    B=2    C=3    D=4    E=5    -----------------   CHI PT
       ---------------------------------------    TEST  ITEM  ITEM    SQR BIR
        H/L    H/L    H/L    H/L    H/L    H/L    MEAN   WT   SCORE     P<=
------------------------------------------------------------------------------
  2.      0      4      2      2     15*     0                        3.1  .57
        0/0    1/3    0/2    0/2   10/5    0/0      73     5    65     Better

  6.      0      5      9      3*     6      0                        0.0  .15
        0/0    3/2    4/5    2/1    2/4    0/0      73     5    13**  

  8.      0      0     12*     3      4      4                        4.4  .41
        0/0    0/0    9/3    1/2    0/4    1/3      73     5    52     Better

 11.      0     19*     2      0      2      0                        1.2  .47
        0/0   11/8    0/2    0/0    0/2    0/0      73     5    83      Good

 16.      0     17*     0      3      3      0                        0.9  .54
        0/0   10/7    0/0    0/3    1/2    0/0      73     5    74      Good

 19.      0     16*     1      3      3      0                        1.8  .67
        0/0   10/6    0/1    1/2    0/3    0/0      73     5    70      Best

 20.      0     18*     2      1      2      0                        2.3  .63
        0/0   11/7    0/2    0/1    0/2    0/0      73     5    78      Best
------------------------------------------------------------------------------
AVERAGE CHI SQUARE FOR THE TEST = 0.84  (3.84 = 5% LEVEL OF SIGNIFICANCE)
   (If below 3.84, combine scores with other tests to set course grades.)
Kuder-Richardson 20 = 0.66
  (Estimate of test homogeneity and reliability.)
Standard Error of Measurement (SEM) = 1.72
  (Estimate of individual raw score reliability.)
The true score is within +/-1 SEM 68% of the time; 2 SEM 95% of the time.

*  The right answer in the Mark Option Distribution.
** 1 BAD QUESTION with a WEIGHT of 5.0

H/L      Number scoring in the higher/lower half of the ranked scores.
MEAN     Average test score as a percent.
WT       Weight of item as percent of test.
SCORE    Difficulty (ease) based on entire class.
CHI SQR  Chi-Square estimate of discrimination ability.
PT BIR   Point Biseral R estimate of discrimination ability.
P<=      Average of Chi-Square and Point Biseral R expressed as Good,
         Better, Best (Level of Significance: 10%, 5%, and 1%).


                                                                      Page S-10

SHORT FORM ITEM ANALYSIS
------------------------

                    DEMO1.ANS  13:41:23  02-18-1998
------------------------------------------------------------------------
                  1...5...10 ....5...20 ....5...30 ....5...40 ....5...50
ANSWER KEY:       CDCABCDBBA ADCDDACCAA 
ITEM WEIGHTS:     1111111111 1111111111 
DIFFICULTY:            B                
DISCRIMINATION:    5    55   T    T  11 
------------------------------------------------------------------------
Difficulty:     Upper Case Letter (A-E) = The most popular choice and
 (THE BAD)                                it was wrong.
                  (A majority believed they knew but did not know.)
                Upper Case Letter (H) = A question performing below
                                        the difficulty level of 37.5%
                  (Discuss the above questions with the class.)
 (THE GOOD)     Blank = Difficulty (Item Score) higher than 37.5%

Discrimination: T = Good, 5 = Better, 1 = Best (Level of Significance:
 (THE BEST)     10%, 5%, and 1%).
                  (These are your best questions for new tests.)

































                                                                      Page S-11

Part 5: Multiple Test Form Item Analysis
========================================

General Protocol
----------------
This takes a bit of planning.  A source question file, a 
means of making multiple forms of a test, and a means of 
combining the answers into one analysis are required. 

True Test Writer can create the unformatted source question
file, format the tests and create their item analysis 
indexes.  True Score combines the answers into one analysis.
A general protocol is:

A. Create an unformatted source question file.

B. Format multiple test forms and item analysis indexes    
   (page S-12).

C. Administer the tests.

D. Scan answers into an answer file for each test form. (See
   Part 3: One Test Item Analysis for details, page S-5.)

E. Copy the answer files to the ANSWERS folder and item 
   analysis index files to the IAINDEX folder.

F. Run True Score and obtain the analysis.

Demonstration
-------------
Copy the DEMO2.ANS and DEMO3.ANS answer files (page S-13)
into the ANSWERS folder.  Copy the DEMO2.IAI to the IAINDEX
folder.  Select all three and click Process to combine the 
answer files.  The single text referenced analysis (DEMO2.TRA)
will appear in the RESULTS window.  The analysis will look 
exactly the same as from DEMO1.ANS (pages S-7 to S-10).

This is because DEMO2.ANS and DEMO3.ANS are each one half of 
DEMO1.ANS.  DEMO2.IAI, page S-14, indicates that the DEMO2.ANS 
file holds questions in the same order as the unformatted 
source file.  The answers were randomized.  DEMO3.IAI, page 
S-15, indicates both the questions and the answers were 
randomized when test form DEMO3 was created.

Either form of the test, DEMO2.IAI or DEMO3.IAI, can be 
selected as the test reference file.  For long term studies, 
the unformatted source file can be selected, UNFORMATTED, as 
the reference.  In such an analysis the right answer is 
always (A).  This makes comparing separate reports easier.





                                                                      Page S-12

TEST PROCESSING RECORD
----------------------
If you plan to combine multiple forms used at one time or 
over a period of time, a check list is needed.  The 
following is one suggestion for naming and organizing files.
Save this record with the other files in one folder.  In 
this way only the folder needs a unique name.  (True Test 
Writer now creates this folder in a KEEP folder.)

Question bank file names must consist of a NAME of up to 
eight characters, a period (.), and an EXTENSION of up to 3 
characters.  Answer files must have the extension .ANS.  The 
remaining file names are generated either by True Test 
Writer or True Score.

In the following example, 50 questions were selected from 
three test banks into the source file.  Three forms of the 
test were generated from the source file.  The three answer 
files were scored and analyzed as one test using the first 
form as the reference file.

------------------------------------------------------------

Course and Test:   Human Physiology, Test 1, W98

Test Bank Names:   Skin.001, Muscle.001, Bone.001

Source File:       POOL1.FIL, 50 questions, or rename to 
                              HumPhy01.w98  

Make Tests:        TEST1.FIL, TEST2.FIL, TEST3.FIL
and Indexes:       TEST1.IAI, TEST2,IAI, TEST3.IAI

Answer Files:      TEST1.ANS, TEST2.ANS, TEST3.ANS

Analyze:           TEST1.TRA

Save all files to: HUPH1W98 (folder name)
------------------------------------------------------------

The first 10 file names are copied to a KEEP folder named
HUPH1W98 by True Test Writer.  Additional forms of the test 
can be made from POOL1.FIL.  These additions can be added to 
the HUPH1W98 folder too.  The new answer files can then be 
combined with the present ones for scoring and one combined
analysis. 

The current analysis, TEST1.TRA, can be saved by True Score.
It can also be saved to HUPH1W98.  These 11 files make a 
full record that can be viewed with any word processor.





                                                                      Page S-13

DEMO2.ANS and DEMO3.ANS
-----------------------
479021001 CBDBCBCCBAADCDDACCAA
480900002 ADCBDADBBAADCDDACCAB
482923003 CBACBBDEBABCCDACBCBB
483065004 CDBABBCEBAADCDBACCDA
483849005 DDBCCCDBBAADCDCACCAA
485080006 CDCCBCDEBABDCDDACBDC
485112007 CDCCCDDDAAADCDDCCCAA
485987008 CDBABBDBAAADCDCACBAA
486765009 CDCABADBBAADCDDACCAA
486926010 CCDABACBBAAACBADAADD
486943011 CDBABADBBAADCDDACCAA
487963012 CDDABADBBAADCDDACCAA
999000111 CDCABCDBBAADCDDACCAA
999000222 AAAAAAAAAAAAAAAAAAAA

(Leave a space below data lines before commenting.)

The above file, DEMO2.ANS, contains the first 12 students in 
the DEMO1.ANS file.  Select both DEMO2.ANS and DEMO3.ANS to 
obtain the same results as from DEMO1.ANS.


489846013 CCBDDCCCBDBCCDDDBBCA
489909014 CABADCCCBDADADBCBBDB
489948015 CCBDDCCCBAAAABBCBDDB
491907016 CCBDDCCCBDACACBCBBCB
494862017 CAABDCCCBDCCADDABBDD
497687018 CAABDCCBAACCBDDCABCB
498741019 CCBDDCBCBAACDCDCBBCB
498749020 CCBDDCCCBADCADACBBCB
499843021 CCBDDCACBABCACBCBBBB
508236022 CCBDDCCCBAACADDCBBCB
515943023 CCBDDCCCBDCCADBCAADA
999000111 CCBDDCCCBDACADACBBCB
999000222 AAAAAAAAAAAAAAAAAAAA

(Leave a space below data lines before commenting.)

The above file, DEMO3.ANS, contains the last 11 students in 
the DEMO1.ANS file with the question order and the answer 
options randomized again.  DEMO2.ANS is the first form.  
DEMO3.ANS is the second form of the test.  Select both 
DEMO2.ANS and DEMO3.ANS to obtain the same results as from 
DEMO1.ANS.

--------------------------------------------------------
1-9       Student ID (Letters or Numbers).
11-111    100 Answers (ABCDE).
999000111 Answer Key ID.
999000222 Weight ID.
          Weights: (Blank = 0, A=1, B=2, C=3, D=4, F=5).
--------------------------------------------------------

                                                                      Page S-14
DEMO2.IAI
---------
            Question and Answer Distributions
---------------------------------------------------------
1.  Test Question Number.
2.  Source File Question Number.
3.  Answer Distribution: 0 = The only Right Answer.
                         1 = First of Two Right Answers.
                         2 = Second of Two Right Answers.
                       5-8 = Wrong Answers.
---------------------------------------------------------
1   1    7605
2   2    8560
3   3    5706
4   4    0675
5   5    5076
6   6    6705
7   7    7850
8   8    70586
9   9    5086
10  10   0578
11  11   0567
12  12   6750
13  13   5706
14  14   6870 
15  15   8560
16  16   0756
17  17   7506
18  18   6705
19  19   0586
20  20   0586

ANSWER KEY:     DEMO2.FIL  01-01-1998  11:41:37

1   C
2   D
3   C
4   A
5   B
6   C
7   D
8   B
9   B
10  A
11  A
12  D
13  C
14  D
15  D
16  A
17  C
18  C
19  A
20  A


                                                                      Page S-15

DEMO3.IAI
---------
            Question and Answer Distributions
---------------------------------------------------------
1.  Test Question Number.
2.  Source File Question Number.
3.  Answer Distribution: 0 = The only Right Answer.
                         1 = First of Two Right Answers.
                         2 = Second of Two Right Answers.
                       5-8 = Wrong Answers.
---------------------------------------------------------
1   13   5706
2   19   8605
3   7    5078
4   18   5670
5   10   5780
6   17   7506
7   4    7506
8   9    6508
9   20   6058
10  5    7650
11  8    05867
12  12   7506
13  14   0687
14  3    6570
15  6    0567
16  15   6508
17  11   7056
18  1    6057
19  2    5608
20  16   6075

ANSWER KEY:     DEMO3.FIL  01-01-1998  11:42:47

1   C
2   C
3   B
4   D
5   D
6   C
7   C
8   C
9   B
10  D
11  A
12  C
13  A
14  D
15  A
16  C
17  B
18  B
19  C
20  B

                                                                      Page S-16

Part 6: Scoring for Knowledge and Judgment
==========================================

Students used test question 1 to select how they wanted to 
take their test and to have it scored (page S-17).
------------------------------------------------------------
l. Please score my test:

   A. For knowledge only.  I will mark all answers and let
      the test estimate what I know.  Scoring will start
      with zero plus (Wrong = 0, Blank = 0, Right = 2).

   B. For knowledge and judgment.  I will take the
      responsibility for reporting what I know and will
      mark what I know with confidence.  Scoring will start 
      with 50% plus (Wrong = -1, Blank = 0, Right = +1).
------------------------------------------------------------

Question 14 (all Cs) was a bad question found during the 
test period.  Its weight was set to zero.  

Question 17 required teacher evaluation.  This can result in 
a number of points added to the test.  Or the teacher can 
mark A for right and B for wrong and have it analyzed with 
the other questions.  Its weight can be set to 1-5 (A-E) 
points.  (If more points are needed, assign two question 
numbers to the essay or problem question and score it in two 
parts each worth up to 5 points maximum.)

The weight line is all Bs (2 points per question).  This is
convenient when using both methods of administering and 
scoring tests.  It eliminates fractions.  Student ~001 
elected to take the test both ways.

The above examples provide insight into the flexibility of 
testing and scoring possible with TRUE SCORE.  Traditional 
testing, for ranking, and testing for knowledge and judgment 
can be mixed as needed.  Students can be given the choice of 
how they are evaluated or all students can use the same method.  

Most students are afraid of a knowledge and judgment test 
when first introduced.  They do not know what they know.  
They have had little practice.  It was not their 
responsibility.  They relied on the teacher to tell them.  

Most students like knowledge and judgment tests after they 
have practiced a bit or have seen how other students in the
class like them.  They like to know how good their own
judgment is.  Judgment is as rewarding and valuable as 
knowledge.  The current trend in testing is headed in this 
direction (quantity with quality).

Select DEMO4.ANS, click Process, for a full set of reports.


                                                                      Page S-17

DEMO4.ANS  Student ID and Answers
---------------------------------

479021001 ACBDBCBCCBAADCDD ACCAA
479021001 BC       BAADCDD ACCAA
480900002 AADCBDADBBAADCDD ACCAB
482923003 BC A BBD BAB CDA  BC B
483065004 ACDBABBCEBAADCDB ACCDA
483849005 BDD   CDBBAADCDC ACCAA
485080006 ACDCCBCDEBABDCDD ACBDC
485112007 BCDC CDDD AADCDD  CCAA
485987008 ACDBABBDBAAADCDC ACBAA
486765009 BCDCAB DBBAADCDD ACCAA
486926010 ACCDABACBBAAACBA DAADD
486943011 BCD ABADBBAADCDD ACCAA
487963012 ACDDABADBBAADCDD ACCAA
489846013 BCDCABBD BAADC A  CCAA
489909014 ACACABDDBBAAACDD ACDCA
489948015 B A ACDDBBAA CDD ACCAA
491907016 ACDBABDDBBAADCDD ACCAA
494862017 BCACAB C BAADCDB  CA A
497687018 ACDCACBCDAADDCAD ACACD
498741019 BCD D BDBBAADCCD ACCAA
498749020 ACDCACCDEBAADCDD ACCAA
499843021 BCC C DD BAADCDD ACCAA
508236022 ACDCACBDBBAADCDD ACCAA
515943023 BB CABDD BADDCDD  CCAA
999000111  CDCABCDBBAADCDD ACCAA
999001222  BBBBBBBBBBBB BB BBBBB

(Leave a space below data lines before commenting.)

------------------------------------------------------------
1-9       Student ID (Letters or Numbers).
11-111    100 Answers (ABCDE).

999000111 Answer Key ID. to Score Knowledge only.
999001111   "     "   "  to Score Knowledge and Judgment.

999000222 Weight ID. with no Student Choice of Scoring.
999001222   "     "  with Student Choice as Question 1.
          Weights: (Blank = 0, A=1, B=2, C=3, D=4, F=5).
------------------------------------------------------------












                                                                      Page S-18

Knowledge and Judgment Results
------------------------------
CAUTION: One item analysis can be obtained from multiple 
forms of each evaluation method.  The two methods cannot be 
combined as traditional testing does not measure judgment. 

Scoring Summary (page S-19)
---------------------------
Both methods of scoring print out together.  Student ~001 
elected to use both methods of evaluation.

Ranked Raw Scores (page S-20)
-----------------------------
Scoring for judgment provides additional information for 
counseling students.  Two students have the same score.  The 
high quality student (80% right or better) should continue 
with her present study habits.  The low quality student 
needs to change his study and reporting habits.

Histogram (page S-21)
---------------------
The histograms for the two methods are similar.  When 
students are given a choice of methods the better students 
tend to select knowledge and judgment evaluation.

The Quality of Student Judgment Base Graph (page S-22)
------------------------------------------------------
Quantity and quality are presented in two dimensions.  
Students can readily see how their performance relates to 
their classmates.  The graph gives quality a meaning that 
students do not get from numbers alone.

The Quality of Student Judgment (DEMO4.ANS) (page S-23)
-------------------------------------------------------
Student ~003, with the lowest score, has this to consider 
when looking at the graph, "If I had taken the 
responsibility to report just what I actually knew, my grade 
could have been 68% rather than 53%."  Developing self-
judgment takes practice (learning and reporting) and a 
meaningful reward.

Item Analysis (Knowledge and Judgment) (page S-24)
--------------------------------------------------
Judgment adds a second estimate of difficulty based on the 
students who elect to answer the question.  Any question 
with a weight of zero is only tabulated.

Short Form Item Analysis (Knowledge and Judgment (page S-25)
------------------------------------------------------------
Judgment presents a "feeling of what the class actually 
knows" comparable to reading essays but with the speed and 
reproducibility of machine scoring.  Students are free to 
report what they know with confidence, what they find as 
real.  Some label this authentic assessment.

                                                                      Page S-19

SCORING SUMMARY
---------------
              DEMO4.ANS  13:34:31  02-18-1998
  (View and Print with Courier or Other Fixed-Pitch Font)
------------------------------------------------------------
 STUDENT  WG  BK  RT  SCORE AN:ACBDBCBCCB AADCDD ACC AA         
479021001  7   0  12  024.0 EV: 200000002 222 22 222 22 

 STUDENT  WG  BK  RT  SCORE AN:BC       B AADCDD ACC AA         
479021001  0   7* 12  031.0 EV: 211111112 222 22 222 22 

 STUDENT  WG  BK  RT  SCORE AN:AADCBDADBB AADCDD ACC AB         
480900002  5   0  14  028.0 EV: 022000222 222 22 222 20 

 STUDENT  WG  BK  RT  SCORE AN:BC A BBD B AB CDA  BC  B         
482923003  6   6*  7  020.0 EV: 210120212 201 20 102 10 

 STUDENT  WG  BK  RT  SCORE AN:ACDBABBCEB AADCDB ACC DA         
483065004  6   0  13  026.0 EV: 220220002 222 20 222 02 

 STUDENT  WG  BK  RT  SCORE AN:BDD   CDBB AADCDC ACC AA         
483849005  2   3* 14  031.0 EV: 021112222 222 20 222 22 

 STUDENT  WG  BK  RT  SCORE AN:ACDCCBCDEB ABDCDD ACB DC         
485080006  6   0  13  026.0 EV: 222022202 202 22 220 00 

 STUDENT  WG  BK  RT  SCORE AN:BCDC CDDD  AADCDD  CC AA         
485112007  3   3* 13  029.0 EV: 222100201 222 22 122 22 

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

 STUDENT  WG  BK  RT  SCORE AN:BCC C DD B AADCDD ACC AA         
499843021  3   3* 13  029.0 EV: 201010212 222 22 222 22 

 STUDENT  WG  BK  RT  SCORE AN:ACDCACBDBB AADCDD ACC AA         
508236022  2   0  17  034.0 EV: 222200222 222 22 222 22 

 STUDENT  WG  BK  RT  SCORE AN:BB CABDD B ADDCDD  CC AA         
515943023  3   3* 13  029.0 EV: 012220212 202 22 122 22 
------------------------------------------------------------
STUDENT:  Examinee's ID number or name (9 characters).
WG:       The number of wrong answers.
RT:       The number of right answers.
AN:       The answers marked by the examinee.

          SCORING FOR KNOWLEDGE ONLY
BK:       Blank = not marked (counted as wrong).
SCORE:    Raw score (right x weight).
EV:       0 = wrong, 1 = not marked, 2 = right.

*:        SCORING FOR KNOWLEDGE AND JUDGMENT
BK:       Blank = do not know.
SCORE:    Raw score (right + do not know x weights).
EV:       0 = wrong, 1 = do not know, 2 = right.

                                                                      Page S-20

RANKED RAW SCORES
-----------------
                    KNOWLEDGE
         DEMO4.ANS  13:34:31  02-18-1998
--------------------------------------------------
RANK  STUDENT    WG   RT  % SCORE  RAW SCORE
--------------------------------------------------
  1  498749020    2   17   089.5     034.0
  2  508236022    2   17   089.5     034.0
  3  487963012    2   17   089.5     034.0
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
 10  479021001    7   12   063.2     024.0
 11  497687018   10    9   047.4     018.0
 12  486926010   12    7   036.8     014.0
--------------------------------------------------
AVERAGE:                   071.9     027.3
--------------------------------------------------
WG:         The number of wrong answers including blanks.
RAW SCORE:  A quantitative estimate (right x weight).

                 KNOWLEDGE AND JUDGMENT
             DEMO4.ANS  13:34:31  02-18-1998
------------------------------------------------------------
RANK  STUDENT    WG  BLK   RT  MARK  % RT  % SCOR  RW SCORE
------------------------------------------------------------
  1  486765009    0    1   18   18    100   097.4    037.0
  2  486943011    1    1   17   18     94   092.1    035.0
  3  489846013    2    3   14   16     88   081.6    031.0
  4  483849005    2    3   14   16     88   081.6    031.0
  5  479021001    0    7   12   12    100   081.6    031.0
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
 10  485112007    3    3   13   16     81   076.3    029.0
 11  494862017    4    4   11   15     73   068.4    026.0
 12  482923003    6    6    7   13     54   052.6    020.0
------------------------------------------------------------
AVERAGE:                        16     84   078.3    029.8     
------------------------------------------------------------
WG:         The number of wrong answers (poor judgment).
BK:         The number of answers reported as unknown.
RT:         The number of right answers (good judgment).
# MARKED:   What a student believed he/she knew or could do.
% RIGHT:    How well a student knew, self-judgment score.
% SCORE:    The raw score expressed as a percentile.
RAW SCORE:  A quantitative score with a qualitative value.

Faculty Advisement and Student Practice:

1. If the % RIGHT is higher than 80% and the raw score low,
   advise the student to study more with current methods.

2. If the % RIGHT is lower than 70%, advise the student to
   a. change study habits from rote memory to making sense.
   b. change test habits from random guessing to reporting
      what he/she actually knows with confidence.

                                                                      Page S-21

HISTOGRAM DEMO4.ANS
-------------------
KNOWLEDGE                     KNOWLEDGE AND JUDGMENT
13:34:31  02-18-1998          13:34:31  02-18-1998
-------------------------     ------------------------------
PCT   SCORE  FREQ             PCT   SCORE  FREQ
-------------------------     ------------------------------
                               97     37     1 :X
                               95     36     0 :
                               92     35     1 :X
 89     34     4 :XXXX         89     34     0 :
 87     33     0 :             87     33     0 :
 84     32     0 :             84     32     0 :
 82     31     0 :             82     31     3 :XXX
 79     30     0 :             79     30     1 :X
 76     29     0 :             76     29     4 :XXXX 
 74     28     3 :XXX          74     28     0 :
 71     27     0 :             71     27     0 :
 68     26     2 :XX           68     26     1 :X
 66     25     0 :             66     25     0 :
 63     24     1 :X            63     24     0 :
 61     23     0 :             61     23     0 :
 58     22     0 :             58     22     0 : 
 55     21     0 :             55     21     0 :
 53     20     0 :             53     20     1 :X
 50     19     0 :
 47     18     1 :X
 45     17     0 :
 42     16     0 :
 39     15     0 :
 37     14     1 :X
-------------------------     ------------------------------
 72     27    12               78     30    12
-------------------------     ------------------------------

The difference in the average scores for the two groups is 
not created by the two ways the students elected to be 
evaluated.  Students who know are confident and tend to 
elect to report what they know.  

Other students see these results and then switch to taking 
responsibility for reporting what they know.  There are two 
typical results.  One is the student with good judgment who 
obtains a better grade.  Studying now becomes a process of 
making sense of assignments rather than just rote memorizing 
nonsense.

The other is the student with poor judgment.  He starts with 
a score of 50% and goes down hill to the same score he got 
from guessing at the answers on a traditional test.  He now 
realizes that it is how he learns and reports that makes the 
difference in grades, not the method of testing.  The same 
amount of time making sense of an assignment will produce 
better grades than if spent memorizing nonsense.

                                                                      Page S-22

THE QUALITY OF STUDENT JUDGMENT (BASE GRAPH)
--------------------------------------------
                                                100>/: 100% 
                                                  /  :
                                                /    :      
                                              /      :      
                                            /        :      
                                       90>/          :  90% 
                                        /   .        :      
                                      /       .      :      
                        S           /           .    :      
                      R           /               .  :
                    E        80>/                   /:  80% 
                  W           /   .               /  :      
                S           /       .           /    :      
              N           /           .       /      :      
            A           /               .   /        :      
                   70>/  R E C I T I N G  /          :  70% 
         T          /   .   S E N S E   /   .        :      
       H          /       .           /       .      :      
     G          /           .       /           .    :      
   I          /               .   /               .  :      
 R       60>/      H I G H      /       L O W       .:  60% 
          /   . Q U A L I T Y /   . Q U A L I T Y .  :      
        /       .           /       .           .    :      
      /           .       /           .       .      :      
    /               .   /               .   .        :      
0>*     S T A R T     /                   .          :  50% 
    \               /   .               .   .        :      
      \           /       .           .       .      :      
        \       /           .       .           .    :      
          \   /               .   .               .  :      
 W       40>\                   .  G U E S S I N G  .:  40% 
   R          \               .   .               .  :      
     O          \           .       .           .    :      
       N          \       .           .       .      :      
         G          \   .               .   .        :      
                   30>\                   .          :  30% 
             A          \               .   .        :      
               N          \           .       .      :      
                 S          \       .           .    :      
                   W          \   .               .  :      
                     E       20>\    C H A N C E    .:  20% 
                       R          \                  :
  . . . . . . . . . . . . . . . . . . . . . . . . . .:        
  0        10        20        30        40        50      
    THE NUMBER OF QUESTIONS MARKED WITH CONFIDENCE    SCORE 

This base graph is replotted on the following page for the 
twelve students who elected to report what they knew with 
confidence.  Unlike a histogram that ranks, this graph shows 
a student's position in the class by both rank and quality.  
See Ranked Raw Scores for the quality indicator, percent 
right (page S-20).

                                                                      Page S-23

THE QUALITY OF STUDENT JUDGMENT
-------------------------------
              DEMO4.ANS  13:34:31  02-18-1998

    THE NUMBER OF QUESTIONS MARKED WITH CONFIDENCE    SCORE
  0         4         8        11        15        19
  . . . . . . . . . . . . . . . . . . . . . . . . . .:
                                    >/         .     :  85 
                                    /           .    :     
                           S       /             .   :     
                                  1         2     .  :     
                         R       /                 . :     
                             80>/                   /:  80%
                       E       / .             1   / :     
                              /   .               /  :     
                     W       /     .             /   :     
                            /       .       4   /    :     
                   S      >/         .         /     :  75 
                          /           .       /      :     
                 N       /             .     /       :     
                        /               .   /        :     
               A       /                 . /         :     
                   70>/  R E C I T I N G  /          :  70%
                     / .                 / .         :     
                    /   .   S E N S E   / 1 .        :     
          T        /     .             /     .       :     
                  /       .           /       .      :     
        H       >/         .         /         .     :  65 
                /           .       /           .    :     
      G        /             .     /             .   :     
              /    H I G H    .   /     L O W     .  :     
    I        /                 . /                 . :     
         60>/   Q U A L I T Y   /   Q U A L I T Y   .:  60%
  R        / .                 / .                 . :     
          /   .               /   .               .  :     
         /     .             /     .             .   :     
        /       .           /       .           .    :     
      >/         .         /         .         .     :  55 
      /           .       /           .       .      :     
     /             .     /          1  .     .       :     
  . . . . . . . . . . . . . . . . . . . . . . . . . .:
  0         4         8        11        15        19
    THE NUMBER OF QUESTIONS MARKED WITH CONFIDENCE    SCORE

The student on the left side of a score line (with the 
highest percent right) has the highest quality work (see the 
82% score line).  See Ranked Scores for the quality 
indicator, percent right, (page S-20).

Students in the LOW QUALITY and GUESSING areas are unaware 
that they do not know.  They often have no idea how to take 
responsibility for learning.  Seeing their position on this 
graph helps them realize they need to change study habits.


                                                                      Page S-24

ITEM ANALYSIS (KNOWLEDGE AND JUDGMENT)
--------------------------------------
                       DEMO4.ANS  13:34:31  02-18-1998
   MARK DISTRIBUTION AND TWO ESTIMATES EACH OF DIFFICULTY AND DISCRIMINATION
           (View and Print with Courier or Other Fixed-Pitch Font)
------------------------------------------------------------------------------
TEST           MARK OPTION DISTRIBUTION            DIFFICULTY (%)     DISCRIM-
ITEM   ---------------------------------------    -----------------   INATION
       DONOT       KNOW, BELIEVE, OR GUESS        KNOW  EASE  K X E   --------
       KNOW    A=1    B=2    C=3    D=4    E=5    -----------------   CHI PT
       ---------------------------------------    TEST  ITEM  ITEM    SQR BIR
        H/L    H/L    H/L    H/L    H/L    H/L    MEAN   WT   SCORE     P<=
------------------------------------------------------------------------------

  1.      0      0     12      0      0      0    (Student Evaluation Choice)
        0/0    0/0    6/6    0/0    0/0    0/0 

  3.      3      2      0      1      6*     0      75    67    50    3.0  .60
        1/2    0/2    0/0    0/1    5/1    0/0      78   5.3    63      Good

  7.      3      1      3      1*     4      0      75    11     8    0.0  .09
        2/1    1/0    2/1    1/0    0/4    0/0      78   5.3    21**  

 14.      0      0      0     12*     0      0    (Informational, Survey Item)
        0/0    0/0    0/0    6/6    0/0    0/0

 21.      2     10*     0      0      0      0      83   100    83    0.6  .75
        0/2    6/4    0/0    0/0    0/0    0/0      78   5.3    92      Good

 ------------------------------------------------------------------------------
AVERAGE CHI SQUARE FOR THE TEST = 0.37  (3.84 = 5% LEVEL OF SIGNIFICANCE)
   (If below 3.84, combine scores with other tests to set course grades.)
Kuder-Richardson 20 = 0.86
  (Estimate of test homogeneity and reliability.)
Standard Error of Measurement (SEM) = 1.49
  (Estimate of individual raw score reliability.)
The true score is within +/-1 SEM 68% of the time; 2 SEM 95% of the time.

*  The right answer in the Mark Option Distribution.
** 1 BAD QUESTION with a WEIGHT of 5.26

H/L      Number scoring in the higher/lower half of the ranked scores.
KNOW     Percent of class answering the question with confidence.***
EASE     Percent of those answering, answering correctly.***
K X E    Difficulty based on students reporting with confidence.***
MEAN     Average test score as a percent.
WT       Weight of item as percent of test.
SCORE    Difficulty based on entire class.
CHI SQR  Chi-Square estimate of discrimination ability.
PT BIR   Point Biseral R estimate of discrimination ability.
P<=      Average of Chi-Square and Point Biseral R expressed as Good,
         Better, Best (Level of Significance: 10%, 5%, and 1%).

*** Only available when scoring for knowledge and judgment.

                                                                      Page S-25

SHORT FORM ITEM ANALYSIS (KNOWLEDGE AND JUDGMENT)
-------------------------------------------------
                    DEMO4.ANS  13:34:31  02-18-1998
------------------------------------------------------------------------
                  1...5...10 ....5...20 ....5...30 ....5...40 ....5...50
ANSWER KEY:        CDCABCDBB AADCDD ACC AA 
ITEM WEIGHTS:     0222222222 2220220222 22 
DIFFICULTY:             D                  
DISCRIMINATION:     T                   T  
------------------------------------------------------------------------
Difficulty:     Upper Case Letter = More than 50% of the class answered
 (THE BAD)                          and chose this wrong answer.
                  (A majority believed they knew but did not know.)*
                Lower Case Letter = Less than 50% of the class answered
                                    and chose this wrong answer.
                  (A minority believed they knew but did not know.)**
                A Number = The most popular answer was right but the
                           question performed below the 37.5% level of
                           difficulty.  The percent of the class
                           answering (3 = 30%, 7 = 70%, H = 100%).
                  (Either a minority believed they knew and were right
                   or a majority believed they knew and were wrong.)***
                  (Discuss the above questions with the class.)
 (THE GOOD)     Blank = Difficulty (K X E and Item Score) higher than
                        37.5%****

Discrimination: T = Good, 5 = Better, 1 = Best (Level of Significance:
 (THE BEST)     10%, 5%, and 1%).
                  (These are your best questions for new tests.)

*     Please keep in mind these students are not being forced to guess 
      at answers on the test.  They are free to use the test to report 
      with confidence what they know or can do.  

      When a majority answers and they select a wrong answer, a serious 
      misconception exists or there is a problem with the question.  
      Look at the mark option distribution for additional clues.  
      
      Which type of student answered?  Those in the higher or lower 
      part of the ranked scores?  Often low ranking students will select 
      a wrong answer that comes directly from an error in lecture or 
      their study materials.

**    The question is not working.  Typically low ranking students will 
      be answering.  If not, ask the students for their reasoning.

***   The question is working.  It is difficult.  Typically high ranking 
      students will be answering.  If the question is also highly 
      discriminating, it is the type testing companies use to separate 
      the scholars from the rest.

****  Questions usable for setting grades.  The quality of these 
      questions is further ranked by the discrimination labels Good, 
      Better, and Best.

                                                                      Page S-26



Part 7: F5 Customize Reports and Export Files (1998)
====================================================

Press F5 to change values.  Select frequently changed
parameters into a menu that shows each time the program starts.

OPERATION      Automatic, see page 9 in HELP.TXT.   
               Manual,    see page 10 in HELP.TXT.

ANSWER SOURCE  Disk A,    see pages 3-4 in HELP.TXT.
               Any Disk,  see pages 3-4 in HELP.TXT.

ANALYSIS       On         Advanced Grouping of questions and students.
               Off        Basic scoring and analysis.

REPORT BANNER  On         Program requests banner for each test.
               Off        File name, date, and times are always printed.
               Customize  Prints the same 144 character maximum banner.

PART BREAK     On         Breaks between report parts.
               1line      One blank line between parts.
               5lines     Five blank lines between parts.

WATERMARK      On         Prints program serial number at end of report.

               Off        No watermark.
               Customize  Prints the same 72 characters maximum.

EXPORT FILE    String     Tab delimited in columns. 
               Comma      A,b,c,d delimited.
               DQcomma    "A","b","c","d" delimited.
               None       No export files produced.

SCORE VALUE    Numerical  Actual count.
               Percent    Percent.
               Both       Both.

SCORE ORDER    Rank       Ordered by score.
               ID         Ordered by student ID.
               Both       Both.

FILE HEADINGS  On         Prints headings, data and statistics for posting.
               Off        Print student ID and score only for grade book 
                          export.


Other features can be added, or moved here from the CustomS.INI file.









                                                                      Page S-27

Part 8: Advanced Question and Student Analysis (1999)
=====================================================

The Answer Key is always scanned with the answer sheets.  
The remaining keys are generated by True Score or entered 
after True Score makes the first analysis.
--------------------------------------------------------------

Answer Key
    999000111 Uses the letters A, B, C, D, E, and no mark.

Weight Key
    999000222 Uses weights of A = 1, B = 2, C = 3, D = 4, E = 5
              G = 7 for zero the question and add 1 point to each score.
              H = 8 for zero the question and add 2 points to each score.
              No mark = 0.  The weight is distributed across the remaining
              questions.  If no key is found, True Score uses A = 1.  (The
              weights are printed above as numbers for easy reference.)

Topic Key         (Question Group)
    999000333 Uses letter A to mark the first group of
              questions, B for the next, and ect.

Level of Thinking (Question Group; type, performance, application)
    999000444 Also uses letters as above.  If no key is found
              True Score creates one: A = Easy, B = Unfinished and
              C = Discriminating

Test Form Key     (Student Group)
    999000555 Also uses letters as above.  If no key is found
              True Score creates one: A = Odd and B = Even (split-test).
              The Key sequence is the same as the student ID sequence
              in the Scoring Summary (TS 1.1).  Students can also be 
              grouped using A to mark the first group and ect.

Information Key   (Student Group)
    999000666 Use any letter to mark each question of interest
              for full analysis (who marked and the average score for
              each option).  True Score always prints an option
              performance summary for all questions.

-----------------------------------------------------------------

All keys can be created, edited and deleted directly within
True Score by double clicking the answer file and using the
True Score edit feature or with the edit feature of Answer Check.
-----------------------------------------------------------------

                            






                                                                      Page S-28

Part 9: License and Warranty
============================

Licenses
--------

A free license is given to students and individual teachers.
Email freetsc@nine-patch.com to register.  For all other
usage, email tsc@nine-patch.com for permission, a license
or current pricing.

Two types of license are available, single-use and site.

Single-Use:  For one computer or one person at one time.

Site:        For a group of computers or of persons, as a       
             classroom, building, department or of persons 
             who can be grouped and labeled by their related 
             activities or organization.

The advantages of group licenses are their lower cost over 
single-use licenses and the reduction in management costs of 
keeping track of who has the single-use license to avoid 
copyright violations.

Satisfaction is guaranteed.  If not satisfied, the license 
fee will be returned (less any agent or processing fees) 
upon receipt, within 100 days of purchase, of the master 
disk that was ordered from Nine-Patch Software.

You may not use, copy, rent, lease, sell, modify, decompile,
disassemble, otherwise reverse engineer, or transfer the 
program without written permission..


Warranty
--------
Users of True Score with Multiple Form Test Item Analysis 
must accept this disclaimer of warranty:

True Score is supplied as is.  The Nine-Patch Software 
disclaims all warranties, expressed or implied, including, 
without limitation, the warranties of merchantability and of 
fitness for any purpose.  Nine-Patch Software assumes no 
liability for damages, direct or consequential, which may 
result from the use of True Score.

In all cases, satisfaction is quarantined or your money back 
as stated in Licenses above.






                                                                      Page S-29

Part 10: Order and License Form 
===============================

------------------------------------------------------------
0100             ORDER and LICENSE FORM 5.0             0002
------------------------------------------------------------

A free license is given to students and individual teachers.
Email freetsc@nine-patch.com to register.  For all other usage,
email tsc@nine-patch.com for permission, a license or current 
pricing.

Name:              _________________________________________

Street:            _________________________________________

City, State & ZIP: _________________________________________

License Name:      _________________________________________
                     (Please print, 35 characters maximum)

SINGLE-USE LICENSE: (One person or one computer.)

  For Windows               Number: _____ x $239.95  _______
  Satisfaction Guaranteed.     

SITE LICENSE: Each five computers   _____ x $239.95  _______

SITE LICENSE: Each five users       _____ X $239.95  _______  

(Check http://www.nine-patch.com for current pricing.)

SHIPPING AND HANDLING:                               _$5.00_

In Missouri, add 6.975% sales tax or tax number:     _______
             TIN:______________________________
TOTAL (Pay to Nine-Patch Software in U.S. dollars.)  _______
------------------------------------------------------------
(_)Check           (_)Money Order          (_)Purchase Order
------------------------------------------------------------
Mail: Nine-Patch Software, PO Box 30562, Columbia, MO 65205
------------------------------------------------------------

Found program by:___________________________________________

Intended program use:_______________________________________









                         TRUE SCORE
                
Main Features
=============


A. Scores and Analyzes Individual Tests.

   1. Score Summary.
   2. Ranked Scores.
   3. Histogram.
   4. Student Quality Plot.
   5. Item Analysis.
      a. Mark Distribution.
      b. Two Estimates of Difficulty.
      c. Two Estimates of Discrimination Ability.

   6. Short Form Item Analysis (Four Lines).
      a. Questions Marked Bad, Good, Better and Best.
      b. Questions Marked for Class Discussion.


B. Produces One Item Analysis from Multiple Forms.

   1. Increases the Quality of the Analysis.
   2. Saves the Time Needed to Collate Separate
      Analyses for each Form of the Test.


C. Scores Two Ways.

   1. Traditional Knowledge for Ranking (Quantity).
   2. Knowledge and Judgment (Quantity and Quality).
   3. Students can Select How They will be Evaluated.
   4. Helps Make the Transition from Traditional Ranking to 
      the Current Trend toward Quantity and Quality.
   5. Uses the Same Questions for Both Types of Scoring.


D. Advanced Analysis.

   1. Groups questions by topic, type, level of thinking.
   2. Groups students by section and by answer options to 
      survey or informational questions.


E. Automatic Operation.

   1. Scores tests saved to a floppy disk.
   2. Save the disk with all test records in your file cabinet.



              Copyright 2000 Richard A. Hart

                            -i-


                         TRUE SCORE 

Table of Contents
=================

Main Features  . . . . . . . . . . . . . . . . . . . . .    i

Table of Contents  . . . . . . . . . . . . . . . . . . .   ii 

Part 1a: The Three List Boxes  . . . . . . . . . . . . .  S-2     

Part 1b: The Control Buttons . . . . . . . . . . . . . .  S-3

Part 2: Windows File Management Setup  . . . . . . . . .  S-4

Part 3: One Test Item Analysis (DEMO1.ANS) . . . . . . .  S-5

Part 4: Sample Results . . . . . . . . . . . . . . . . .  S-6

   SCORING SUMMARY . . . . . . . . . . . . . . . . . . .  S-7
   RANKED RAW SCORES . . . . . . . . . . . . . . . . . .  S-8
   ITEM ANALYSIS . . . . . . . . . . . . . . . . . . . .  S-9
   SHORT FORM ITEM ANALYSIS  . . . . . . . . . . . . . . S-10

Part 5: Multiple Form Test Item Analysis . . . . . . . . S-11

   TEST PROCESSING RECORD  . . . . . . . . . . . . . . . S-12
   DEMO2.ANS and DEMO3.ANS . . . . . . . . . . . . . . . S-13
   DEMO2.IAI . . . . . . . . . . . . . . . . . . . . . . S-14
   DEMO3.IAI . . . . . . . . . . . . . . . . . . . . . . S-15

Part 6: Scoring for Knowledge and Judgment . . . . . . . S-16

   DEMO4.ANS Student ID and Answers  . . . . . . . . . . S-17
   Knowledge and Judgment Results  . . . . . . . . . . . S-18

      SCORING SUMMARY  . . . . . . . . . . . . . . . . . S-19
      RANKED RAW SCORES  . . . . . . . . . . . . . . . . S-20
      HISTOGRAM DEMO4.ANS  . . . . . . . . . . . . . . . S-21
      THE QUALITY OF STUDENT JUDGMENT (BASE GRAPH) . . . S-22
      THE QUALITY OF STUDENT JUDGMENT  . . . . . . . . . S-23
      ITEM ANALYSIS (KNOWLEDGE AND JUDGMENT) . . . . . . S-24
      SHORT FORM ITEM ANALYSIS (KNOWLEDGE AND JUDGMENT   S-25

Part 7: F5 Customize Reports and Export Files (1998) . . S-26

Part 8: Advanced Question and Student Analysis (1999)  . S-27

Part 9: License and Warranty . . . . . . . . . . . . . . S-28

Part 10: Order and License Registration Form . . . . . . S-29

                            -ii-








                       TRUE SCORE
                          with

         DETAILED QUESTION AND STUDENT ANALYSIS





















                      Version 5.00
             COPYRIGHT 2000 RICHARD A. HART





















                         Posted
                           at
                http://www.nine-patch.com

