Title Page (Project Title, Name, Date)

Enhanced Position Location Reporting System Maintenance Training for the Ground Common Organizational Repairer Brian Lafferty 20 March 2006

Introduction - 2 points (Describe the problem, why it is a problem, why it is a significant problem etc. what the context of the problem etc is.) Currently Marines serving in Military Occupational Specialty (MOS) 2844, Ground Common Organizational Repairer are not trained to perform Organizational Level repair on the AN/VSQ-2C, Enhanced Position Location Reporting System (EPLRS) radio system. This is a problem because this MOS is responsible for troubleshooting and repairing single channel radios to the Line Replaceable Unit (LRU) and EPLRS fits this category. This problem contributes to increased down time for these critical command and control assets. Literature Review - 10 points (preliminary literature review, three sources minimum) The challenge in creating an effective resource that is more than a slick computer based regurgitation of the technical manual lies in selecting a design model that will result in a product that has educational value greater than the sum of its parts. I came across Using the ARCS Motivational Process in Computer-Based Instruction and Distance Education and found that it addressed many of my concerns. (Keller 39-47) Keller authored the ARCS process in 1988 and pioneered its use in creating and selecting motivational tactics “…based on an analysis of the target audience and existing instructional materials.” (Keller 39-47) The ARCS model is a seven-step system for enhancing learner motivation in four categories, Attention, Relevance, Confidence and Satisfaction. The ARCS model has been the subject of numerous studies and critiques. The 1999 article cites two such studies and details the use of ARCS in distance learning applications. Keller admits that the challenge faced in applying ARCS to DL is that the ability of the instructor to change tact in response to observations of and feedback from the students is greatly reduced or eliminated. One possible solution to this challenge is to pick the most important element of ARCS, focus on it, and let the other three take care of themselves. The body of research conducted to prove or disprove Keller’s ARCS is vast and the results are inconsistent. (Means et al. 5-18) Keller’s opponents take the stance that you can’t please all the people all the time and trying to do so results in more harm than good. They suggest that Relevance is the single element of ARCS that can consistently and predictably impact learners’ motivation. In the mid 1990’s Means, Jonassen and Dwyer did research to test this theory. They tested two independent variables relating to relevance. The first was intrinsic relevance: students who had a demonstrable need for the information versus those who did not. The second was extrinsic relevance; some students received content that was deliberately enhanced to relate to them such as using personal pronouns (“your heart” vs. “the human heart”) and associating the unfamiliar to the familiar. The groups that had both intrinsic and extrinsic relevance outperformed the other groups in achievement and reported more positive responses on satisfaction surveys. (Means et al. 5-18) This research in ARCS and the impact of enhanced relevance will be useful in shaping the design of my recourse. Another area where I sought research that would help shape my resource was how to measure success. My initial vision for my resource is that it will be delivered in a myriad of locations, with zero instructor interaction and at a time of the learner’s choosing. In

my mind this pointed toward some type of survey instrument to capture the students’ reaction to the course or to query the student’s supervisor regarding any improvement in job performance or equipment status. Nonresponse to surveys among military members is a well-known dilemma and the journal Military Psychology published an extensive examination of the problem in 2004. Research showed that the top three reasons for nonresponse to surveys were that (1) response does not matter because results to not precipitate change, (2) general apathy toward the survey subject or survey itself, and (3) surveys are too long. (Newell et al. 265-276) In addition to documenting the nature and scope of the problem the authors explored possible methods for increasing the response rates. These suggestions included providing a summary of survey results to those participating in it, increasing the involvement of the (military) command, providing incentives, and shortening the surveys themselves. (Newell et al. 265-276) This information has direct implications in this project from the standpoint of avoiding the common pitfalls of using surveys to measure the success of learning. Method - 25 points as specified below Describe your analysis plan 5 points DR= Document Recovery FG= Focus Group INT= Interview WHO: Who is currently being trained on any aspects of EPLRS operation and maintenance? (DR, FG) *Whose job is it to diagnose and repair EPLRS hardware at the using unit? (DR, FG) Who is diagnosing and repairing EPLRS hardware at the using unit? (DR, FG, INT) *Whose job is it to diagnose and repair EPLRS firmware/software at the using unit? (DR, FG) Who is diagnosing and repairing EPLRS firmware/software at the using unit? (DR, FG, INT) WHAT: What are the applicable Individual Training Standards (ITS) for this MOS and duty area? (DR) What knowledge and skills do 2844’s need to affect maintenance on this equipment? (DR, FG, INT)

What knowledge and skills do 2844’s graduating from Ground Common Organizational Repairer’s Course have? (DR, INT) What tools and Test Measurement and Diagnostic Equipment (TMDE) are needed to effect maintenance on this equipment? (DR) What operator and technical manuals exist for this family of equipment? (DR) What peripheral devices are used with EPLRS? (DR) What host devices are used with EPLRS? (FG, INT) What defects are the most common causes of failure in this equipment? (INT, FG, DR) What training already exists for other items that may be applicable to this equipment? (DR, INT) What efforts to train 2844’s on EPLRS have been made in the past? (INT, DR) What is the expected lifecycle for this family of equipment? Is it worth taking the time to produce training? (DR) What are the plans to upgrade the firmware and software for EPLRS that may impact training? (DR) What Information Security (INFOSEC) considerations may impact design and development? (DR) What distance learning tools and techniques are currently being used by 2844’s? (DR, FG) HOW: How are the proficiencies of 2844’s measured now? (DR, INT, FG) What are effective ways to measure effectiveness of maintenance training? (MLB) What kind of training assessment do 2844’s prefer? (FG) What kind of training assessment do supervisors of 2844’s prefer? (INT, FG) How can the mean time between failures (MTBF) and mean time to repair (MTTR) before and after training be captured? Are these valid measures of training success? (DR, INT) Present the salient details of the analysis you conduct so far 5 points

I’ve been beating the drum for this training for nearly five years and to date I’ve had little luck drumming up interest in making it happen. There is a possibility that the problem that I want to solve exists only in my perception of the way things were in my specific unit during a specific period of time. For this reason it seemed logical to me to begin my analysis by recovering and analyzing the documents that contain the official job descriptions and training standards for MOS’s that use and maintain data and communication equipment. This part of the analysis addressed the “who?” questions marked with an asterisk above, “Whose job is it…” I began with a review of the Marine Corps Order 1200.16 of April 2005. This manual contains the job descriptions, prerequisites and training requirements for all MOS’s in the Marine Corps. The job descriptions, in all but a few examples, do not cite specific equipment, rather they speak in generalities. I scoured the manual for information that could be related to EPLRS. What I found confirmed my worst fears. MOS 0656 has the task of maintaining network systems including routers and MOS 2844 has the task of troubleshooting communication systems. The obvious overlap between these two areas allows both to point the finger at the other where EPLRS is concerned. I followed up this discovery by researching the Individual Training Standards for these two MOS’s that are found in Marine Corps Orders 1510.118 and 1510.44 (both published in 2001) In the manual covering MOS 0656 there are numerous training standards that specify EPLRS, but none of them are under MOS 0656, instead they are listed under 0621, Radio Operator. None of the EPLRS related operator tasks specify or allude to troubleshooting. The manual for the 2844 has the generic task “Perform corrective maintenance on single channel radios (SCR) to the line replaceable unit (LRU) level” which would include EPLRS. The results of this analysis of “Whose job is it…” won’t truly be complete until I can conduct a focus group and round-out my document recovery by pulling the equipment specific manuals. At this point the important thing is that the job definitions while not specifying EPLRS as a 2844 responsibility, they do not rule it out, and they do not specify it for a different MOS. Describe your design process 5 points; identify key decisions and how you will make them 1. Determine the scope and depth: The key decision regarding the scope of this project is deciding weather to make it a complete soup-to-nuts product that covers all aspects of the operation and maintenance or does it just focus on those areas that are either specific to EPLRS and/or not covered in existing training? For example EPLRS is an Ultra High Frequency (UHF), Secure (Comsec), Electronic Counter-Counter Measure (ECCM) Radio. It is also software programmable, Internet Protocol (IP) capable, remote configuration and Over the Air Rekey (OTAR) capable . The first half of these are common to any number of systems that 2844’s are trained on while the other half are unique to EPLRS. My instinct for this is to

make the comprehensive resource; if the technician is faced with a problem regarding UHF line of sight (LOS) and that information was omitted from the final resource, then it would be useless in resolving the issue and may even lead the tech away from the root of the problem. 2. Determine the delivery mechanism: In order for my resource to be of value to the user it will need to be capable of being used in a fully asynchronous manner. They will need to be able to begin at any time they wish and they need to be able to control the length of a given session. These constraints are pretty universal in Military distance education projects of this type. A final consideration that is very relevant to this product is that much of the content, while unclassified, is For Official Use Only (FOUO) in that it cannot be released to the public. There are three delivery mechanisms that I see that may be considered. The first is to seek authorization from the College of Continuing Education (CCE) to create a resource that could be made available on MarineNet. This would allow worldwide enrollment, world-class collection of data on student performance and a built-in evaluation and feedback ability. Not to mention that the obligation for safeguarding FOUO information would lie with them. The key problem with this choice is that CCE would require me to program the resource in Articulate™ using their pre-existing templates and navigation system; hardly a project that would merit a Master’s degree. My second avenue would be to create a resource that could be wholly contained on a CD. Proper labeling would satisfy the FOUO hurdle, but in my mind this is an extreme compromise of the level of skill that I plan to exit the ETEC program with. At this point, my preferred choice would be to create the resource for distribution via the World Wide Web and have it hosted on Twentynine Palms’ own TechNet. TechNet is SSL encrypted, PKI enabled, and only accessible from a .mil domain. This would enable me to do whatever is necessary to demonstrate to the University that the project was a significant educational and technological challenge, as well as enable me to create a database to store student data and progress. The trade-off here is that students can only access it from their work sites and not from their homes. Supervisors would have to allow time and access to computers in the workspace. 3. Determine student interactions: The constraints I’ve outlined above limit the types of student interactions that are available to me. Student-student and Student-instructor are almost completely off the table leaving only Student-content. Being limited to only S-C interaction should not be a significant limiter. Incorporating video elements, interactive Flash™ elements as well as critical content from the technical manuals should result in a product that is effective in getting across the information as well as keeping the student engaged. 4. Select a navigation system. This is probably the simplest element. My audience for this product has all participated in at least one of the Marine Corps’ Computer Based Training products. They all, more or less, use a similar navigation system. It makes sense to at least recreate the look and feel of these common navigation systems even if the code and behind-the-scenes stuff is different.

5. Determine what equals success: Success in a project like this is really only achieved if the student’s behavior is changed as a result of participating in the training. There is no simple way to translate achievement in the training to achievement on the job. There is considerable literature on the subject and as of this writing I have collected a few articles, such as Kraiger’s Application of Cognitive, Skill-Based, and Affective Theories of Learning Outcomes to New Methods of Training Evaluation. (Kraiger et al. 311-328) Present the design specifications 5 points; what rules will guide your development? 1. It is a maintenance course, not an operator’s course: focus on maintenance: While an understanding of proper operation is prerequisite to any troubleshooting process, it is too easy to get bogged down in the minutia of an operator’s course. Symptom identification and fault isolation are what set apart the technician from the operator. 2. You can’t have it all: The troubleshooting flowcharts in the manual are the comprehensive source of troubleshooting procedures. The resource should focus on the most common causes of failure so they can be identified and corrected quickly. A mechanism for teaching the technician when to grab the manual or ask for more help is a significant design consideration. 3. You can’t do it all: The flowcharts will, in many cases take you all the way to the bad circuit card or discrete component, Organizational maintenance, by definition, seeks to identify and replace the bad Line Replaceable Unit (LRU) not the Secondary Replaceable Unit (SRU) or component. 4. Align the resource to symptoms: If the technician has a power problem (won’t initialize or blows fuses) then the theory of operation and troubleshooting process for these symptoms should be grouped. The user should not have to traverse numerous modules of the resource to troubleshoot a single symptom. Explain alpha and beta testing protocols 5 points (you don't have to do the testing) The nature of my project will require me to alpha test in two parts. First, I plan to have ETEC classmates validate the educational value of the resource in terms of acceptable education practices, student interactions, and the like. The second part will be to tap into my peers in the Command, Control, Communications and Computer (C4) community to validate the scope, depth, and accuracy of the EPLRS content. At this point I’d still be open to making significant changes. At this early stage I’m not certain which of these two parts of the alpha test would be conducted first, I’m banking on the idea that a course of action will become clear as the development process matures. My EPLRS alpha testers are likely to be the supervisors of the Marines who will later be my pool of beta

testers. My beta test will serve to “proof read” the nearly completed resource. I don’t foresee making broad changes during the beta test. By happenstance, the very act of my alpha testers being confident enough in the resource to offer their Marines as beta testers has significant value as a form of evaluation of the product’s ability to teach. Summary - 3 points - (What did you learn about instructional design? What did you learn about the topic area in which you worked? Do you think you would be able to turn this into a full project? What would you change if you did turn this project into a Master's project? Etc.) Analysis, analysis analysis. The first time I saved this document I had about ten whowhat-how questions in my analysis plan. That list quickly grew to more than two-dozen. Each nibble of the analysis raises additional questions that have to be answered. I can easily see the list growing to 75 or 100 items as each step of the full analysis unfolds. Every journal article leads to another journal article. The citations in one article, even if only marginally relevant, point back to the foundational work in every area. ARCS, the science of surveys, and accepted design models all came out by digging into the bibliography of peripheral works. Try not to jump to making the thing. This project only served to confirm what was so obvious in our group work, that is, the natural tendency to want to jump into creating the resource without doing a proper analysis. For this project I have, in one fashion or another, been designing it in the back of my mind for years; that first sentence of the literature review was no small hurdle. Even the small amount of analysis that I have done so far has enlightened me to the incredible volume of things that I don’t know about EPLRS. Any inkling I may have had about serving as my own full time Subject Matter Expert dissipated quickly. As for growing this into a full project, I’m unsure. There are many challenges in taking this particular choice on, and since I’ve barely dipped my toe into the murky waters of the ETEC program, I feel like I don’t appreciate what alternatives may be available to me. If I were to take this on, one of the possible changes would be to broaden the scope. As stated earlier, a comprehensive product would contain sections or modules that would apply to different items of equipment in the inventory. It struck me that I might consider making a resource that pieced together a tailored string of lessons when you select an item of equipment from a menu. This is not unlike the concept of Reusable Learning Objects (RLO) that was discussed in my annotated bibliography. (Harden 43-51)

Citations: Keller, John M. "Using the ARCS Motivational Process in Computer-Based Instruction and Distance Education." New Directions for Teaching and Learning 78 (1999): 39-47. Means, Tammy B., David H. Jonassen, and Francis M. Dwyer. "Enhancing Relevance: Embedded ARCS Stratgies vs. Purpose." Educational Technology Research and Development 1997: 5-18. Newell, Carol E., Paul Rosenfeld, Rorie N. Harris, and Regina L. Hindelang. "Reasons for Nonresponse on U.S. Navy Surveys: A Closer Look." Military Psychology 2004: 265276. Kraiger, Kurt, J. Kevin Ford, and Eduardo Salas. "Application of Cognitive, Skill-Based, and Affective Theories of Learning Outcomes to New Methods of Training Evaluation." Journal of Applied Psychology April 1993: 311-328. Harden, Ronald M. "A New Vision for Distance Learning and Continuing Medical Education." The Journal of Continuing Education in the Health Professions 2005: 43-51. MCO P1200.16 MOS MANUAL. Quantico: US Government, 2005. MCO 1510.118 INDIVIDUAL TRAINING STANDARDS (ITS) SYSTEM FOR THE COMMAND AND CONTROL SYSTEMS (ENLISTED) OCCUPATIONAL FIELD (OCCFLD) 06. Quantico: US Government, 2001. MCO 1510.44C INDIVIDUAL TRAINING STANDARDS (ITS) SYSTEM FOR GROUND ELECTRONICS MAINTENANCE OCCUPATIONAL FIELD (OCCFLD) 28. Arlington: US Quantico, 2001.