Applicants must follow standard procedures for their sponsoring organization in applying to a graduate education program (APM, section 44 ). Applicants should have the sponsoring organization forward their letter of application to the Director of Admissions at the Naval Postgraduate School, Monterey, CA 93943. U.S. military officers, international military officers, U.S. Government civilians, and employees of allied nations may apply.

The application should include the following documentation:

1. Certified transcripts of all courses taken at the university level, including both undergraduate and graduate courses. An official copy of the results of a recent Graduate Record Examination (GRE) general test. International students who are not native speakers of English must also provide scores on the Test of English as a Foreign Language (TOEFL) examination.
2. Each applicant is requested to provide a 2–4-page letter describing their motivations, preparations, and career objectives for getting a MOVES Ph.D. What practical M&S experiences have you had and how do you expect that your efforts will make a difference? Experience has shown that such an essay assists applicants in preparing for Ph.D. work and assists faculty in providing counsel and guidance on how to best proceed.
3. Since the Ph.D. is a research degree, applicants must also include material demonstrating an ability to perform research, such as a master's thesis and/or research papers.
4. Three Reference letters should also be included with the application. The letters should include why the applicant has the necessary qualifications to successfully complete the rigorous academic program that leads to a Ph.D. At least one letter should be from someone who has direct experience with the academic performance of the applicant.

The MOVES Ph.D. program generally builds on the scientific knowledge gained from the MOVES MS program. An applicant should have a master's degree in modeling, virtual environments, and simulation (MOVES) or a related field. Examples of related fields include (but are not limited to) Computer Science, Human Factors (HF), Human-Systems Integration (HSI), Systems Engineering, and Information Sciences. Generally, an acceptable Ph.D. applicant must have above-average grades in a typical master's degree program. Because MOVES is a cross-disciplinary mix of multiple scientific domains that are built on a foundation of analytic and programming skills, a wide range of academic and applied backgrounds are possible. The committee will also take other evidence of research or academic ability into account in making a recommendation as to whether to admit an applicant.

The MOVES Ph.D. program committee will evaluate each applicant to gauge the minimum amount of time they may need to complete the program. Admitted Ph.D. students may begin in any quarter. However, given the MOVES course schedule, it is advantageous to being in the summer quarter (beginning in July) if possible.

Students are cautioned that admission to the Ph.D. program does not guarantee successful completion of the program. The primary criteria for successful completion of the program will be the academic quality of the work done by the student both in exams and in the content of the Dissertation. Success will require substantial effort, but effort alone is not enough. The work must reach the high academic standards expected of Ph.D. level work.

It is significantly more difficult to assess a student's qualifications for a Ph.D. admission than for other degrees. That is because the research work required for the Ph.D. requires substantial creativity and independence. In addition, past experience suggests that not all of the students admitted will complete the program. The purpose of the written and oral qualifying examinations (see below) is to give students early warning (within the first eighteen months of work towards the degree) if they are encountering trouble in the Ph.D. program.

The Ph.D. degree requires the equivalent of normally three academic years of study beyond the Master’s level, with at least one academic year (or its equivalent) being spent in residence at NPS.

Each student must complete the following milestones, which are detailed in the corresponding sections of this document:

1. Prepare for written exams and demonstrating MOVES core understanding
2. Pass a written qualifying examination
3. Form a dissertation committee
4. Pass an oral qualifying examination
5. Have dissertation proposal approved
6. Advancement to candidacy
7. Complete a dissertation
8. Pass a final examination

The following quarterly progression is an example student timeline and considered a baseline norm for new MOVES Ph.D. students. Actual student progression may deviate from this timeline.

Students are encouraged to finish preparation for and the completion of the written Qualifying exams within one year in order to dedicate the majority of their study period to dissertation-related research.

• Q1. MOVES Core initial assessment with Mentor, pick 3 topics for written exam, define program of study, meet with MOVES faculty, and meet other MOVES students with similar interests.
• Q2. Prepare for and pass the MOVES Core Breadth Assessment.
• Q3-Q4. Prepare for and pass the written qualifying exam based on the chosen 3 areas of study.
• Q5-Q6. Prepare for oral qualifying examination, prepare dissertation proposal, identify dissertation committee members, pass oral qualifying examination, and formally designate committee members. The committee reviews and approves the dissertation proposal and then proposes Advancement to Candidacy for approval by the Academic Council.
• Q7-12. Conduct doctoral research, write dissertation, successfully defend the dissertation, and publish the dissertation. It is important and recommended to publish one or more journal articles describing your work promptly, so that relevant scientific communities of interest know about your results.

No courses are specifically required for the MOVES Ph.D. degree unless the MOVES Ph.D. Committee so stipulates.

Since the MOVES Ph.D. program does not have a core course requirement, there is a need to ensure that all MOVES Ph.D. recipients have a common core knowledge of what constitutes Modeling and Simulation and what it means to do research at the Ph.D. level. The following process is intended to assess the knowledge level of the student at the start of the program, help identify the program of study for the student and provide a way to demonstrate an understanding of these core areas.

For this requirement the student will be expected to satisfy the MOVES Core Breadth Assessment (MCBA). This assessment will be conducted by the MOVES Ph.D. Committee normally in the middle of the second quarter after the student has started the program. The outcome of this assessment will either be that the student has demonstrated the expected level of understanding of the MOVES core areas and can proceed to focus full-time on preparation for the Written Qualifying Exams. If deficiencies are identified, a remediation plan will be established. In this later case, a second MCBA will be held or other criteria established to re-evaluate the student’s understanding of the deficient areas after completing the studies identified in the plan. This second evaluation will be held at a time based on the plan developed for addressing the deficiencies but must be completed before the written qualifying exams can be administered. If in the second assessment, the Ph.D. Committee is satisfied that the deficiencies have been successfully addressed, the student proceeds on to prepare for and take the Written Qualifying Exams. If, however, significant deficiencies are still present, the committee can recommend that the student consider terminating their pursuit of the Ph.D. because the likelihood of being able to pass the written and oral qualifying exams is very low.

To prepare for the MCBA, the student will be given a list of questions/topics soon after they start the program which will be the basis of the assessment. They will go through these questions/topics with their Mentor (see below for the Mentor role) and determine how prepared the student feels to be able to discuss each of the topics with the Ph.D. Committee. The goal of MCBA is not to be able to demonstrate deep expertise in the specified areas, but rather to show a basic familiarity with and a general understanding of these topics. In terms of the revised Blooms Taxonomy, the student should be able to demonstrate at least Cognitive Level 2 (Understanding) in all these areas. Based on the evaluation with the Mentor, a plan of study will be developed to ensure that the student can gain the knowledge needed to demonstrate the expected level of understanding in all of the core areas. This plan is presented to the MOVES Ph.D. Committee Chair for approval. This plan can include taking classes, directed study with faculty or reading various reference materials. As part of this preparation process the student should prepare notes and other materials that they will be able to use during the assessment with the Ph.D. Committee.

The MCBA will be held at a meeting of the MOVES Ph.D. Committee with the student where the student will discuss the foundational breadth topics with the committee. A minimum of four members of the MOVES Ph.D. Committee must participate in the meeting. Any subject on the topic list the student was provided at the start of the program can be a topic of discussion. The student will be allowed to bring the notes and material they prepared and use them during the assessment discussions but will not be allowed access to other resources. The focus of the assessment will be on understanding the material more than on recalling facts. If a second MCBA is required, the topics during that assessment will be limited to the areas that were identified as deficiencies during the first assessment. The MCBA meeting will be closed and only the Ph.D. Committee members and the student will attend. The student can, if they choose to, have a faculty member of their choice who is not on the Ph.D. Committee attend the meeting as an observer.

The main topic areas of the MCBA are:

1. Ph.D. research basics
2. M&S Foundations
3. HF/HSI foundational principles
4. General Statistical knowledge
5. Augmented Reality/Virtual Reality/Mixed Reality
6. Applications of M&S
7. Behavior representation and Artificial Intelligence
8. Interoperability
9. Software development processes

A full detailed list of questions/topics will be provided to the student after the start of their program.

While the MOVES Ph.D. committee is responsible for overseeing the successful completion of the student’s program of study through the passing of the Oral Qualifying Examination, the day-to-day management of the student's work will be assigned to a Faculty Mentor who will work directly with the student. The Mentor may end up being the Dissertation Supervisor and/or Chair but does not necessarily need to be. The most important requirement for the mentor will be to understand NPS administrative and academic processes/policies and the MOVES Ph.D. policies to be able to help the student with these issues and to ensure the student stays on track to complete their program of study. The mentor will report regularly on the student’s progress to the MOVES Ph.D. Committee. Quarterly if progress is on track and more often if issues come up.

The Mentor supports the student’s academic work in three important ways. First, the Mentor helps the student pick the topics for the Written Qualifying Exam that they are planning to take. Second, the Mentor helps the student determine how best to prepare for the MCBA. Finally, the Mentor works with the student to define the specifics of the student’s program of study that will be informed by the preparation needed for the MCBA and the Written Exam topics chosen.

The role of the Mentor is important to ensuring the student’s success, so ideally a mentor should be identified and assigned in the first two weeks after the student starts the program. The Mentor will play a critical role in helping the student prepare for the MBQA and in selecting the Written Qualifying Exam topics, so it is essential to be able to assign a Mentor as soon as possible. It is recommended that the student meet with MOVES faculty as soon as they can to find someone who is willing to be their Mentor. If the student cannot find someone to be their Mentor in the first two weeks after starting the program, the Ph.D. Committee will assign a Mentor to the student. The Ph.D. Committee needs to approve any change of Mentor.

During a student's initial year of work, an essential activity is learning about the many academic research efforts occurring across the MOVES faculty. Working with faculty in classes or directed studies is excellent preparation for understanding which faculty members might best guide novel research as part of a student's dissertation committee. Knowing about faculty interests and expertise provides direct benefit when considering topics of original, novel research.

Although formal finalization is not required by NPS until later in the process, forming a dissertation committee is an essential activity for every Ph.D. student. Each student is expected to begin carefully forming a dissertation committee to oversee their research program promptly after beginning the Ph.D. program, and in any event, not later than the end of the first year of studies. The dissertation committee is responsible for supervising candidates' completion of their dissertation research, and production of the dissertation document. The dissertation committee is also responsible for administering and determining the results of the final dissertation defense (final oral examination). Once the OQE is passed, responsibility for guiding the student’s work passes from the Faculty Mentor to the Dissertation Committee.

The committee is chosen by the Ph.D. student and, following acceptance by proposed committee members, subsequently approved by the MOVES Ph.D. program committee and the NPS Academic Council. Any necessary subsequent changes to the dissertation committee require the same chain of approval.

One of the committee members from the MOVES curriculum faculty must be designated as the dissertation supervisor and serves as the student’s primary technical contact. The supervisor should be knowledgeable about the area of the proposed dissertation and will have prior experience serving on dissertation committees. The student should therefore choose the general area for the proposed dissertation prior to forming the committee. To serve as a Dissertation supervisor, a faculty member should have previously served on at least two other NPS Ph.D. dissertation committees as a voting member.

Each committee must also have a chair, who may be the same as the dissertation supervisor. The role of the chair is primarily administrative to manage and lead the committee so that the working of the committee is consistent with NPS academic policies and procedures. The Dissertation Committee Chair should have served on at least four previous NPS Ph.D. dissertation committees as a voting member.

The dissertation committee must have at least three members of the MOVES curriculum faculty. Committee composition must also include at least one faculty member from outside the MOVES curriculum; this means that the member cannot hold an appointment of any type in MOVES. At least five members are required, of which one may be from another appropriate academic institution or hold domain-relevant expertise. At least four members of the committee must have earned doctorates. While the Dissertation Supervisor will take the lead on providing technical guidance for the student’s research, it is expected that the other committee members will take an active role in guiding and advising the student as the research is being conducted and while the dissertation is being written. 

Serving on a Ph.D. dissertation committee is a significant commitment to the student by the faculty member to provide the requested support in a timely manner. By agreeing to serve on a student’s MOVES Ph.D. committee, the faculty members agrees to provide prompt feedback on all reasonable requests from the student. If the faculty member does not feel they can honor such a commitment due to other obligations, they should not agree to be on the dissertation committee. Similarly, the student is responsible for requesting feedback and decisions with a reasonable time horizon. As a rule of thumb, the student should give a minimum of 2 weeks for committee members to respond to chapter review requests and at least three days for other requests.

When selecting the members of the Dissertation Committee the student should take into consideration the faculty members’ areas of expertise and how those areas relate to the proposed dissertation work. Choosing committee members with relevant expertise will provide a broader resource base to help the student in conducting their research.

Every MOVES Ph.D. student is expected to maintain regular contact with the members of their committee regarding progress towards the degree. The student should plan on meeting with their Dissertation Supervisor at minimum on a weekly basis. It is strongly recommended that the student schedule regular face to face status meetings with the entire committee – at least on a quarterly basis and more frequently as the work approaches completion.

The purpose of the written qualifying examination is to check the student's analytical skills and ability to solve problems in the research area. These abilities are critical for success in the Ph.D. dissertation. In contrast to the MOVES Core Requirements, these exams focus on having the student demonstrate a depth of understanding and reasoning in each chosen topic area. In the Revised Blooms Taxonomy, the expectation is to demonstrate Cognitive Domain Level 5 (Evaluating). The specific exams areas should be chosen based on the expected area of research for the dissertation work. The student gets at most two chances to pass (see Naval Postgraduate School - 5.4.8 Written Qualifying Examination). The student should work with their Faculty Mentor in choosing the exam areas. This should be done as early as possible since the details of the student’s program of study will depend on the topics chosen. Ideally, this should be done while preparing for the MCBA.

The exam will typically be completed at the end of the first year of the doctoral program. Questions will come from the candidate's three major areas chosen from the list below and agreed upon by the student, the Faculty Mentor, and the MOVES Ph.D. Committee. The topic areas should be identified as early as possible after starting the degree since what is selected will affect the program of study.

• Networked and Web-Based Simulation: Scalable integration of interactive three-dimensional graphics, distributed modeling, local-area and wide-area networking, unicast versus multicast routing, integrated multimodal environments, network protocols. Live, Virtual, Constructive Simulation, and applications of virtual reality.
• Discrete Event Modeling and Optimization: Methods for simulating phenomena best conceptualized as sequences of distinct transitions between states described by discrete variables. Such simulations can be run on single, parallel, or distributed computer systems; can be run either in real time or asynchronously with real time; and can be deterministic or stochastic. Also includes the study of inferences from experiments on such simulations.
• Simulation Software Development: Software engineering principles applied to the development and acquisition of simulation systems of all varieties. Software requirements specification, design and architecture, implementation, and deployment. Software configuration management, lifecycle maintenance and upgrading. Software testing and quality assurance issues. Project management and scheduling, risk management.

• Training Systems and Human Factors: Cognition and perception, human information processing model, task analysis, learning theories and principles, multimodal interfaces, spatial orientation and navigation, interaction techniques, interaction devices, virtual ergonomics, health and safety issues including cybersickness, usability engineering, performance evaluation, immersion, presence and co-presence, representation of the virtual humans, multiuser environments, evaluation of training effectiveness, team training, transfer of training, game-based systems, adoption of technology, and related Human Systems Integration (HSI) topics.
• Agents and Cognitive Modeling: Software tools to help people accomplish tasks involving finding, access, extraction, analysis, and summarization of information. Techniques are drawn from software engineering, artificial intelligence, information retrieval, and exploratory data analysis. Such tools may be used to build models and simulations, or may be embedded in artificial players within a simulation. Integrative architectures for modeling of individuals, including neural networks; rule-based systems, attention and multitasking phenomena, memory and learning, human decision-making, situation awareness, planning, behavior moderators, modeling of behavior of organizational units, modeling of military operations, machine perception and reasoning, understanding with language models and knowledge graphs, and modeling of information warfare.
• Combat Modeling and Analysis: Design and application of military modeling, simulation, wargaming and analysis to include: taxonomies of models; hierarchies of models; characteristics of models required to meet the needs of acquisition, test and evaluation (T&E), training, wargaming, analysis, or experimentation communities; mathematical models for search and acquisition, probability of hit (PH) and probability of kill (PK) models; attack and combat adjudication; characteristics of aggregate, entity-based, and semi-autonomous force models; stochastic versus deterministic Lanchester-type formulations; measures of effectiveness; measures of performance; approaches to effectively using models to assist decision-making; modeling decision-making; simulating C4ISR processes; terrain and movement algorithms; Verification, Validation, and Accreditation (VV&A); aggregation and disaggregation; variable resolution modeling; representation of human behavior (e.g. suppression, unit breakpoints); approaches to modeling joint warfare; data support for models; logistics considerations; C31 process modeling; modeling of behavior of organizational units in military operations; information warfare modeling; and use of artificial intelligence. Knowledge of current DOD models, including their theoretical and scientific foundations as well as strengths and weaknesses, is also expected.
• Systems Engineering: Systems Engineering includes interdisciplinary engineering management that focuses on how to design, integrate, and manage complex systems over their life cycles. Additional emphases include process models, requirements generation, M&S support in cost estimation, test and evaluation (T&E), and analysis of alternatives/decision making. Relevant modeling and simulation support techniques include analysis of interactions between multiple engineering domains, risk-benefit analysis, work-process optimization, requirements generation, cost estimation, analysis of alternatives and decision making, and test and evaluations. Emphases include Model-Based Systems Engineering (MBSE), Digital Twin, design thinking, developmental and environmental testing managing complexity and other advanced techniques improve synergies between MOVES and effective practice in other disciplines.

The written qualifying examination must be on material fundamental to the expected areas of research. Typically, this will involve mathematical questions, programming questions, and questions requiring demonstration of analytical abilities. This examination will be prepared by one or more faculty members who are subject matter experts in the chosen areas and will be reviewed by the MOVES Ph.D. committee before the exam is passed to the student. The student’s answers are evaluated by at least two faculty members with expertise in the area. After the assessment is completed, the results are passed to the Ph.D. program committee who make the final assessment and inform the student of the results.

Within no more than one year after the successful completion of the written qualifying exam, the student must successfully complete the oral qualifying examination. All courses in the study plan must be completed before the student can take the oral exam. The student gets at most two chances to pass (see Naval Postgraduate School - 5.4.9 Oral Qualifying Examination). A portion of the oral qualifying examination is open to the public.

The oral qualifying examination is overseen by the MOVES Ph.D. program committee by appointing the examining committee. This committee will have one member from the MOVES Ph.D. program committee who serves as the committee chair. There should be some members who were involved with administering the written exam. There should also be one member from the proposed dissertation committee – preferable the faculty who will be the Dissertation Supervisor. In most cases, it is expected that some members of the committee will fulfill multiple requirements. The committee needs to have a minimum of five members. If additional members are needed, they should be selected from the proposed Dissertation Committee. In addition to the examining committee members, the Academic Council sends a representative to attend the exam to ensure the examination is conducted in accordance with NPS policy.

A draft dissertation proposal should be submitted to the Oral Qualifying Exam Committee at least one week before the oral qualifying examination. The purpose of presenting the dissertation proposal is to provide the committee with the information needed to help guide the questioning to best assess if the student is sufficiently prepared to pursue their area of study. The exam questions should not be tied to any specific aspects of the dissertation proposal.

The committee chair schedules the oral portion of the qualifying exam. The oral examination is the culmination of the course of study; the purpose is to test basic knowledge and creative ability, also demonstrating the student's capacity to thoughtfully apply concepts and material from their course of study.

Current MOVES Ph.D. students typically attend oral exams being taken by other Ph.D. students. Experience has shown that observing these events provides great insight. It is customary to provide at least one week's advance notice in order to encourage attendance at this important event. It is wise for each examinee to ask fellow Ph.D. students to take notes about questions and responses, helping them to "stay in the moment" and best focus on matters at hand.

Oral examination committees normally plan for at least a three-hour session, providing each prospective candidate with plenty of time for responding to questions and also sufficient time for faculty discussion assessing their performance.

The committee asks questions regarding the student's major areas of study pursued during the written exams and any other questions that members think may help decide whether the student has sufficiently broad knowledge of the relevant scholarly work and possesses sufficient analytic capabilities to begin full-time Ph.D. research. The focus of the questioning should be on topics relevant to the expected area of the dissertation research rather than on broader MOVES foundation topics since the breadth requirement has been met through the MCBA. Time permitting, other faculty members in attendance may also ask questions of the student. The questions may be on any reasonable topic.

When the committee is satisfied that the student has been questioned thoroughly and that they can render a pass/fail judgement on the student’s performance, the student, guests and other faculty members who do not have voting privileges for the oral qualifying examination leave the room. The committee members then discuss their concerns and vote whether to pass the student; a unanimous vote is required. The final overall decision regarding passing or failing the entire qualifying examination is made by the committee after the oral examination is completed.

A draft dissertation proposal should be written with help of the proposed Dissertation Supervisor (and committee members if they have been identified) prior to the oral examination. The purpose of the dissertation proposal is to provide the dissertation committee with the information needed to determine whether the proposed research topic is suitable for a Ph.D. dissertation. The proposal should describe the student's best current estimate of their research plan. With the supervisor's approval, the details in the proposal may be changed later as the research subject is understood in more detail. This proposal must establish that successful completion would make original and significant contributions to knowledge in the candidate's major areas of study.

Upon successfully completing the qualifying examinations and approval of the dissertation topic, the student must petition the academic council for "advancement to candidacy" for the doctorate. A memo must be prepared to state that the requirements for advancement to candidacy have been successfully completed. Notification of advancement to candidacy is provided in writing by the Academic council.

A minimum of six months after passing the oral qualifying exam, when the dissertation research is almost complete and a dissertation draft has been completed and is available, the final oral examination (also known as the dissertation defense) may be scheduled. It is expected that prior to scheduling the Dissertation Defense, the student will work with the Dissertation Committee Chair to coordinate with all committee members and have their agreement that the dissertation work is ready to be defended. No degree completion administrative process should be started until the full committee has agreed that the dissertation is ready to be defended.

It is customary to provide at least one week's advance notice prior to a dissertation defense, also making preliminary presentation materials available, in order to encourage attendance at this important public event. All MOVES Ph.D. students should make every effort to attend and observe each defense, offering constructive feedback to the defending student afterwards.

This examination is administered by the Dissertation committee and consists of the following:

1. Submission of the dissertation draft to the MOVES Ph.D. program committee at least one week before the date of the final oral examination.
2. An open (public) presentation of the findings of the research by the candidate, including response to questions from the audience within an allotted time period. It is the responsibility of the chair of the dissertation committee to ensure that all MOVES faculty are informed well in advance of the time and location of this examination. The entire Academic Council is invited to attend, and the same offer can be extended to faculty in other departments. All members of the dissertation committee are required to attend, and the Academic Council shall designate a representative, who must attend the dissertation defense.
3. In the final dissertation defense, the candidate presents the dissertation and is subject to such questions as the entire dissertation committee deems appropriate. The extent of participation of all parties is determined by the dissertation committee chair. The question and comment phase is typically open to attending faculty when time constraints permit.

A closed session is held to discuss the defense involving only the members of the student's committee and the academic council representative. A unanimous vote by the committee is required for a successful outcome.

Publishing a dissertation is a major event, both for the author and for the scientific community. When the dissertation has been revised and clarified to each committee member's satisfaction, based on originality, clarity of presentation, and advancement of fundamental knowledge, the student and committee members sign it. Academic Council procedures then apply for final disposition of the document.

Some doctoral programs at NPS have a minor requirement in a field other than that of the degree-granting department. For those doctoral students who wish to complete a Ph.D. minor in MOVES, this consists of:

1. Three courses at the 4000 level that form a coherent sequence relating to the field of modeling, virtual environments, and simulation. The selection and rationale for these courses require approval of the MOVES Ph.D. Committee prior to taking the courses.
2. The MOVES Ph.D. program committee chair provides a letter attesting that the student has fulfilled the requirements upon the request of the student.

Program Officer
Charles Rowan, LTC, USA, PhD
Watkins Hall, Room 274
(831) 656-7872, DSN 756-7872
charles.rowan@nps.edu
 
Academic Associate, MOVES
Chris Darken, Ph.D.
Code CS/Cd, Watkins Hall, Room 382
(831) 656-2095, DSN 756-2095
FAX (831) 656-7599
cjdarken@nps.edu
 
Chair, MOVES Ph.D. Program Committee
Imre Balogh, Ph.D.
Research Associate Professor
(831) 656-2865
ilbalogh@nps.edu