Quarterly Questions Program Specs

FAQs / Program Specifications

What is Quarterly Questions?

Quarterly questions is exactly what it sounds like--questions delivered on a quarterly basis--for a total of 50 questions in 2017. The pilot adopts new methods of assessing cognitive knowledge and is being tested as an alternative to the current proctored recertification examination required once every 10 years.

Will the pilot, if successful, replace the current DOCK Examination?

Yes. Data from the 2017 pilot will be used to make decisions about the future of the pilot and the current DOCK Examination.

Who is eligible to participate?

Diplomates who are actively meeting MOC requirements.

I am a permanent certification holder. Can I participate in the pilot?

Yes, if you enroll in MOC.  

Is there a subspecialty pilot?

Not yet. The 2017 pilot is for fundamental (“core”) knowledge useful for all ophthalmologists. If the pilot is successful, it will be expanded to include subspecialty topics.

Is there a fee for the pilot?

No. Eligible diplomates will not be charged additional fees to participate.

What if I do not want to participate in the pilot and would rather take the examination?

You can. The 2017 pilot is voluntary and participation is not required. Diplomates can still choose to take the DOCK examination to maintain certification. However, if the pilot does replace the DOCK, you may participate in the new program at the beginning of your next certification cycle, regardless of when you took your last proctored exam.

Will the pilot continue in 2018, and what will the program look like in the future?

The 2018 version of the pilot will depend on how the program performs in 2017. We anticipate that the 2017 pilot will be successful and that the program will continue in 2018, but we also intend to make improvements based on feedback we receive in 2017. Regardless, as long as the pilot continues in 2018, participants in the 2017 pilot will be allowed to continue participating in 2018 if they have "meaningfully engaged" in the program.

What are the meaningful participation requirements in the pilot?

There are three requirements a diplomate must meet in 2017 to allow the pilot to count as the Core Ophthalmic Knowledge module of the DOCK examination and to continue participation in subsequent years:

  • Answer all 50 questions;
  • Achieve roughly 40% correct on all questions; and,
  • Complete a brief survey at the end of each quarter.

How do I register for the pilot?

Registration is not necessary. Eligible diplomates will be sent an email in early 2017 notifying them that the pilot option is available.

How long is the registration period?

There no formal registration period. All eligible diplomates will be able to participate via their ABO Status Page throughout the year. The ABO encourages diplomates to begin participation at the beginning of the year and to answer the questions as they become available at the start of each quarter. 

Is there an incentive to participate in the pilot?

Yes. So long as diplomates continue to meet the pilot's meaningful engagement requirements, they will not have to take the Core module of the DOCK examination.

Are there any other incentives for participating in the pilot?

No. Participating in the pilot in lieu of the Core module of the DOCK examination is the only incentive that will be offered.

What happens at the end of 2017?

Assuming the pilot continues, if a diplomate meets the 2017 meaningful engagement requirements, there will be an option to continue the pilot in 2018.

Will the 2017 meaningful engagement requirements change in subsequent years (i.e., 2018 and beyond)?

Possibly. At this time, the meaningful engagement requirements for future versions of the pilot (i.e., in 2018 and beyond) have not been determined. If the pilot is adopted, we anticipate that the score for “passing” will be set consistent with other ABO exams.

What happens if I don’t meet the meaningful participation requirements in 2017?

Diplomates not meeting the meaningful engagement requirements will be required to take their Core Ophthalmic Knowledge module of the DOCK Examination in the year the exam is due. 

How many surveys will there be?

Participants will be asked to complete three surveys, one after each quarter (Q2, Q3, and Q4). The three surveys will be approximately 5-10 minutes in length.

How will I receive the surveys?

An email with a link to the survey will be sent to participating diplomates. 

What if I start the pilot, but do not finish?

The 2017 pilot is voluntary, and a diplomate may choose to stop participating at any time. However, if the 2017 meaningful engagement requirements are not met, diplomates will be required to pass the Core Ophthalmic Knowledge module of the continuous certification examination if they wish to maintain certification.

If I register for the pilot and want to opt out during the pilot, can I do so? What is the process?

The pilot is completely voluntary. Registered diplomates may opt out of the pilot but will then have to pass the Core Ophthalmic Knowledge module of the continuous certification examination. If diplomates wish to opt out, they can do so with a link provided once the pilot is started or can simply not answer the questions by the end of the year.

Will I receive CME credits?

CME credit is not available for participating in the 2017 pilot.

What are the system requirements for doing the pilot questions on a desktop computer?

The pilot platform will be compatible with most common browsers including Google Chrome, Mozilla, and Internet Explorer. Diplomates will also need a reliable Internet connection.

Can the questions be answered on a mobile device?

Yes. The platform will be mobile friendly, but will not have a dedicated app at this time. 

How many questions do I have to answer?

Diplomates will answer a total of 50 questions. 20 questions will be delivered in the second and third quarters and 10 journal-based questions will be delivered in the fourth quarter. The pilot questions focus on 1.) fundamental knowledge needed in everyday practice of ophthalmology (40 questions); and 2.) the application of knowledge from up to five self-selected journal articles from a list of 15 options (10 questions). 

Pilot participants will select and read five journal articles from a list of 15 articles. The articles will be available at the beginning of the year and the ten article–related questions will be delivered in the fourth quarter (Oct. 1) The 40 fundamental knowledge questions should not need additional preparation. However, a content outline is available.

Questions not answered will be scored as incorrect. 

How often will I receive the questions?

Twenty questions will be available on the first day of each quarter, starting with Q2 (April 1). Diplomates are encouraged to answer questions each quarter, but will have until December 31.

Do I have to answer all questions at once?

No. Questions can be answered at a diplomate’s convenience so long as all 50 questions are answered by the end of the year. Questions may be answered one at a time, in small batches, or all in one sitting.

Where can I answer questions?

Diplomates can answer questions using any computer or mobile device that has Internet access. This allows diplomates to answer questions at a time and place of their choosing.

How much time am I given for each individual question?

You will have 1 minute to answer each question.

Why only 1 minute?

Questions are designed to assess “walking around knowledge” that does not require calculations, peer collaboration, or the use of outside resources. Since one of the goals of this pilot to find an alternative to the closed-book examination, the 1-minute time limit best mimics the summative assessment environment that is designed to determine whether an individual possesses the knowledge required to answer each question without engaging in external assistance. 

Will question times vary?

Question times will be the same length during the 2017 pilot, but may vary in the future.

What happens if my time runs out on a specific question before I submit my answer?

If an answer is selected, that answer will count. If an answer is not selected, the question will be scored as incorrect.

Can I use resources to answer questions?

No. Questions do not require advance study, calculations, or the use of books or reference guides while answering. For article-based questions, thoroughly pre-reading your selected journal articles will serve as sufficient preparation.

Can I consult with my colleagues while answering a question?

No. At this time, the pilot is designed as independent activity.

Will the questions include graphics, videos, or audio recordings?

Yes. In 2017, you may see images and graphics in the questions. The ABO is exploring the addition of videos and audio recordings as future enhancements.

What are the journal articles?

15 journal articles are available as options for selection. Diplomates will select 5 articles and answer 10 (2 questions per article) knowledge application questions related to the article. 

What happens if I encounter technical difficulties while taking a question?

Diplomates who encounter technical difficulties should contact the ABO. ABO will work with diplomates on a case-by-case basis.

Can diplomates skip over a question and return to it later?

Not at this time. All questions are 1 minute in length, and once a diplomate starts a question, the timer begins.

Can I stop the timer during the question?

No. The timer does not stop once a question begins. However, questions may be answered one at a time, in small batches, or all in one setting based on diplomate preference.

Why am I being asked about confidence and relevance?

After answering each question, you will be asked to rate your level of confidence in selecting the correct answer and the relevance of the question to your practice. These ratings do not impact your score. However, we still need you to answer them honestly as they are important data points for evaluation and quality improvement. Eventually, the ABO hopes to provide you with information regarding what you answered and how confident you were in that knowledge. Relevance data will improve the quality and relevance of questions that are issued.

How can I go back to see questions I have previously answered?

Previously answered questions are available on the dashboard under a diplomate’s “Historical Questions” tab.

What happens if I do not answer my questions by the end of quarter?

Nothing. Questions not answered by the end of the quarter will be available until the end of the year.

Why is it important to do my best when answering pilot questions?

Your performance on the 2017 pilot questions will be used to determine whether you meet the 2017 meaningful engagement requirements. Collective diplomate performance will also impact decisions about the future of the Quarterly Questions program and the DOCK Examination.

How is the pilot scored?

All 50 questions will be scored. For 2017, diplomates must achieve a meaningful engagement score equivalent to roughly 40% of questions answered correctly.

What type of feedback will I get with regard to my performance?

Diplomates will see their raw score performance (i.e., total number of questions, number of questions answered correctly, and percentage of questions answered correctly) and performance as compared to peers. 

How will the passing standard be set?

The performance standard (i.e., the level of performance required to pass the exam) for the operational program will be set in early 2018 using pilot data from 2017.

How will I know when new questions are available?

At the beginning of each quarter, diplomates will be sent an email notification to the email address on record indicating that new questions are ready. 

How often will reminder notifications be sent?

Diplomates will be sent a notification when questions are first available at the beginning of the quarter as well as a reminder 30 days, 1 week, and 1 day before the end of the quarter if there are unanswered questions remaining. 

How do I let the ABO know about an issue or feedback I have for a particular question?

After answering a question and selecting confidence and relevance, diplomates will be taken to the question feedback page. At the bottom of this page there is a free text box where diplomates can provide feedback on the question.

How do I let the ABO know of feedback I have for the pilot?

There is a feedback page on the pilot platform where diplomates can provide general pilot feedback. Additionally, surveys will be sent intermittently throughout the pilot soliciting feedback. Diplomates also have the option of emailing the ABO at moc@abop.org to share their feedback or ask questions.

How do I get started?

Log into your Status Page and scroll to the Quarterly Questions information box for a link to begin selecting your journal articles.