In the old test/quiz, the fill-in-the-blank question type allowed us to have a word bank, which then allowed basically the functionality of the new assessment fill-in-the-blank drag-and-drop question. The test/quiz would then randomize the order of the words in the word bank. However, the new assessment question type doesn't do that randomizing - the teacher has to put the answers (including distractor answers) in random order manually. If there are just one or two blanks in a question, that isn't too hard. But if there are a number of blanks in a question, the work flow is generally like this: Type the question, with the underscores for answers, then go below and enter the correct answers in the "possible responses" area (generally in the same order of the answers), then put them in the correct blanks in the "Correct answer setup." But I then have to go back and manually randomize the correct answers, or they show up for students in the order of the blanks in the question stem. We should have the ability (maybe with a radio button option) to automatically randomize those answers.
I have contacted Schoology support about this, which I discovered when I used the conversion tool from test/quiz to assessment - all the answers were in order. Schoology support said that is the way the question type is designed - it does not randomize the answers. But I think it should at least be an option for the teacher.