The proctoring system is designed to confirm the results of online assessments when passing assessments in Learning Management Systems (LMS). System allows:
There are no special technical requirements for the administrator's PC or laptop. You need just any updated web browser. It is recommended to use Chrome 72+, Opera 59+, or Edge 79+.
To enter the proctoring system, open its URL in your web browser and type in your login and password. To access the administrator interface, the user must be given the appropriate rights.
When logged in, the top panel of the interface contains language switching, full screen mode, settings and the exit buttons. There is the navigation menu on the left side. And there is a section selected in the menu in the central part. To avoid problems with time display, make sure that the time and time zone on your computer are set correctly.
Participants of assessments (test-takers) undergo proctoring sessions in accordance with the life cycle, the diagram of which is shown in below. It includes two logically separated blocks: preliminary steps and supervision. If there is no date and time specified in the "Remove at" field, the session is stored for the duration of the license agreement.
The session schedule table displays a list of all sessions or for a specific period of time. The following functions are available for the administrator:
When integrating with the LMS, sessions are created automatically, but the administrator can create a session manually or upload a session list from a CSV file. To change the parameters of an existing session, select it in the list and press the "Change" button in the toolbar or double-click on the selected session. The session card editing dialog will be opened. To create a new session manually, click the "Add" button on the toolbar. You can use the search field to display only the necessary sessions. You can use the syntax of regular expressions in your search. Here are some examples:
^123$ (where 123 is the required identifier, requesting an accurate match);student1 (where student1 is the login of the test-taker);^entrance (case-insensitive);exam.The search query can be formulated more precisely using the syntax of regular expressions. Search queries support the operators listed in the Table below. The list of the names of all supported fields can be seen in the data export dialog (via the “Export” button on the toolbar) or in Section 4.3. Date and time formats are in ISO-8601 format (YYYY-MM-DDThh:mm:ss).
Operators for search expressions
=status = template | status = [accepted,rejected]>=averages.b1 >= 50<=score <= 50>createdAt > 2019-12-01()startedAt > 2019-12-01 & (status = rejected | score < 50)<duration < 5&createdAt > 2019-12-01 & duration < 5|status = accepted | score > 90~*subject ~* ^exam~student.username ~ ^student\d+$!=conclusion != positive==members == [proctor1,proctor2]!==members !== []" " ' 'subject ~* "some name"On the "Parameters" tab of the session card, the following fields are available for editing:
identifier) — A unique identifier of the session, which is formed in the used LMS (allowed characters: A-Za-z0-9_-).provider) — Specifies the name of the integration provider for this session (for example: jwt, lti, plain, etc.).attempt) — The number of additional attempts to pass the session; i.e., if 2 is specified, then one main attempt and two additional attempts will be given, for a total of three.subject) — Name of the session for displaying in the proctoring system interface.student.username) — A user with the "student" role for whom the session has been created aka test-takeropenAt) — Date and time until which a test-taker will not be able to start a proctoring session. If not specified, it is not taken into account.closeAt) — Date and time of forced termination of the session. If not specified, it is not taken into account.lifetime) — Duration of the session in minutes after its launch; when the specified time is over, the session ends automatically. The force termination time cannot be greater than that specified in the "closeAt" field. If not specified or equal to 0, then it is unlimited.timeout) — Waiting time in minutes for the test-taker in case of disconnection from the session without its completion (closed the tab, was cut off the Internet); after the specified timeout, the session will be terminated automatically. If the test-taker has time to get back, the timeout is reset and the next time the test-taker is disconnected, the countdown starts over. If not specified or equal to 0, then it is unlimited.removeAt) — Date and time after which the session should be deleted. If not specified, it is not taken into account.expires) — Number of days from the start of the session, after which the session must be deleted (an alternative to the "removeAt" field).locale) — PDF protocol language, for example, "en".timezone) — PDF protocol time zone, for example, "Europe/Berlin".rules) — Page address with the rules of the assessment, which is displayed when the "rules" add-on is enabled. There are the following example web pages:url) — Assessment page address that will be opened in an IFRAME.code) — Access code (password) for a quiz in the LMS. This code is automatically injected into the page specified in the IFRAME URL field.api) — API address of the LMS to send the session results.tags) — Comma-separated list of keywords that can be searched in the session schedule table.A participant can go through each proctoring session only once; i.e., if the session has been completed, it cannot be continued. If for some reason you need to continue a completed session, then you can change the "deadline" field, indicating a time later than the current one.
Also, in addition to the session card, there are parameters that are filled in by the system during its operation:
template) — Template identifier that was used while creating the session.duration) — Actual duration of the session in minutes.createdAt) — Session creation date and time.startedAt) — Session start date and time.stoppedAt) — Session end date and time.signedAt) — Date and time when the conclusion was made by the proctor.status) — Session status:template — Session template;created — Created, but not started yet;started — Started, in progress;paused — Paused, the participant has disconnected (no connection for 2-3 minutes), but they are able to get back within the specified timeout;skipped — Skipped, was not started before the deadline;stopped — Completed, but not rated by the proctor;accepted — Positively rated by the proctor;rejected — Negatively rated by the proctor;removed — Removed, leaving only the session information.score) — Credibility score of the session.proctor) — Login of the proctor who made the conclusion.incidents) — Number of incidents sent by proctors in chat during the session.conclusion) — Conclusion of the session credibility:positive — Positive;negative) — Negative;null — No conclusion.comment) — Proctor's comment, left when making the conclusion.ipaddress) — Test-taker's IP address at the moment the session was started.useragent) — Contains the browser (application) name and its version, along with information about the computer's operating system.error) — Contains the text of the error in case it occurs.On the "Addons" tab of the session card, the following fields are available for selection:
rules) — Includes the stage of displaying the rules of the assesment and requires participants to agree with them.check) — Includes the stage of checking equipment; the list of checks is formed based on the selected indicators, including checking the browser, network, camera, microphone, and screen.profile) — Includes the stage of filling in the full name by the participant themselves.face) — Includes the stage of photographing the face by the test-taker. The taken photo is compared with the photo of the face from the participant’s profile. The result of the comparison is either saved in the session card or blocks the transition to the next step in case of a mismatch when the "verify" add-on is enabled.passport) — Includes the step of photographing an identity document by the test-taker. The taken photo is compared with the photo of the ID from the participant’s profile. The result of the comparison is either saved in the session card or blocks the transition to the next step in case of a mismatch when the "verify" add-on is enabled.scan) — Includes the step of uploading the scan of the identifying document by the test-taker. The taken photo is compared with the photo of the scan from the participant’s profile. The result of the comparison is either saved in the session card or blocks the transition to the next step in case of a mismatch when the "verify" add-on is enabled.verify) — In case verification fails (by face or document at the “face” or “passport” / “upload” stages), do not proceed to the next step until the problem is resolved.extrapages) — Includes the ability to add additional pages at the stage of photographing or uploading a scan of an identity document. If you want to upload a document photo without any checks, you can enable the “extrapages” add-on and disable the “passport” and “scan” add-ons.name) — Performs verification of the name inputted by a test-taker and the name in the scanned/photographed document. In case verification fails, the test-taker cannot proceed with the test.qrcode) — Includes the step of connecting an additional smartphone camera using a QR code.ready) — Turns on a dialog before starting a session, where the possibility of starting a session and confirmation of readiness by the participant is checked.proctor) — Allows checking proctor availability; the session will start only if the proctors assigned to the session have been online within the last 60 minutes (you should also set up invites).schedule) — Enables checking the availability of the assessment planning stage, where the participant must select the date and time of the assessment from the free slots indicated in the list of proctors (the "invites" field).track) — Starts tracking indicators in the session.record) — Saves the video recording of the session.mobile) — Gives the opportunity to pass the test with proctoring on mobile devices.overview) — Includes the stage of recording an overview of the room before the start of the session.screen) — Allows participants to use the video from the screen.webrtc) — Activates the functionality check of WebRTC at the step of the equipment check.seb) — This add-on lets a test-taker take a test only through the Safe Exam Browser installed on their computer. See more here.content) — Enables protection against copying content (test) in a session:preview) — Displays the preview in the form of a circle with a camera’s image of the test-taker and a toolbar, allowing test-takers to receive feedback from the proctoring system and adjust their behavior.header) — Displays a panel with title and control buttons in IFRAME mode.chat) — Allows the test-taker to open a chat with a proctor by clicking on the preview.calculator) — Allows the participant to open the calculator from the toolbar menu.upload) — Gives an opportunity to a test-taker to upload files to the chat.lock) — Activates the freeze of the screen when the system detects violations (depends on the threshold parameter), which have a duration of more than 20 seconds; the block deactivates 10 seconds after violations are addressed.shared) — Makes the session available by a link without authorization at “/api/report”.finish) — Shows the “Finish” button in thetest-taker’s interface.auto) — Allows the proctoring system to set the final conclusion for the session based on confidence assessment.xapi) — Turns on support for displaying xAPI content in IFRAMEs.
On the “Metrics” tab of the session card, the following metrics are available for selection:
b1)b2)b3)c1)c2)c3)c4)c5)c6)m1)m2)n1)n2)n3)s1)s2)You must also specify the Threshold (threshold) parameter, which is the threshold value for notifications to the test-taker and proctor, trust score for automatic conclusion, and triggering of the “lock” function. Usually, the threshold is set to 50. If the score is less than the threshold value, then violations are counted.
Each metric has its own weight, which is equal to 1 by default. This weight is used to form a confidence rating. The fixed indicators in the session are multiplied by it. The formation of a confidence assessment will be described below.
On the “Members” tab of the session card, a list of proctor logins that will have access to this session is indicated. The proctors specified in this list will see all the sessions assigned to them in their interface. You can add either the user's login or his group to the list of observers. User groups are conveniently used to organize access to sessions to a large number of participants without the need for each session to specify this large list. The order in which participants are added is not important. Administrators see all sessions; adding them to the list of participants is not required.
On the “Invites” tab of the session card, a list of proctors to be added to the session participants according to some predefined rules is indicated. Invitations can be used when it is necessary to organize the distribution of proctors by session so that each proctor sees only their own part of the sessions. As a list of proctors, as well as in the list of observers, their logins are indicated. However, here you can specify the rules for appointing proctors to a session; these rules will be applied to the specified list, and the proctors selected by these rules will be added to the observers in the session. Proctors are distributed equally between the test-taker. If proctors are assigned by a schedule (the “schedule” add-on is enabled), then the selection of a proctor and their moving to the list of proctors (the “members” field) occurs not at the time the session is started, but at the moment a participant selects the date and time.
The following assignment rules are supported:
quorum) — A minimum number of proctors for the session. If the quorum is greater than zero, the specified number of proctors is required to start the session. Prior to version 4.0, this parameter was called "quota".concurrent) — A limited number of simultaneous performances per proctor. If "0" is specified, it is unlimited.planAfter) — Limits the planning range of the assessment on the part of the participant; no time slots will be offered before the specified date and time.planBefore) — Limits the planning range of the assessment on the part of the participant; no time slots will be offered after the specified date and time.Registering the sessions through the external source through the integration it is possible to set up the default parameters. These will be displayed for the units, which were not transferred through the integration. There is a special type of session named “Template”. The sessions with this status could be created by the admin when the session does not have a “student” role account attached. Each template has its own ID, which should be transmitted later when registering sessions through the API in the field “template”. For the created sessions using templates there is a field “template” inside the card of the session. By default, if there was not a particular template, the system will use the template with the ID “default” when registering new sessions. It is convenient to use templates for the identification of some default parameters, which could be needed for the different categories of the tests. It is enough to create all the necessary templates in advance. From the LMS side it is only needed to give one the templates’ ID and for the new session the parameters listed in this template will apply.
The session card and PDF report include a table of averaged metric values with the session duration divided into 12 equal intervals. The table contains two parts, separated by a column with the name of metrics. The main part displays the average values of the metrics for each of the 12 time intervals. The last column shows the average values of each metric for the entire session. Each metric is calculated with its weight and the value limited from 0 to 100. Take a look at the table example:
Legend:
The credibility of the assessment results is calculated using the following formula:
$$E=100-\sum_{k\in M}^{}(W_kX_k)$$
where $$E\in[0,100]$$ is the credibility score (if E < 0, then E = 0), $$X_k$$ is the session-averaged value of the metric “k”, $$W_k$$ is the weighting coefficient of the metric “k”, and $$M \in { b_1, b_2, c_1, c_2, \ldots }$$ are the metrics. Thus, the credibility score shows the total average percentage of violations for the entire session; the duration of the session is taken as the actual one, i.e., the difference between when the test-taker started the session and when they finished it.
Calculation Example
For instance, you have switched on three metrics (b2, c2, and m2). All the other metrics are switched off. The weights of metrics: $$W_{b2}=1.5$$, $$W_{c2}=1$$, $$W_{m2}=0.5$$
Session duration is equal to 300 seconds. The system estimated a credibility score of 40 with the averaged score of metrics: $$\overline{b2}=30$$, $$\overline{c2}=20$$, $$\overline{m2}=10$$
You should interpret this as follows:
$$100-(\overline{b2}+\overline{c2}+\overline{m2}) = 100-(30+20+10)=40$$
b2 metric was detected during 60 seconds:$$\frac{\overline{b2}}{W_{b2}}\times \frac{\textrm{duration}}{100}=\frac{30}{1.5}\times\frac{300}{100}=60$$
c2 metric was detected during 60 seconds:$$\frac{\overline{c2}}{W_{c2}}\times \frac{\textrm{duration}}{100}=\frac{20}{1}\times\frac{300}{100}=60$$
m2 metric was detected during 60 seconds:$$\frac{\overline{m2}}{W_{m2}}\times \frac{\textrm{duration}}{100}=\frac{10}{0.5}\times\frac{300}{100}=60$$
In sum, the violation was recorded during 180 seconds out of 5 minutes (300 seconds), which is equal to 60% of the session duration.
The proctoring system recognizes each participant's face and compares it to that participant's profile and the profiles of other participants. Before starting a session, the user takes a photo of his/her face, which is matched against the user's profile. Throughout the session, the system periodically takes pictures of the participant's face and checks it in three ways:
Look at these photos:
There can be two states here, when a violation is found (red) and when all is well (green). The sources of the photo are as follows: (1) the current photo just taken; (2) the photo that was taken before this session started; (3) the photo that was saved in the user profile; (4) similar profiles if found.
The administrator can observe each session in real time. To do this, on the right side of the line of each ongoing session there is a “Supervision” button. When moving to observing the selected session, the interface opens as shown below. In the central part, thetest-taker’s video is displayed, in the right part there is a chat, and in the upper part of the video link button with the test-taker and the “Finish” button, which allows you to finish the session.
All audio, video, text messages, violations, and attachments are stored in the system and bound to the session protocol. Only proctors and administrators can view the protocols. Proctors can only view the protocols of those sessions in which they were participants. The session protocol includes:
At the bottom of the protocol, the session timeline is displayed, which consists of a timeline, the current playback position (vertical moving line), and three horizontal sections. The color (from green to red) highlights the minutes for which metrics in the proctoring system are collected. The greater the sum of the metrics for this minute, the more red the minute is highlighted. In the first section, events are displayed: the moment of photographing a person for verification of identity, chat messages, proctor notes. The second section displays the video tracks. The third section displays a pop-up histogram of metrics for each minute.
Moving along the timeline, you can find the violation moments and play back the video recordings of these sections. The histogram of average values of indicators can determine the most frequent and long-term violations.
Throughout the session, data is collected on the metrics listed in Section 4.5. Based on the collected metrics, the credibility score is calculated; the formula is given in Section 4.9.
Because data is stored per minute, the credibility is not set for sessions with a continuous duration of less than 1 minute. Identity verification and the c4 metric appear after the total presence of a face in front of the camera for at least 1 minute, provided that the quality of the photo allows recognition of the face with a biometrics subsystem.
The administrator, along with the proctor, can evaluate the sessions, indicating a positive or negative conclusion and commenting on it.
To call the conclusion dialog for the session, click the "Rate" button in the protocol interface.
In addition to the main online report, you can get a PDF version of the report, which can be downloaded using the “PDF” button. This is an abridged version, including a test-taker’s profile and photo, a session card, a map of the distribution of violations by time, as well as averaged indicators for the session. Pages 3 and above of this PDF can contain proctor notes with screenshots added via the session report interface. Images below show an example of such a protocol; the QR code contains a session identifier.
In some cases, there is a need to download all data for a particular session. This can be done with the “Download ZIP” button from the session table interface or from a specific session log. When downloading the archive, you can select which data to include. The following data can be selected:
The administrator can log in as a different user with the “student” role and the specified session. To do this, use the button in the session card.
You can create special links to access a specific session protocol. By these links, you can open the session protocol without logging in. To allow opening sessions using a link without authorization, you need to enable the "Make available by link (shared)" add-on for a specific session or in the template. In this case, it is recommended to use complex random session IDs, which are protected from brute-force attacks. The link to the protocol is formed as follows:
https://demo.proctoring.app/api/report/<identifier>
where demo.proctoring.app is the address of the proctoring system server, and <identifier> is the session identifier.
You can get the link from a session card with the button in the header
A list of all users of the system is available to the administrator. Users are either added automatically when configuring a connection with an external authorization provider, or are imported from a CSV file, or are added manually by the administrator. The following functions are available to the administrator:
When adding or changing a user account, it is required to fill in the required fields, which are marked with red asterisks.
username) — A unique user ID that is generated on the software side (allowed characters: A-Za-z0-9_-).provider) — Specifies the name of the integration provider for this user.password) — Password for logging in without integration with the software. The password must meet the following rules:role) — The user’s role in the proctoring system. It can take the following values: Student, Proctor, Administrator.nickname) — The name of the user displayed in the proctoring system interfaces.group) — An arbitrary identifier of the user's group, which indicates that the user belongs to a specific group. This allows connecting multiple proctors to the same group and adding this group to session participants. In this way, all proctors with this group, both old and newly created, will automatically become participants in sessions with this group.lang) — The language of the user interface; by default, the browser’s language is used.labels) — A comma-separated list of labels that can be used for searching.locked) — The blocked user will not be able to log in to the proctoring system.verified) — The flag comparing the photo from the camera and the photo on the identity document.
The administrator can view a photo of the user's face and the ID, or add/delete them. When you upload photos, the face is recognized on them, so you can only upload photos where there is only one face and it can be recognized.
If a similar profile was found based on the user's photo, the similar tab will be available, where you will see a list of similar profiles found. Please note that the list of faces is compiled at the time a photo is uploaded or updated to a user's profile and does not change even if new similar profiles are added.
Once the sessions are passed, test-taker profiles are rated if their sessions have been issued a conclusion. The rating can be used to rank test-takers according to the results of proctoring sessions they have completed. The rating is calculated using the following formula: $$R=\frac{(R_0+C)}{2}$$ where $$R\in[0,100]$$ — new rating, $$R_0$$ — previous rating (the first is $$R_0=C$$), C — proctor’s conclusion (100 — Positive, 0 — Negative).
In the Drafts interface, an administrator can create a list of predefined messages to communicate with participants. This list will be visible to all proctors in the chat interface. Using this list, proctors can respond more quickly to typical questions or send regular information to participants. Messages are bound to a template and can be grouped by a specified key. If no template is specified, such messages will be linked to sessions without a template.
You can add a new draft message or edit an existing draft message.
Each draft has the following parameters:
* (asterisk) — Displays in all sessions;In the proctor interface, the draft message list can be accessed via the button next to the chat text entry field.
In the “Files” interface, the administrator can upload static files of arbitrary types, which will be available without authorization via a unique link.
This functionality can be used to publish static HTML pages with tasks and tests, including in xAPI format. You can also upload pages with rules there that need to be displayed at the first stage before the start of each session to participants.
Uploaded files are available without authorization via unique links in the format /api/files/:path, where :path is the relative path to the file in the file manager.
In the “Scheduler” interface, the administrator can view, add and delete time slots for a specific proctor in 15-minute increments. If the schedule is filled, test-takers will be able to select slots designated by the proctor on the calendar, and the proctor will become an observer for those sessions.
The scheduler interface is presented in the form of a weekly calendar, where you can select an arbitrary number of time slots on any future day.
In the “Supervision” interface, the administrator can connect to any of the ongoing sessions, carry out selective group monitoring and view the reports of past sessions.
The “Statistics" interface displays the average number of active sessions and online users. You can specify the range of dates and times for the statistics you want to view. The top part of the interface displays the maximum values of the indicators that were recorded for the specified period of time,e.g. Active Sessions, Online Users, CPU and RAM workload. If no date is set, the interface will display data for the last hour, and the graphs will be updated every minute. You can set a date and time period in order to capture the statistics for a certain period. The time interval is calculated from the current time (input field "End date"), and the interval corresponds to one minute. If you do not change the end date, then every minute there will be a request for data from the server. If the date is set, then the schedules will not be updated. To view data for a longer period, you can increase the interval.
The Journal interface displays the log of changes in the proctoring system data. This function is designed to provide transparency and control over user actions in the system.
Changes in the “config”, “user”, “room”, “draft” models are saved in the log. If the change is related to a specific user (for example, changed session parameters or logged into the system), then the "actor" of this change is indicated. If the change is not directly related to the user, then "actor" is not specified (for example, when the session status changes). For example, you can find all changes for a specific session or user by copying their Internal ID from the Sessions tab or Users tab accordingly. Then, paste it into the filter editor on the Journal tab and select the "Record ID (record)" field
If you specifically need to find all the changes made by a user, you can use the "User ID (actor.id)" field. In the dropdown list of fields in the filter editor on the Journal tab, you can choose from numerous parameters to filter the data according to your needs. The log can be exported to CSV. And using the filter, you can select only the changes of interest from all.
If for some reason a proctoring session was accidentally terminated by a test-taker or proctor, it might be necessary to extend the session. To do this, it is enough to find it, proceed to editing, and change the openAt field so that the date and time in it exceed the current one. After that, you need to save the changes; the session status will change to "started", and the test-taker will be able to reconnect.
This can be done through a session template or by distributing invite parameters through the integration. To equally assign proctors from a predefined list to each new session, you need to go to the session template and go to the "Invitations" tab. You can add proctor logins to the list one by one or enter several at once, separating them with commas. Make sure that proctors with the listed logins have already been pre-registered in the system; otherwise, during the distribution, test-takers will be assigned to a non-existent proctor. Next, you need to configure the parameters of the equal distribution. Quorum is a parameter that determines how many proctors will be assigned to onetest-taker (session). For example, if the quorum is 1, only one proctor will be bound to each session, and if 2, no more than two proctors are allocated to one session. Concurrent is a parameter that limits the number of sessions per proctor. When using this parameter, there is a risk that some test-takers will be left without a proctor if the total number of concurrent sessions exceeds the number of proctors multiplied by the concurrent session limit.
The beginning of a session means a successful start of testing. Rechecking the hardware after returning to the previously left session would take extra time, so it is not conducted.
However, in this case, there is a risk that the participant returns to testing using equipment that is not supported by the proctoring system. To track this behavior, the system provides the metric b1 — Browser is not supported.
You can create special links to access a specific session protocol. By these links, you can open the session protocol without logging in. To allow opening sessions using a link without authorization, you need to enable the "Make available by link (shared) add-on for a specific session or in the template. In this case, it is recommended to use complex random session IDs, which are protected from attacks. The link to the protocol is formed as follows:
https://demo.proctoring.app/api/report/<identifier>
where demo.proctoring.app is the address of the proctoring system server, and <identifier> is the session identifier.
When the system is used without integration via IFRAME, it can be convenient to upload sessions and users in one package from a CSV file. To do this, you need to use the import function. The minimum set of columns in a CSV table is recommended as follows (in brackets are names of the fields that should be inserted as the first line in the file as a header):
identifier);subject);url);student.username).username);password);nickname).The CSV file must be saved in Unicode (UTF-8) encoding, with a "," separator (comma).
Next, this file must be imported in the "Users" and "Sessions" sections.
The above-mentioned steps result in a template file for filling in users and sessions. Now you need to fill in the information on users and sessions.
If sessions with identifier or users with username already exist in the system, then to update such sessions, you need to add a field — the internal identifier id.
Microsoft Excel, can read a UTF-8 encoded CSV file in a few simple steps:
," — comma.In order to export sessions, if it is necessary:
All sessions can be exported by clicking the "Export CSV" button without entering any filters.
