Summative Assessment

The FREMA Domain Definition gathered information about projects, standards, software and services, internationally that were concerned with on-line assessment. The resulting view of assessment is very broad, including systems that augment or facilitate traditional assessment processes. However, the most common scenario was one of Computer Aided Assessment (CAA): it concerns a lecturer or teacher who can set summative assessments on-line, so that they can be taken remotely, and possibly within a flexible timeframe. These assessments are then marked (automatically or manually) and marks and subsequent grades are generated and stored digitally. We call this the End-to-end Summative Assessment Scenario.

Use Case Diagram
Below is the use Case for this scenario. Broadly speaking it has three parts: The first models the authoring of the assignment (and potentially of the items within the assignment). The second represents the run-time system, including the assessment event itself. The last part models the post-assessment process of marking and grading. There is no clear distinction between the parts. For example, scheduling is part of authoring and the run-time, and feedback is part of the run-time and the marking/grading.



This usecase diagram contains the following use cases:

Is decomposed from::Summative Assessment (Artefact Assessment)

Service Descriptions
The figure below shows the final set of SRCs that the FREMA team derived after several iterations of our factoring and re-factoring process. The core services that we believe are needed to support this activity are shown within the large Summative End-to-end bubble, with services that may be used via collaborations around the outside. The core services are divided into the three parts identified within the use case earlier (authoring, run-time, and post-assessment), although this is purely to add clarity to the diagram and has no engineering consequences.

In our re-factoring process we identified a number of core services that seemed to be involved in many collaborations, these were Notify, Track and Metadata Tagger, these are shown in a separate layer at the bottom of the bubble. These services are still a core part of the summative scenario, which is why they have been detailed here, however they may also be useful in other scenarios. The other collaborations that lie around the outside of the main bubble seemed less important, but may well be core services for another scenario. We have tried to group these into likely areas, such as Grading and Previewing, but again this grouping is purely to add clarity.



Authoring


 * described by::Author Assessment Service v1.0
 * described by::Validate Assessment Service v1.0
 * described by::Author Item Service v1.0
 * described by::Validate Item Service v1.0 (not yet linked to a use case)

Runtime


 * described by::Assign Cohort Service v1.0
 * described by::Assign Assessment Service v1.0
 * described by::Schedule Assessment Service v1.0
 * described by::Take Assessment Service v1.0
 * described by::View Feedback Service v1.0

Marking


 * described by::Mark Assessment Service v1.0
 * described by::Grade Assessment Service v1.0
 * described by::Moderate Assessment Service v1.0

Common Layer


 * described by::Notify Service v1.0
 * described by::Track Service v1.0
 * described by::Metadata Tagger Service v1.0

FREMA is not the only reference model; other models exist that describe how services fulfill scenarios to do with other domains (e.g. curriculum details or course validation). Some of the services identified as collaborators here might belong in these models; we have made some suggestions in the diagram as to where they may belong. Others might be common services that appear in several reference models (such as Authenticate), or a domain that is used by many other reference models (such as Repositories). The SRCs give an impression of the granularity of services within the scenario and describe the individual capabilities of each; they are designed to promote reuse and understanding of the design, while retaining flexibility in the implementation.

Interaction Diagrams
The WS-I for Summative End-to-end Assessment is rather large, and so it has been split into three parts. The figure below shows the WS-I for the first part of the scenario, the authoring. The WS-I describes how the core services interact so no collaborations are shown; this is because at this level the services do not need to know how other services are implemented, merely that they fulfill their responsibilities. The WS-I effectively shows which services interact, and in what order, in order to make the scenario happen. State is not shown, because that is an implementation detail, and the data passed around is described verbally, but not formally, for the same reason.



The next two figures show the WS-Is for the run-time environment and post-assessment marking and grading. These are interwoven to a certain extent as Feedback is generated by the post-assessment services, but delivered by the run-time services.



The FREMA SRCs and WS-I diagrams are not intended to provide a complete description of interacting services; it is a reference model, and not an interface description or detailed process model. However, we would expect systems builders to be able to use the reference model to describe their particular implementations and to aid the construction of interoperable interfaces. Developers can use the SRCs to decide what responsibilities their service implementations will take, and the WS-Is to see what consequences this will mean for interfaces to other services.

Gap Analysis
The following table shows a Gap Analysis for the has::Summative Assessment (Artefact Assessment) Use Case. You can find out how to interpret this here.



Is decomposed from::Summative Assessment (Artefact Assessment) Service Description (full) Service Description (part)



Partially fulfills:: [[Is decomposed from::Summative Assessment (Artefact Assessment) ]] Software Providers Software Consumers



Fulfills:: [[Is decomposed from::Summative Assessment (Artefact Assessment) ]] Software Providers Software Consumers Software Implementers