4.8.2 AMM Scoring Methodology

In the proposed AMM, the set of accountability controls (cf., section 3.3) is complemented with a scoring methodology to (quantitatively) represent how well all of these controls have been implemented by the organisation under assessment. It is worth highlight the fact that there is no standard scoring methodology adopted by state of the art maturity models, in many cases even the semantic associated with the numeric output (and also the actual number of maturity levels) is different. In order to align the proposed AMM with relevant industrial practices, the rest of this section leverages the scoring methodology used by CSA CCM. A suitable procedure for assigning maturity levels based on a CCM assessment has been developed in the context of the Open Certification Framework (OCF). When an organisation is audited, a Management Capability Score (i.e., maturity level) will be assigned to each of the control areas on the CCM. For the sake of usability, the management capability of the domains (not the individual controls) is scored on a scale of 1-15. These scores have been divided into five different categories that describe the type of approach characteristic of each group of scores:

a) 1-3: No formal approach.

b) 4-6: Reactive approach.

c) 7-9: Proactive approach.

d) 10-12: Improvement-based approach.

e) 13-15: Optimising approach.

When assigning a score to a control domain, the following five factors are considered (all or any applicable combination of them):

1. Communication and Stakeholder Engagement.
2. Policies, Plans and Procedures, and a Systematic Approach.
3. Skills and Expertise.
4. Ownership, Leadership, and Management.
5. Monitoring and Measuring.

The lowest score against any one of those five factors will be the score awarded for the control domain. The organisation under evaluation will be awarded the lowest score it achieved for any of the factors assessed against the CCM domains. Once the assessor has assessed all of the control domains, there will be 16 scores (one per-domain of the CCM). The average score will be used to assign the overall Management Capability Score (or award) for the organisation, according to the following rules:

• If the organisation has an average score of less than 3, it will receive a certificate with no award.
• If the organisation has an average score between 3 and 6, it will receive a bronze award.
• If the organisation has an average score between 6 and 9, it will receive a silver award.
• If the organisation has an average score greater than 9, it will receive a gold award.

A typical (state of the art) maturity model would only implement the two elements discussed above (controls and scoring methodology); however, two limitations appear in relationship to (i) the subjectivity associated with the underlying assessment process, and (ii) the level of automation that could be achieved. In order to overcome these limitations, EU projects like CIRRUS ^[59] , CloudWatch ^[60] , SPECS ^[61] and Cumulus ^[62] have been looking at potential mechanisms to implement the continuous assessment of security/privacy in a semi-automated manner for cloud systems. A promising solution is based on the use of metrics, just as proposed also by A4Cloud. The next subsection elaborates the relationship between the AMM and the accountability metrics.